hive的表结构如何导出 100
我自己搭了个测试集群环境,用的是hadoop,以及其组件的最新版,想线上的清洗在我测试集群中跑,看看行不行,好为将来升级做准备,但是线上hive中的表很多,我不是要在我的...
我自己搭了个测试集群环境,用的是hadoop,以及其组件的最新版,想线上的清洗在我测试集群中跑,看看行不行,好为将来升级做准备,但是线上hive中的表很多,我不是要在我的hive中一个一个建表吧。。。有没有大神推荐个工具或者有好的办法的
展开
展开全部
由于hive 0.10之前没有 导出建表脚本的接口 ,就写了这样的脚本。
欢迎指正,留言,提供更好的方法。
cat tables|while read node
do
echo "node : $node"
/usr/local/hive/bin/hive -e "describe $node" >> $node.txt
awk '{print $1,$2","}' $node.txt>>$node.txt1
echo "#!/bin/bash " >>$node.sh
echo "source ../../config/set_env.conf" >>$node.sh
echo "\$HIVE << EOF " >>$node.sh
echo "drop table $node;" >>$node.sh
echo "CREATE EXTERNAL TABLE $node(" >>$node.sh
cat $node.txt1>>$node.sh
echo ")">>$node.sh
echo "ROW FORMAT DELIMITED FIELDS TERMINATED BY '\,' LINES TERMINATED BY '\n'">>$node.sh
echo "STORED AS textfile ">>$node.sh
echo "location '/user/hadoop/idap/base/';">>$node.sh
echo "alter table $node set serdeproperties('serialization.null.format' = '');">>$node.sh
#echo "LOAD DATA INPATH \"/user/hadoop/idap/base/$node.txt\" INTO TABLE $node; ">>$node.sh
echo "exit;">>$node.sh
echo "EOF">>$node.sh
echo "if [ $? -ne 0 ];then">>$node.sh
echo " exit 2">>$node.sh
echo "fi">>$node.sh
rm $node.txt
rm $node.txt1
echo "================"
done
欢迎指正,留言,提供更好的方法。
cat tables|while read node
do
echo "node : $node"
/usr/local/hive/bin/hive -e "describe $node" >> $node.txt
awk '{print $1,$2","}' $node.txt>>$node.txt1
echo "#!/bin/bash " >>$node.sh
echo "source ../../config/set_env.conf" >>$node.sh
echo "\$HIVE << EOF " >>$node.sh
echo "drop table $node;" >>$node.sh
echo "CREATE EXTERNAL TABLE $node(" >>$node.sh
cat $node.txt1>>$node.sh
echo ")">>$node.sh
echo "ROW FORMAT DELIMITED FIELDS TERMINATED BY '\,' LINES TERMINATED BY '\n'">>$node.sh
echo "STORED AS textfile ">>$node.sh
echo "location '/user/hadoop/idap/base/';">>$node.sh
echo "alter table $node set serdeproperties('serialization.null.format' = '');">>$node.sh
#echo "LOAD DATA INPATH \"/user/hadoop/idap/base/$node.txt\" INTO TABLE $node; ">>$node.sh
echo "exit;">>$node.sh
echo "EOF">>$node.sh
echo "if [ $? -ne 0 ];then">>$node.sh
echo " exit 2">>$node.sh
echo "fi">>$node.sh
rm $node.txt
rm $node.txt1
echo "================"
done
追问
这个在网上也看到过,是您写的么?您测试过么?
推荐律师服务:
若未解决您的问题,请您详细描述您的问题,通过百度律临进行免费专业咨询