Question

如何获取文件在hdfs中的路径

Answer 1

如何获取文件在hdfs中的路径
hadoop有提供相应的脚本去验证文件目录是否存在的：

-bash-3.2$ hadoop fs -help
...
-test -[defsz] <path>: Answer various questions about <path>, with result via exit status.
-d return 0 if <path> is a directory.
-e return 0 if <path> exists.
-f return 0 if <path> is a file.
-s return 0 if file <path> is greater than zero bytes in size.
-z return 0 if file <path> is zero bytes in size.
else, return 1.

测试的hdfs目录中：

-bash-3.2$ hadoop fs -ls /user/hive/warehouse/yhd_gmv_month
Found 3 items
drwxr-xr-x - deploy supergroup 0 2014-08-25 11:15 /user/hive/warehouse/yhd_gmv_month/ds=2014-08-24
drwxr-xr-x - deploy supergroup 0 2014-08-26 13:02 /user/hive/warehouse/yhd_gmv_month/ds=2014-08-25
drwxr-xr-x - deploy supergroup 0 2014-08-27 08:09 /user/hive/warehouse/yhd_gmv_month/ds=2014-08-26

检验昨天产生的目录是否产生的shell脚本：

yesterday=$(date -d '-1 day' '+%Y-%m-%d')
hadoop fs -test -e /user/hive/warehouse/yhd_gmv_month/ds=$yesterday
if [ $? -eq 0 ] ;then
echo 'exist'
else
echo 'Error! Directory is not exist'
fi

验证存在的输出结果如下：

-bash-3.2$ hadoop fs -test -e /user/hive/warehouse/yhd_gmv_month/ds=$yesterday
if [ $? -eq 0 ] ;then
echo 'exist'
else
echo 'Error! Directory is not exist Or Zero bytes in size'
fi
-bash-3.2$ if [ $? -eq 0 ] ;then
> echo 'exist'
> else
> echo 'Error! Directory is not exist Or Zero bytes in size'
> fi
exist