HDFS Permissions

The Hadoop Distributed File System (HDFS) implements a permissions model for files and directories that shares much of the POSIX model. Each file and directory is associated with an owner and a group. The file or directory has separate permissions for the user that is the owner, for other users that are members of the group, and for all other users. For files, the r permission is required to read the file, and the w permission is required to write or append to the file. For directories, the r permission is required to list the contents of the directory, the w permission is required to create or delete files or directories, and the x permission is required to access a child of the directory.

This assignment will create a new user, assign a folder in HDFS for him to demonstrate the permission capabilities.


Add a Unix user

hadoop@gandhari:~$ sudo groupadd feeder
hadoop@gandhari:~$ sudo useradd -g feeder -m feeder
hadoop@gandhari:~$ sudo passwd feeder

Create a folder in HDFS and assign permissions

hadoop@gandhari:~$ hadoop fs -mkdir /feeder
hadoop@gandhari:~$ hadoop fs -chown -R feeder:feeder /feeder
hadoop@gandhari:~$ hadoop fs -ls /
Found 6 items
-rw-r--r--   1 hadoop supergroup       1749 2016-08-24 06:01 /data
drwxr-xr-x   - feeder feeder              0 2016-09-05 15:34 /feeder
drwxr-xr-x   - hadoop supergroup          0 2016-09-05 15:15 /hbase
drwxr-xr-x   - hadoop supergroup          0 2016-08-24 13:53 /pigdata
drwxrwx---   - hadoop supergroup          0 2016-08-24 16:14 /tmp
drwxr-xr-x   - hadoop supergroup          0 2016-08-24 13:56 /user

We need to enable the permissions in hdfs-site.xml

hadoop@gandhari:~$ vi etc/hadoop/hdfs-site.xml

After this change, we need to restart dfs daemon.

hadoop@gandhari:~$ stop-dfs.sh
hadoop@gandhari:~$ start-dfs.sh

Let’s test the permissions using another user kannan who does not have write permission to /data/feeder

kannan@gandhari:~$ /opt/hadoop/bin/hadoop fs -put javashine.xml /data/feeder
put: Permission denied: user=kannan, access=EXECUTE, inode="/data":hadoop:supergroup:-rw-r--r--

See you in another interesting post!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s