2 Ranger installation
2.1 Environment preparation
Ranger2.0 requires the corresponding Hadoop to be 3.x or above, Hive to be 3.x or above, and JDK to be 1.8 or above!
2.2 Install RangerAdmin
2.2.1 Database environment preparation
Create a database where Ranger stores data in the Mysql database:
mysql> create database ranger;
Create user:
mysql> grant all privileges on ranger.* to ranger@'%' identified by 'ranger';
2.2.2 Install RangerAdmin
Decompression software:
[atguigu@hadoop103 target]tar -zxvf ranger-2.0.0-admin.tar.gz -C /opt/module/ranger
To configure:
[sarah@hadoop103 ranger-2.0.0-admin]$ vim install.properties
Modify the following configuration content:
#mysql driver SQL_CONNECTOR_JAR=/opt/software/mysql-connector-java-5.1.27-bin.jar #mysql's host name and root user's username and password db_root_user=root db_root_password=000000 db_host=hadoop103 The database name and user information required by #ranger correspond one-to-one with the information created in 2.2.1 db_name=ranger db_user=ranger db_password=ranger #Other user passwords required by ranger admin rangerAdmin_password=atguigu123 rangerTagsync_password=atguigu123 rangerUsersync_password=atguigu123 keyadmin_password=atguigu123 #ranger stores the path of the audit log, the default is solr, here is not set for convenience audit_store= #The url of the policy manager, which machine the rangeradmin is installed on, the host name is the corresponding host name policymgr_external_url=http://hadoop103:6080 #Start the linux user information of the ranger admin process unix_user=atguigu unix_user_pwd=atguigu unix_group=atguigu #hadoop configuration file directory hadoop_conf=/opt/module/hadoop-3.1.3/etc/hadoop
Then switch to the root user and execute the installation
[root@hadoop103 ranger-2.0.0-admin]# ./setup.sh
The following message appears, indicating that the installation is complete
2020-04-30 13:58:18,051 [I] Ranger all admins default password change request processed successfully.. Installation of Ranger Policy Manager Web Application is completed.
Create a ranger configuration file soft link to the web application:
[root@hadoop103 ranger-2.0.0-admin]# ./set_globals.sh usermod: no change [2020/04/30 13:58:47]: [I] Soft linking /etc/ranger/admin/conf to ews/webapp/WEB-INF/classes/conf
2.2.3 Start RangerAdmin
Configure the configuration information of the RangerAdmin web application:
[root@hadoop103 ranger-2.0.0-admin]# cd /etc/ranger/admin/conf/
[root@hadoop103 conf]# vim ranger-admin-site.xml
<property>
<name>ranger.jpa.jdbc.password</name>
<value>ranger</value>
<description />
</property>
<property>
<name>ranger.service.host</name>
<value>hadoop103</value>
</property>
start:
[root@hadoop103 conf]# ranger-admin start Starting Apache Ranger Admin Service Apache Ranger Admin Service with pid 7058 has started.
ranger-admin has been configured to start automatically when it is installed, so there is no need to start it manually afterwards! View the process after startup:
[root@hadoop103 ranger-2.0.0-usersync]# jps 7058 EmbeddedServer 8132 Jps
stop:
[root@hadoop103 conf]# ranger-admin stop
After that, you can visit http://hadoop103:6080, if the following interface appears, it means that the ranger-admin startup is complete!
2.2.4 Log in to the administrator user
By default, you can use the username: admin, and the password is atguigu123 configured before to log in! After login, the interface is as follows:
Chapter 3 Installing RangerUsersync
3.1 Introduction to Ranger Usersync
RangerUsersync, as a management module provided by Ranger, can synchronize the user and group information on the Linux machine to the RangerAdmin database for management!
3.2 Ranger Usersync installation
Decompression software:
[root@hadoop103 conf]# tar -zxvf /opt/software/apache-ranger-2.0.0/target/ranger-2.0.0-usersync.tar.gz -C /opt/module/ranger/
Configuration software:
[root@hadoop103 ranger-2.0.0-usersync]# vim install.properties
Modify the following configuration information:
url of #rangeradmin POLICY_MGR_URL=http://hadoop103:6080 #Synchronization interval, unit (minutes) SYNC_INTERVAL = 1 # The linux user running this process unix_user=atguigu unix_group=atguigu #rangerUserSync user password, refer to the configuration of install.properties in rangeradmin rangerUsersync_password=atguigu123 #hadoop configuration file directory hadoop_conf=/opt/module/hadoop-3.1.3/etc/hadoop
Use root user to install
[root@hadoop103 ranger-2.0.0-usersync]# ./setup.sh
The following message appears, indicating that the installation is complete
ranger.usersync.policymgr.password has been successfully created. Provider jceks://file/etc/ranger/usersync/conf/rangerusersync.jceks was updated. [I] Successfully updated password of ranger usersync user
3.3 Ranger Usersync start
Before starting, view the user information on the web-UI interface of ranger admin as follows:
Start with root user
[root@hadoop103 ranger-2.0.0-usersync]# ranger-usersync start Starting Apache Ranger Usersync Service Apache Ranger Usersync Service with pid 7510 has started.
After startup, check the user information again:
Description ranger-usersync works fine!
The ranger-usersync service is also self-starting at boot, so there is no need to start it manually afterwards!
Chapter 4 Installing the Ranger Hive-plugin
4.1 Introduction to Ranger Hive-plugin
Ranger Hive-plugin is a plugin for Ranger to manage permissions for hive. Ranger Hive-plugin can only manage permissions for requests to access hive using jdbc, and hive-cli is not limited!
4.2 Ranger Hive-plugin installation
Decompression software:
[root@hadoop103 ranger-2.0.0-usersync]# tar -zxvf /opt/software/apache-ranger-2.0.0/target/ranger-2.0.0-hive-plugin.tar.gz -C / opt/module/ranger/
Configuration software:
[root@hadoop103 ranger-2.0.0-hive-plugin]# vim install.properties
#url address of policy manager POLICY_MGR_URL=http://hadoop103:6080 #Component name can be customized REPOSITORY_NAME=hivedev #hive installation directory COMPONENT_INSTALL_DIR_NAME=/opt/module/hive The starting user of the #hive component CUSTOM_USER=atguigu #hive component starts the group to which the user belongs CUSTOM_GROUP=atguigu
Install the hive configuration file as a soft link to the Ranger Hive-plugin directory:
[root@hadoop103 ranger-2.0.0-hive-plugin]# ln -s /opt/module/hive/conf/conf
Enable Ranger Hive-plugin as root user
[root@hadoop103 ranger-2.0.0-hive-plugin]# ./enable-hive-plugin.sh
After that, you need to restart hive to take effect!
4.3 Configure hive plugin on ranger admin
4.4 Test connection to hiveserver2
Start hiveserver2:
[atguigu@hadoop103 hive]$ hive --service metastore & [atguigu@hadoop103 hive]$ hiveserver2
Test whether the plug-in can connect to hiveserver2
The following prompt appears to indicate that the connection is successful!
Chapter 5 Using Ranger to Manage Hive Rights
5.1 First experience of permission control
View the default access policy, at this time only the admin user has access to all libraries, tables and functions
Verification: Use the atguigu user to try to log in. After the login is successful, execute the query statement:
Then use the admin user to log in to complete all operations of Hive
5.2 Configure permissions for users
For example, configure the read permission of all columns in the emp and dept tables of the default database for the atguigu user, and configure the read and write permissions of all the columns of the emp and dept tables of the default database for the jack user. Click the Add New Policy button:
Fill in the strategy name, and the library, table, column and other information designed by this strategy:
Fill in the allowed user rights to design this policy:
Then click the Add button and find that it has been added on the panel:
Test: beeline does not need to reconnect to hiveserver2, executes the query again, and finds that the atguigu user can already query, but can only query the table information that he has permission to query
For the following two tables, there is read permission and no write permission:
Test the jack user again, and try to query after writing data to the dept table:
0: jdbc:hive2://hadoop103:10000> insert into table dept values(50,'SECURITY',1800); 0: jdbc:hive2://hadoop103:10000> select * from dept;
5.3 Desensitization operation
Through desensitization operations, users can be restricted from accessing a certain column, and sensitive data will not be exposed to users!
Case: Specify the atguigu user to desensitize the year and month of hiredate when querying the emp table!
First of all, you need to ensure that the user has access to the specified column, you can refer to 5.2 for configuration!
Click the Masing tab, then click Add New Policy
Specify tables and columns
Specify user and desensitization operation:
After that click the save button! Then only atguigu users will trigger this policy when they query!
5.4 Row-level filtering
Through row-level filtering, the data in the table can be conditionally filtered and then exposed to users!
For example: the atguigu user is only allowed to query the user information whose job type is SALESMAN in the emp table.
Similarly, row-level filtering also requires users to have access permissions to the specified table! Refer to the configuration of 5.2!
Select the Row Level Filter tab and click Add New Policy:
Select the corresponding library and table:
Add filter rules and users:
Then hit the add button!
verify:
question:
1. I have two users, sarah and jack, and I have set all the permissions for these two users. Now I want to verify whether the table creation and insertion can be successful:
Create a table using the jack user
CREATE EXTERNAL TABLE ods_log_inc3
(
id int
) COMMENT 'Event Information Form'
PARTITIONED BY (`dt` STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.JsonSerDe'
LOCATION '/tmp/ods_log_inc4';
can succeed
Insert data:
insert into ods_log_inc3 values (2,'1');
Report an error:
org.apache.hadoop.ipc.RemoteException: Permission denied: user=jack, access=WRITE, inode="/tmp/ods_log_inc4":sarah:supergroup:drwxr-xr-x
Solution: Change the permission of /tmp/ods_log_inc4 to 777
hdfs dfs -chmod -R 777 /tmp/ods_log_inc4
verify again
Successful execution!
This bug has been stuck for a long time
Chapter 6 Installing Ranger hdfs-plugin
Normal operation:
Unzip to the ranger directory
[sarah@hadoop102 ranger]$ ll total 16 drwxrwxr-x. 9 sarah sarah 4096 Apr 8 02:10 ranger-2.0.0-admin drwxrwxr-x. 4 sarah sarah 4096 Apr 11 21:08 ranger-2.0.0-hdfs-plugin drwxr-xr-x. 4 sarah sarah 4096 Apr 10 01:27 ranger-2.0.0-hive-plugin drwxr-xr-x. 10 sarah sarah 4096 Apr 8 05:30 ranger-2.0.0-usersync
Modify the configuration file:
[sarah@hadoop102 ranger-2.0.0-hdfs-plugin]$ vim install.properties POLICY_MGR_URL=http://mycluster:6080 REPOSITORY_NAME=hdfsenv COMPONENT_INSTALL_DIR_NAME=/opt/module/hadoop CUSTOM_USER=sarah CUSTOM_GROUP=sarah
Start under root:
[root@hadoop102 ranger-2.0.0-hdfs-plugin]# ./enable-hdfs-plugin.sh
Distribute ranger-2.0.0-hdfs-plugin, the rule is to distribute several copies by several namenodes
Otherwise, a bug will be reported: Class org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer not found