Installation environment and installation package
centos7
hue-4.5.0
Node.js 10.6.0
Note: The linux system first creates a hue user, and installs the mysql-5.7 database in the hue user
Download the installation package hue-4.5.0
The extraction address of the hue installation package
Link: Baidu Netdisk Please enter the extraction code
Extraction code: 09fs
Download npm
Download Node.js 10.6.0 from the official website:
Link: Index of /download/release/v10.6.0/
Decompression: tar -xf node-v10.6.0-linux-x64.tar.xz
Configure environment variables vim /etc/profile.d/my_env.sh
#NPM_HOME
NPM_HOME=/home/hue/node-v10.6.0-linux-x64 (node.js decompression address)
export PATH=
P
A
T
h
:
PATH:
PATH:NPM_HOME/bin:$NPM_HOME/sbin
source /etc/profile.d/my_env.sh
Configure Taobao Mirror
npm config set registry https://registry.npm.taobao.org
Compile and install hue
unzip hue
tar -zxvf hue-4.5.0.tgz
Install dependencies (python 2.7 + /python 3.5 +)
sudo yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c + + krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql mysql-devel openldap-devel python-devel sqlite -devel gmp-devel
Configure mysql database
Create hue database:
CREATE DATABASE hue DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
Create hue user in mysql:
CREATE USER ‘hue’@’%’ IDENTIFIED BY ‘123456’;
Add all permissions of the hue database to the hue user:
GRANT ALL ON . TO ‘hue’@’%’;
Modify the hue.ini configuration file
vim /home/hue/app/hue-4.5.0/desktop/conf/hue.ini
[desktop]
http_host=dev-wicv-003-05
http_port=8888
time_zone=Asia/Shanghai
server_user=hue
server_group=hue
default_user=hue
[[database]]. Hue uses the SQLite database to record related metadata by default, replace it with mysql
engine=mysql
host=dev-wicv-003-05
port=3306
user=hue
password=123456
Name database
name=hue
Initialize the database
Initialize hue’s metadata database
cd /home/hue/app/hue-4.5.0/build/env/bin
./hue syncdb
./hue migrate
Compile hue
Enter the hue source code directory and compile
cd /home/hue/app/hue-4.5.0
make apps
Configuration file modification
Big data component configuration modification
Integration of hadoop, hive and other components
Modification of Hadoop configuration
Add configuration in hdfs-site.xml
<!-- HUE -->
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>
Add configuration in core-site.xml
<!-- HUE -->
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hdfs.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hdfs.groups</name>
<value>*</value>
</property>
httpfs-site.xml file, written in the file
<configuration>
<!-- HUE -->
<property>
<name>httpfs.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>httpfs.proxyuser.hue.groups</name>
<value>*</value>
</property>
</configuration>
1 Remarks: After modifying the HDFS related configuration, you need to scp the configuration to each machine in the cluster and restart the hdfs service
HUE configuration modification
vim /home/hue/app/hue-4.5.0/desktop/conf/hue.ini
Configure hadoop and yarn
[hadoop]
#Configuration for HDFS NameNode
-------------------------------------------------- ----------------------
[[hdfs_clusters]]
#HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://dev-wicv-003-01:9000
# NameNode logical name.
## logical_name=
# Use WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs host.
# Default port is 14000 for HttpFs.
webhdfs_url=http://dev-wicv-003-01:50070/webhdfs/v1
# Change this if your HDFS cluster is Kerberos-secured
## security_enabled=false
# In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
# have to be verified against certificate authority
## ssl_cert_ca_verify=True
# Directory of the Hadoop configuration
## hadoop_conf_dir=$HADOOP_CONF_DIR when set or '/etc/hadoop/conf'
## hadoop_conf_dir=/home/hadoop/app/hadoop/hadoop-2.6.5/etc/hadoop/
# Configuration for YARN (MR2)
# -------------------------------------------------- -----------------------
[[yarn_clusters]]
[[[default]]]
# Enter the host on which you are running the ResourceManager
resourcemanager_host=dev-wicv-002-02
# The port where the ResourceManager IPC listens on
resourcemanager_port=8032
# Whether to submit jobs to this cluster
submit_to=True
# Resource Manager logical name (required for HA)
# logical_name=
# Change this if your YARN cluster is Kerberos-secured
## security_enabled=false
# URL of the ResourceManager API
resourcemanager_api_url=http://dev-wicv-002-02:8088
# URL of the ProxyServer API
proxy_api_url=http://dev-wicv-002-02:8088
# URL of the HistoryServer API
history_server_api_url=http://dev-wicv-002-02:19888
# URL of the Spark History Server
## spark_history_server_url=http://localhost:18088
# Change this if your Spark History Server is Kerberos-secured
## spark_history_server_security_enabled=false
# In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
# have to be verified against certificate authority
## ssl_cert_ca_verify=True
# HA support by specifying multiple clusters.
# Redefine different properties there.
# e.g.
[[[ha]]]
# Resource Manager logical name (required for HA)
logical_name=rm2
# Un-comment to enable
submit_to=True
# URL of the ResourceManager API
resourcemanager_api_url=http://master2:8088
history_server_api_url=http://master1:19888
#...
Configure hive
[beeswax] # Host where HiveServer2 is running. # If Kerberos security is enabled, use fully-qualified domain name (FQDN). hive_server_host=dev-wicv-003-01 # Port where HiveServer2 Thrift server runs on. hive_server_port=10000
Run hue
cd /home/hue/app/hue-4.5.0 ./build/env/bin/supervisor
?
-
Just open the host:port configured in hue.ini