Getting started with HDFS 1 File operations 1.1 Command line interaction 1.1.1 Overview A common command format is as follows: hdfs [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS] SUBCOMMAND: Admin Commands, Client Commands, Daemon Commands. At the same time, HDFS Shell CLI supports operating a variety of file systems, including local file systems (file:///) and distributed file systems […]
Tag: hdf
“Big Data-2.2” uses commands to operate the HDFS file system
Table of Contents 1. Basic information of HDFS file system 1. HDFS path expression form 2. The difference between HDFS and Linux root directories 2. Use commands to operate the HDFS file system 0. Two command systems of Hadoop 1. Create a folder 2. View the contents in the specified directory 3. Upload files to […]
Create, delete and query HDFS directories and files
Article directory Create, delete and query HDFS directories and files 1. Experimental goals 2. Experimental requirements 3. Experimental content 4. Experimental steps Attachment: series of articles Creation, deletion and query operations of HDFS directories and files 1. Experimental objectives Proficient in hadoop operating instructions and HDFS command line interface Master how to create HDFS directories […]
“Big Data-2.0” Install Hadoop and deploy HDFS cluster
Table of Contents 1. Download the Hadoop installation package 2. Install Hadoop 0. Necessary preparations before installing Hadoop 1. Log in to the master node virtual machine as the root user 2. Upload the Hadoop installation package to the master node 3. Unzip the installation package to the /export/server/ directory 4. Build soft links 3. […]
1. Configure HDFS HA (high availability)
Prerequisites First build the zk environment and start it: https://www.cnblogs.com/zhugq02/p/15759195.html Architecture planning: 192.168.167.82 node2 nn\zk1\zkfc 192.168.167.83 node3 jn\zk2\dn 192.168.167.84 node4 jn\zk3\dn 192.168.167.85 node5 nn\jn\dn 1. Environment initialization (all nodes) 1. Turn off the firewall and iptables systemctl stop firewalld & amp; & amp; systemctl status firewalld service iptables stop & amp; & amp; service iptables […]
HDFS file creation and writing
Article directory HDFS file creation and writing 1. Experimental goals 2. Experimental requirements 3. Experimental content 4. Experimental steps Attachment: series of articles HDFS file creation and writing 1. Experimental objectives Proficient in Hadoop operating instructions and HDFS command line interface Master HDFS principles Proficient in using HDFS API Master the method of writing a […]
Use flume to import local files to HDFS specified path
Use flume to import local files to HDFS specified path 1. Flume operation 1. Edit the configuration file uploadPic.conf 2. Execute the flume command 2. Monitoring (1) flume exposes http information to the outside world, add the following parameters when starting (2) Start flume-exporter (3) prometheus receives flume-exporter indicators (4) Grafana adds dashboard and draws […]
bin/hdfs dfs command
appendToFile Usage: hdfs dfs -appendToFile … Append one or more files to the hdfs specified file. You can also read input from the command line. hdfs dfs -appendToFile localfile /user/hadoop/hadoopfile hdfs dfs -appendToFile localfile1 localfile2 /user/hadoop/hadoopfile hdfs dfs -appendToFile localfile hdfs://nn.example.com/hadoop/hadoopfile hdfs dfs -appendToFile – hdfs://nn.example.com/hadoop/hadoopfile Reads the input from stdin. Exit Code: Returns 0 […]
Hadoop experiment-reading and writing HDFS files
Hadoop experiment-reading and writing HDFS files 1 Experimental purpose Through this experiment, students can learn to install the Eclipse Hadoop plug-in locally (Windows environment); they can write Java code to read and write HDFS files in the Eclipse environment; they can learn to package code based on Eclipse and run the code on the cluster. […]
HadoopHDFS API operation list
Blogger “Driving a tractor home” takes you Go to New World.? Personal homepage–Driving a tractor home_Linux, big data operation and maintenance-CSDN blog ? I hope this article can bring you some help The article is superficial, please criticize and correct me! ?Thanks for the likes and attention, we will make a little progress every day! […]