1. Cluster planning hadoop01(209.2) hadoop02(209.3) hadoop03(209.4) HDFS NameNode DataNode DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager Do not put NameNode and SecondaryNameNode on the same server 2. Create user useradd atguigu passwd *** Configure atguigu user permissions vim /etc/sudoers ## Allow root to run any commands anywhere root ALL=(ALL) ALL ## Allows people in […]
Tag: oop
Hadoop3.3.4 distributed installation
Installation prerequisites: The Java environment has been configured and password-free SSH login between all machines. Note: flinkv1, flinkv2, and flinkv3 below are aliases of the three servers. 1. Cluster deployment planning Note: NameNode and SecondaryNameNode should not be installed on the same server Note: ResourceManager also consumes a lot of memory. Do not configure it […]
Build a Hadoop fully distributed cluster
Table of Contents Preparations required for construction: Building process: 1. Install the virtual machine 2. Configure the network 3. Modify the host name 4. Bind the host name and IP 5. Configure password-free login 6. Use remote connection tools to upload jdk and hadoop 7. Install jdk and Hadoop 1. Unzip jdk and hadoop 2. […]
HBase and Hadoop integration
1. Start the hadoop service process and hbase service process [Command 001]: start-all.sh start-hbase.sh 2.Create the directory /root/experiment/datas on HDFS [Command 002]: hadoop fs -mkdir -p /root/experiment/hbase/file1.txt /root/experiment/datas 3. Upload the local directory /root/experiment/datas/hbase/file1.txt file to the /root/experiment/datas directory of HDFS [Command 003]: hadoop fs -put /root/experiment/datas/hbase/file1.txt /root/experiment/datas 2) Experimental process 1. Double-click the “IDEA” […]
Replace for loops and make Python code more pythonic!
Text | Parson Sauce Source: Python Technology Why challenge yourself not to write a for loop in your code? Because this forces you to learn to use more advanced and more idiomatic syntax or libraries. The article uses python as an example to talk about a lot of syntax that everyone has seen in other […]
An article to understand the running mechanism of js – event loop
1. Explain the execution mechanism of JavaScript. The execution mechanism of JavaScript is based on the event loop. The event loop includes a task queue (Task Queue) and a microtask queue (Microtask Queue). When a function is called, it is added to the microtask queue. Each iteration of the event loop will first execute all […]
Hadoop 3.3.6 distributed cluster environment construction
1 System Configuration A total of 3 centOS virtual machines have been prepared, master, slave1, slave2 Configure hosts resolution vim /etc/hosts 192.168.10.11 master 192.168.10.12 slave1 192.168.10.13 slave2 Modify hostname #Execute the following commands on the corresponding server hostnamectl set-hostname master hostnamectl set-hostname slave1 hostnamectl set-hostname slave2 Turn off firewall #Close firewall systemctl stop firewalld.service #Disable […]
IDEA integrates Hadoop3.x under Windows
1. Hadoop development package download 1.1 Download the corresponding hadoop development package on the windows system, for example: hadoop3.3.5 version 1.1 Unzip the development package and rename it 2. Patch 2.1 Since it is a windows development system, some patches need to be installed to run normally. Install them in the bin directory of hadoop. […]
Event loopEventloop
Event Loop Browser process model What is a process? A program needs its own dedicated memory space to run. This memory space can be simply understood as a process. Each application has at least one process, and the processes are independent of each other. Even if they want to communicate, both parties need to agree. […]
WSL + Vscode one-stop to build Hadoop pseudo-distributed + Spark environment
Wsl + Vscode one-stop to build Hadoop + Spark environment If you want to build an environment such as Linux, Hadoop, Spark, etc., the common practice now is to install a virtual machine on VM, Virtualbox and other software This article introduces how to build a relevant environment on the windows subsystem (Windows Subsystem for […]