1. Hadoop3.1.3 cluster construction

1. Cluster planning hadoop01(209.2) hadoop02(209.3) hadoop03(209.4) HDFS NameNode DataNode DataNode SecondaryNameNode DataNode YARN NodeManager ResourceManager NodeManager NodeManager Do not put NameNode and SecondaryNameNode on the same server 2. Create user useradd atguigu passwd *** Configure atguigu user permissions vim /etc/sudoers ## Allow root to run any commands anywhere root ALL=(ALL) ALL ## Allows people in […]

Build a Hadoop fully distributed cluster

Table of Contents Preparations required for construction: Building process: 1. Install the virtual machine 2. Configure the network 3. Modify the host name 4. Bind the host name and IP 5. Configure password-free login 6. Use remote connection tools to upload jdk and hadoop 7. Install jdk and Hadoop 1. Unzip jdk and hadoop 2. […]

Clustering algorithm–K-Means algorithm

cluster learning 1. Clustering concept: 1. Unsupervised problem: We have no labels in our hands, and the computer needs to learn the labels by itself. 2. Clustering: In a pile of data, similar things are grouped into a group, and a group is a category, also called a cluster. 3. Difficulty: How to evaluate the […]

Kubernetes cluster expansion

1. Please explain the basic concepts and principles of Kubernetes cluster expansion. Basic concepts and principles of Kubernetes cluster expansion: Kubernetes is an open source container orchestration platform used to automate the deployment, scaling and management of application containers. In Kubernetes, cluster expansion refers to increasing the computing resources (such as CPU, memory, etc.) in […]

Springboot2.x integrated lettuce connection redis cluster reports timeout exception Command timed out after 6 second(s)

Original/Zhu Jiqian Background: Recently, I was doing a stress test on a newly developed Springboot system. I found that when I first started the stress test, I could access data from the redis cluster normally. However, after a few minutes of pause, and then when I continued to use jmeter to perform the stress test, […]

Hadoop 3.3.6 distributed cluster environment construction

1 System Configuration A total of 3 centOS virtual machines have been prepared, master, slave1, slave2 Configure hosts resolution vim /etc/hosts 192.168.10.11 master 192.168.10.12 slave1 192.168.10.13 slave2 Modify hostname #Execute the following commands on the corresponding server hostnamectl set-hostname master hostnamectl set-hostname slave1 hostnamectl set-hostname slave2 Turn off firewall #Close firewall systemctl stop firewalld.service #Disable […]

ZooKeeper+Kafka+ELK+Filebeat cluster construction to realize large batch log collection and display

The general process: After collecting the logs of the nginx server (web-filebeat) through filebeat, store them in the cache server kafka, and then logstash to the kafka server to retrieve the corresponding logs. After processing, they are written to the elasticsearch server and displayed on kibana. 1. Cluster environment preparation 4c/8G/100G 10.10.200.33 Kafka + ZooKeeper […]

Zookeeper cluster deployment

Table of Contents (1) Unzip the Zookeeper installation package 1. Go to the browserApache ZooKeeper to download the zookeeper compressed package 2. Upload the zookeeper compressed package to softwares 3. Unzip to package (2) Configure environment variables and distribute 1. Modify configuration file 2. Configure environment variables 3.?wqSave and exit 4.Use scp to copy the […]

kubeadm installs k8s high availability cluster

Table of Contents 1. Environmental planning 2. Things to note: 3. Environment preparation: 1. Turn off firewall rules, turn off selinux, and turn off swap: 2. Modify the host name 3. Modify the hosts file on all nodes: 4. Time synchronization of all nodes: 5. All nodes implement Linux resource limits: 6. Upgrade the kernel […]

A redis3 master-3-slave cluster built based on hash slots using docker containers

1. Construction background Cluster mode-docker version, hash slot partition for billion-level data storage 1. 100-200 million pieces of data need to be cached. How to design it? Answer: Using distributed storage, there are generally three solutions in the industry The first type: hash remainder partition (generally used by small factories) Disadvantages: The biggest disadvantage of […]