1.Zookeeper introduction Zookeeper is a distributed coordination component developed by Yahoo based on Chubby’s ideas and later donated to Apache. It is mainly used to solve distributed data consistency problems. Distributed applications can implement functions such as data publishing/subscription, load balancing, naming service, distributed coordination/notification, cluster management, Master election, configuration management, distributed locks and distributed […]
Tag: cluster
Deploy hadoop-3.3.6, hbase-2.5.6, apache-zookeeper-3.8.1 cluster on linux
1. Introduction to hadoop Hadoop is a distributed system infrastructure developed by the Apache Foundation. Users can develop distributed programs without understanding the underlying details of distribution. Make full use of the power of clusters for high-speed computing and storage. Hadoop implements a distributed file system (Distributed File System), one of which is HDFS (Hadoop […]
Deploy cluster on hadoop 3.3.6 under linux
1. Introduction to hadoop Hadoop is a distributed system infrastructuredeveloped by the Apache Foundation. Users can develop distributed programs without understanding the underlying details of distribution. Make full use of the power of clusters for high-speed computing and storage. Hadoop implements a distributed file system (Distributed File System), one of which is HDFS (Hadoop Distributed […]
The situation when the Hadoop cluster DataNode cannot be started under Linux
1. After formatting the file, restart the hadoop cluster and find that the DataNode cannot be started. Here too There is no node data, but it is empty. This graph is normal. After searching for the reason for a long time, I found that the datanode was not started. Then query the log. Found this […]
Configure kafka cluster on windows10
1. Introduction to kafka Kafka is an open source stream processing platform developed by Apache and written in Scala and Java. Kafka is a high-throughput distributed publish-subscribe messaging system that can process all behavioral record data of consumers in the website 2. kafka download kafka address 3. kafka cluster configuration 1. Zookeeper configuration # Licensed […]
Hadoop cluster environment setup
Hadoop environment setup write in front Statement and pitfall summary Software preparation Introduction to Hadoop Officially built Prerequisite environment preparation Install Ubuntu20.04 Server XShell remote connection Install jdk Install hadoop SSH Static IP configuration Cluster construction Cluster creation Modify hostname and IP address SSH configuration Hadoop configuration workers core-site.xml hdfs-site.xml yarn-site.xml mapred-site.xml Scripting xsync myhadoop.sh […]
k8s[kubernetes] cluster construction (0>1)
Article Directory 1. Planning for building a K8S environment platform 2. Server hardware configuration requirements 3. Set up k8s cluster deployment method 4. Deployment in kubeadm mode – system initialization operation 5. Deployment in kubeadm mode – deploy master node 1Install Docker on all nodes 2Configure Alibaba Cloud Docker and kubernetes images on all nodes […]
15Ethernet link aggregation and switch stacking and clustering
1. Network reliability requirements 1. Network reliability Network reliability refers to the ability to ensure uninterrupted network services when single or multiple points of equipment or links fail. Network reliability can be achieved at multiple levels, including boards, devices, and links. 2. Single board reliability 2.1.Single board reliability-1 A modular switch consists of a chassis, […]
Install flink 1.18 cluster under linux
1. Introduction to flink Flink is an open source stream processing framework built for distributed, high-performance, ready-to-use, and accurate stream processing applications 2. Environment configuration export JAVA_HOME=/usr/local/jdk1.8.0_391 export JRE_HOME=/usr/local/jdk1.8.0_391/jre export HBASE_HOME=/usr/local/bigdata/hbase-2.5.6 export HADOOP_HOME=/usr/local/bigdata/hadoop-3.3.6 export FLINK_HOME=/usr/local/bigdata/flink-1.18.0 export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JAR_HOME/lib export PATH=.:$JAVA_HOME/bin:$JRE_HOME/bin:$FLINK_HOME/bin:$HADOOP_HOME/bin:$HBASE_HOME/bin:$PYTHON_HOME/bin:$PATH 3. Prepare three servers 192.168.42.139 node1 192.168.42.140node2 192.168.42.141node3 4. Create logs and data under flink, and […]
RabbitMQ cluster deployment
Table of Contents 1.Environment preparation 1.1 Turn off the firewall and selinux 1.2 Local analysis 1.3 Install rabbitmq software 1.4 Start the service 1.5Create user 1.6 Enable user remote login 2. Start deploying the cluster (operate on all three machines) 2.1 First create the data storage directory and log storage directory 2.2 Copy erlang.cookie 2.3 […]