Specific introduction to zookeeper Can the point of optimization be zookeeper?
How to install and use?
#include
1. Configure the java environment JDK first, because it needs to be compiled with java;
2. Download zk source code and decompress it;
3. Rename the configuration file zoo_sample.cfg
4. Start zookeeper, check whether it is successfully started, and use the zookeeper client to connect to the server
How to use it on the C++ client after installation? zk has a C-style API, and we can customize classes to implement C++.
First of all, the zookeeper header file must be included when defining the class, which contains the commonly used interfaces of zk. Then link to specify link zk_mt library (multi-client)
Server call code: Generate a zk class, open it, and publish nodes (including services and methods, services are permanent nodes, methods are temporary nodes, and ip + port will be added)
//Register all the services to be published on the current rpc node to zk, so that the rpc client can discover services from zk
ZookeeperClient zk_client;
zk_client.start();
//Create a node in the configuration center
for (auto & sp : service_map_)
{<!-- -->
string service_path = "/" + sp.first;
zk_client.create(service_path.c_str(), nullptr, 0);
for (auto & mp : sp. second. method_map_)
{<!-- -->
string method_path = service_path + "/" + mp.first;
char method_path_data[128] = {<!-- -->0};
sprintf(method_path_data, "%s:%d", ip.c_str(), port);
//ZOO_EPHEMERAL indicates a temporary node when znode
zk_client.create(method_path.c_str(), method_path_data, strlen(method_path_data), ZOO_EPHEMERAL);
}
}
Client: Just use the get_data method to find the server’s ip + port
//Get ip and port
ZookeeperClient zk_client;
zk_client.start();
string method_path = "/" + service_name + "/" + method_name;
string host_data = zk_client.get_data(method_path.c_str());
The zookeeper class here is as follows: it mainly defines the start function, the node creation function and the data acquisition function, and of course a zk handle
#pragma once
#include <semaphore.h>
#include <zookeeper/zookeeper.h>
#include <string>
using namespace std;
class ZookeeperClient
{<!-- -->
public:
ZookeeperClient();
~ZookeeperClient();
//Start the connection--"zkserver
void start();
//Create a znode node in zkserver according to the specified path
void create(const char *path, const char *data, int datalen, int state = 0);
//According to the znode node path specified by the parameter, get the value of the znode node
string get_data(const char *path);
private:
// zk's client handle
zhandle_t *zhandle_;
};
The constructor is to initialize NULL for the handle, and the destructor is to release the handle zookeeper_close(zhandle_);
start zk
void ZookeeperClient::start()
{<!-- -->
string host = RpcApplication::get_instance().get_configure().find_load("zookeeper_ip");
string port = RpcApplication::get_instance().get_configure().find_load("zookeeper_port");
string con_str = host + ":" + port;
cout << con_str << endl;
zhandle_ = zookeeper_init(con_str.c_str(), global_watcher monitoring point function, the function is to check whether the connection status is successful, get the handle semaphore if successful, perform sem_post synchronization, add 1 to the semaphore, 30000 expiration time, nullptr, nullptr, 0);
if (zhandle_ == nullptr)
{<!-- -->
RPC_LOG_FATAL("zookeeper init error");
}
sem_t sem;
sem_init( & sem, 0, 0);
zoo_set_context(zhandle_, & amp;sem); //Set semaphore s
sem_wait( & amp;sem); //This is equivalent to either getting the resource semaphore minus 1, or blocking and waiting
RPC_LOG_INFO("zookeeper init success");
}
It can be seen that the zookeeper_init function of zk’s original interface is actually called to generate a handle. Then there are some semaphore synchronization operations, the specific meaning is not clear to me.
The function of creating a node is also synchronously checking whether the path exists flag = zoo_exists(zhandle_, path, 0, nullptr); if there is no call to the zoo_create function. Obtaining data also uses the zoo_get function.
Talk about your understanding of zk:
zk is an open source distributed coordination service, which provides consistent software for distributed. Implement functions such as data publishing/subscription, load balancing, naming service, distributed coordination/notification, cluster management, master election, distributed lock and distributed queue.
My project does not involve the concept of zk clusters, but the publication and subscription of data, using the create and getdata functions is relatively simple.
The advantage of zk is that it encapsulates some key services that are complex and error-prone (especially services with strong consistency) into simple interfaces, which are easy to use.
The main components of zk are
File system: Similar to ordinary file systems, especially directory nodes can also store data. For low latency, the general data length is limited to about 1M, and nodes are divided into permanent nodes (unless manually deleted, nodes will always exist on Zookeeper) and ephemeral nodes. The temporary node is established by the client. Once the client disconnects, the client will become invalid. There are also permanent order and temporary order. A self-incrementing integer number maintained by the parent node will be appended to the node name. (four node types)
Notification mechanism; watcher mechanism. Zookeeper allows the client to register a Watcher with a Znode on the server. When some specified events on the server trigger the Watcher, the server will send an event notification to the specified client to implement the distributed notification function, and then the client will follow the Watcher Notify state and event types to make business changes.
(Here we are talking about a single instance, mainly the file system and notification mechanism. The following will talk about the cluster, that is, the master node, the slave node and the observer node)
How does zk ensure consistency?
Order is a very important feature in zookeeper. All updates are globally ordered, and each update has a unique timestamp. This timestamp is called zxid (Zookeeper Transaction Id ). The read request will only be ordered relative to the update, that is, the return result of the read request will have the latest zxid of this zookeeper.
How does zk ensure the security of permissions
For example, Linux and redis are all in UGO mode, user/group/others. Coarse grained
The ACL access control list permission mechanism is a fine-grained and more secure. Specifically, it can be based on the ip address, or account password, or root and other modes. The permissions for reading, writing, and deleting management nodes of each mode are different. And the managed object is an entity.
What roles does a zk instance have?
leader core node, processing requests, synchronizing data, scheduling (why there is a master node to avoid double calculation)
The follower writes the request and forwards it to the leader to participate in voting
observer does not participate in voting. In order to increase the non-transaction processing capability of the cluster
Data synchronization uses the ZAB protocol.
If the leader goes down, the leader will be re-elected.
ZAB protocol is an atomic broadcast protocol specially designed for distributed coordination service Zookeeper to support crash recovery. The ZAB protocol includes two basic modes: crash recovery (leader down, more than half of the nodes down, restart the system) and message broadcast. After more than half of the machines in the cluster complete data synchronization with the Leader server, exit the recovery mode and enter the message broadcast mode, and the Leader server starts to receive transaction requests from clients and generate transaction proposals for transaction request processing.
Data synchronization is also divided into full synchronization, differentiated synchronization, rollback and resynchronization, etc., which have not been studied in detail.
(ZAB differs from raft)
Because raft came out later than zab, maybe some things in raft will learn from the zab protocol, essentially maintaining a replicated log. The core is log replication, that is, data synchronization. Raft is a bit simpler to implement than zab.
Same point:
1. Use quorum to determine the consistency of the entire system (that is, the recognition of a certain value). This quorum is generally implemented by more than half of the servers in the cluster. Zookeeper also provides a weighted quorum implementation.
2 Write operations are all written on the leader
3. Which one to choose as the leader is similar, raft is term + log index, and zab is the latest quorum as the leader.
4. There is a heartbeat mechanism to check whether the leader is alive, and the timeout period.
difference
Zab uses the combination of epoch and count to uniquely represent a value, while raft uses term and index.
Raft protocol data only one-way from leader to follower (one of the conditions for becoming a leader is to have the latest log), and in zab’s zookeeper implementation, a prospective leader needs to send its own The log is updated to the latest log in the quorum, and then the logs of other machines in the quorum are synchronized to be consistent during the synchronization phase.
To sum up: they are all protocols for consistency in distributed systems. The core is the log replication and leader election mechanism.
The difference lies in the different criteria for selecting a leader and the strength of the leader. The strong leader of Raft makes the implementation easier.
In addition, raft can achieve strong consistency (linear consistency read), ZAB does not know if it can.
Two-phase commit and three-phase commit
See the article on distributed transactions
Data publishing and subscription system/configuration center/this project registration discovery center
Features: The amount of data is small, and the configuration may be updated.
Implementation based on Zookeeper
Data storage: store data (configuration information) to a data node on Zookeeper
Data acquisition: the application reads data from the Zookeeper data node at the startup initialization node, and registers a data change Watcher on the node
Data change: When changing data, update the corresponding node data of Zookeeper, and Zookeeper will send a data change notification to each client, and the client can re-read the changed data after receiving the notification.
zk implements distributed locks
1. Zookeeper stipulates that at the same time**, multiple clients cannot create the same node**. We can use this feature to implement distributed locks. The zookeeper temporary node only exists in the session life cycle, and will be automatically destroyed when the session ends.
2. The watcher mechanism, when the node representing the lock resource is deleted, it can trigger the watcher to unblock and reacquire the lock, which is also a major advantage of the zookeeper distributed lock over other distributed lock solutions.
The first is based on ephemeral nodes. Only the A thread that acquires the lock can create nodes. Others are blocked and set to monitor. After A ends, the node is automatically deleted, and other threads re-compete.
This scheme will produce a shocking group phenomenon and affect performance.
The second type is based on sequential temporary nodes. Each node only listens to the information of the previous node. The one with the smallest sequence number acquires the lock, and deletes itself after the end. Then the second smallest one obtains the lock when it is monitored.
The role of the message queue
Decoupling, peak clipping, asynchronous (unnecessary logic runs asynchronously to speed up response)
Kafka
First of all, there is a concept of topic, which is similar to a table.
Partition partition: There are multiple partitions under a topic, and these partitions will be stored on different servers, or in other words, different directories are actually built on different hosts. Multiple partitions and multiple threads, and parallel processing of multiple threads will definitely be much better than single thread. Topic is also a logical concept, and Partition is a distributed storage unit.
Replica copy mechanism: Partition can set multiple copies for each Partition in order to ensure data security.
And in fact, each copy has a role, they will choose one copy as the leader, and the rest as followers.
When our producers send data, they send it directly to the Leader Partition, and then the Follower Partition will go to the Leader to synchronize the data by themselves. When consumers consume data, they also consume data from the Leader.
Consumer Group Consumer Group:
When we consume data, we will specify a group.id in the code, and this id represents the name of the consumption group. Different groups can have only one consumer to consume the data of the same topic.
Kafka is also a master-slave architecture. The master node is called Controller, and the rest are slave nodes. Controller needs to cooperate with Zookeeper to manage the entire Kafka cluster.
Distributed deployment of brokers requires a registration center for unified management. Zookeeper uses a dedicated node to save the Broker service list, which is /brokers/ids. When the broker starts, it sends a registration request to Zookeeper, and Zookeeper creates the broker node under /brokers/ids, such as /brokers/ids/[0…N], and saves the broker’s IP address and port.
This node is a temporary node. Once the broker goes down, this temporary node will be automatically deleted.
Zookeeper will also allocate a separate node for the topic, and each topic will be recorded in Zookeeper in the form of /**brokers/topics/[topic_name]**. A topic message will be saved to multiple partitions, and the corresponding relationship between these partitions and brokers also needs to be saved in Zookeeper. When the leader failover of the partition also requires the participation of zookeeper.
The consumer group will also register with Zookeeper, and Zookeeper will allocate nodes for it to store relevant data. The node path is /consumers/{group_id}
The election of Controller depends on Zookeeper. After the election is successful, Zookeeper will create a /controller temporary node.
The controller is used to monitor partition changes (metadata changes, Leader failures, increase or decrease of partitions), topic changes, and broker changes.
The kafka disadvantage is that two systems need to be deployed at the same time. Moreover, when the number of partitions increases, the pressure on the Zookeeper cluster increases. After reaching a certain level, the monitoring delay increases, which affects the work of Kafaka. And if a single control node fails, after the new Controller is elected successfully, it will pull metadata from Zookeeper again for initialization, and all other brokers need to be notified to update the ActiveControllerId. The old Controller needs to close the listener, event processing thread and timing task. When the number of partitions is very large, this process is very time-consuming, and the Kafka cluster cannot work during this process.
(There are new ones that use multiple control nodes and use the raft protocol to ensure their consistency)
What makes Kafka perform better?
1. Write to disk sequentially. If you write randomly, you modify the data at a certain position in the file, and the performance will be lower.
2. Zero-copy technology. The following is a non-zero copy, the data is copied from the kernel state to the kafka process space, and then copied to the socket cache. Time-consuming is relatively high.
Kafka uses Linux’s sendFile technology (NIO), which saves process switching and a data copy, making performance better. The data from the kernel is sent directly to the network card.
A little more about zero-copy technology. The vector of C++ also involves zero copy. For example, emplace_back is zero copy, because there is no need to copy a temporary object, and the rvalue reference is used? Especially when it comes to large data copies, it consumes time and space resources
And here’s zero-copy in the network:
Traditional: read first DMA copies from disk to kernel page cache, and then copies to user space. write is first copied from the user to the socket kernel cache, and then copied to the network card;
mmap: A type of zero copy. The main idea is to share the kernel space and user mode, so there is no need to copy to user mode and context switch.
sendfile: The Linux2.1 kernel began to introduce the sendfile function, which is used to transfer files through sockets. The main optimization is to record the position and offset of the current data in the page buffer in the kernel socket buffer. In this way, it can be sent to the network card directly from the page buf, without the process of copying.
3. Log Segmented Storage Kafka stipulates that the maximum size of .log files in a partition is 1G. The purpose of this limitation is to facilitate loading .log into memory for operation. In fact, redis also has restrictions on the size of RDB files during master-slave replication, otherwise it will be very slow to load into memory.
4. Kafka network design
Enhanced Reactor network threading model. It is a three-tier design prosessors-qequest-thread pool.
The acceptor does not process it but encapsulates it into a channel for processors. Then when the consumer thread consumes these socketChannels, it will get Request requests one by one, which contains data, and then the thread pool will read them.
So if we need to enhance and tune Kafka, increase the Processor and increase the processing threads in the thread pool, the effect can be achieved.
ngix
It is a http proxy and reverse proxy web server, with less memory of 15MB, fast startup and high concurrency. It can also achieve load balancing and dynamic and static separation.
Application scenario:
1. Do http server, static website, for example: we use OpenVPN.
2. Reverse proxy realizes load balancing. When the number of website visits reaches a certain level and a single server cannot satisfy user requests, multiple server clusters are required to use nginx as a reverse proxy, and multiple servers can evenly share the load. It will make the client not feel it, and still enter the original URL.
3. Security configuration firewall function
The process of processing a request:
Read the configuration file at startup to get the ip and port that need to be monitored. Then the main process will create a socket (socket, bind, listen)
Then fork many child processes to compete to accept new connections. At this time, the client can establish a three-way handshake with ngix. A child process will accept successfully, get the Socket of the established connection, and then create nginx to encapsulate the connection
Next, set the read and write event processing functions, and add read and write events to exchange data with the client. Finally, Nginx or the client will take the initiative to close the connection. At this point, a connection is dead
Reason for high concurrency: Because webserver is IO-intensive, mainly network IO. So IO multiplexing + asynchronous non-blocking IO is also used
It also uses single thread + epoll and redis.
Static and dynamic separation Do cache operation according to the characteristics of static resources, this is the core idea of website static processing
Strictly speaking, it can be understood as using nginx to process static pages, and tomcat or PHP to process dynamic pages. Static resources are accessed multiple times and the source code remains unchanged (CSS, jpg, js). Dynamic resources: When users access this resource multiple times, the source code of the resource may be sent to change.
Of course, because CDN services such as Qiniu and Alibaba Cloud are very mature now, the mainstream approach is to cache static resources in CDN services to improve access speed.
Compared with the local Nginx, the CDN server can realize the nearby access of users because it has more nodes in the country. Moreover, CDN services can provide greater bandwidth, unlike our own application services, which provide limited bandwidth.