Build environment: windows10, VMware16.2.3, centOS7.9, jdk-8u162-linux-x64.tar.gz, hadoop3.1.0 Links to Hadoop for various versions of Hadoop jdk8 download link jdk 1. Basic configuration preparations 1.1 Static IP address and other configurations Note: If you are prompted for insufficient permissions, you need to use root permissions. 1.1.1 Non-clone machine configuration static IP address ####1.1.2 Configure a static […]
Tag: distribute
[Distributed] tensorflow 1 distributed code practice and explanation; running 2 distributed worker threads on a single node
tensorflow.python.framework.errors_impl.UnknowError: Could not start gRPC server 1. tf distributed A computer = server = server is a node that contains multiple GPUs. First of all, the distributed method is to let the GPUs on multiple computers work together. Distributed work is divided into two parts, parameter server (ps) and worker. PS and worker look familiar, […]
Hadoop fully distributed construction
Foreword There are always various problems encountered when configuring a Hadoop cluster. Here is a complete summary of the complete Hadoop fully distributed configuration process, hoping to help more people. Virtual machine preparation A distributed cluster requires three virtual machines. I won’t explain too much here, just clone them directly. 1. Modify the host name […]
How to design a reconciliation system that can handle tens of millions of data volumes, be distributed and highly available?
The path to growth as a programmer Internet/Programmer/Technology/Data Sharing focus on This article should take approximately 11 minutes to read. From: juejin.cn/post/7259949655583506490 1Background At present, online business volume is increasing day by day, with daily orders exceeding tens of millions, capital flows are large, and capital security has become a key concern. In order to […]
Implementing distributed locks based on redis
Article directory Implementing distributed locks based on redis Basic implementation Anti-deadlock Prevent accidental deletion Atomicity cannot be guaranteed in high concurrency scenarios. Use Lua to ensure deletion atomicity Encapsulate the redis lock into a method Implementing distributed locks based on redis Basic implementation With the help of the command setnx(key, value) in redis, if the […]
Use Docker to deploy the open source distributed task scheduling system DolphinScheduler
Blog homepage: The little sheep has insomnia. Series of columns: “C Language” strong> 《Data Structure》 《Linux》《Cpolar》 Thank you everyone for likingfavorites?comments Use Docker to deploy the open source distributed task scheduling system DolphinScheduler A few days ago, I discovered a giant artificial intelligence learning website. It is easy to understand and humorous. I couldn’t help […]
Quickly build the open source distributed task scheduling system DolphinScheduler and access it remotely
Use Docker to deploy the open source distributed task scheduling system DolphinScheduler Article directory Use Docker to deploy the open source distributed task scheduling system DolphinScheduler Preface 1. Install and deploy DolphinScheduler 1.1 Start the service 2. Log in to the DolphinScheduler interface 3. Install intranet penetration tools 4. Configure Dolphin Scheduler public network address […]
Research on distributed model predictive control of heterogeneous vehicle platoon under unidirectional topology (Matlab code implementation)
Welcome to this blog Advantages of bloggers:Blog content should be as thoughtful and logical as possible for the convenience of readers. Motto:He who travels a hundred miles is half as good as ninety. The directory of this article is as follows: Table of Contents 1 Overview 2 Operation results 3 References 4 Matlab code, data, […]
Distributed lock 2 in redisson
Fair Lock The Redisson distributed reentrant fair lock based on Redis is also a RLock object that implements the java.util.concurrent.locks.Lock interface. It also provides asynchronous (Async), reflective (Reactive) and RxJava2 standard interfaces. It ensures that when multiple Redisson client threads request locks at the same time, priority will be given to the thread that made […]
Minio distributed cluster deployment (4 nodes 4 clusters)
Minio distributed cluster deployment (4 nodes 4 clusters) Centos7.6 Minio Minio distributed cluster deployment (4 nodes 4 clusters) 1.Environment preparation 2-Add new disk and mount it 3. Install minio cluster 4. Configure systemctl system service 5. Login page testing 1. Environment preparation 4 servers with multiple hard drives Node data directory (cannot be under /root) […]