Installation and deployment of flume in virtual machines (super detailed)

Table of Contents

Installation and deployment of flume in virtual machines (super detailed)

1. Download the Flume package (it is recommended to use version 1.9.0 here, the reason: you will know after you try it with version 1.11.0)

1. Download steps

2. Upload:

2. Unzip the flume installation package

1. Unzip: Unzip where the compressed package is placed

2. Create soft links

3. Configure environment variables

1. Modify the configuration file

4. Modify the configuration file

1. Switch to the flume folder to configure the file

2. Copy and change the configuration file name

3. Modify the configuration file

4. Add java environment variables

5. Check the flume version

Installation and deployment of flume in virtual machines (super detailed)

1. Download steps

Website: http://flume.apache.org

Click directly to download the compressed package

2. Upload:
  • You can directly use rz to select the compressed package to upload (but it is very slow)
  • We choose to upload directly below, the speed is so fast that you can’t imagine (note: be sure to go to the location of the file you want to upload, otherwise you will not know where it has been uploaded)

After uploading, let’s check the file again and it’s in it.

To view all files in a folder we can use:

ll: View the file in detail

ls: Display only the file name view

  • You can also view it directly like this:
ll /opt/tage/

Among them, /opt/tage/ is the path of the folder where you want to view the file

2. Unzip the flume installation package

1. Decompress: decompress where the compressed package is placed

tar -zxvf apache-flume-1.9.0-bin.tar.gz -C ../server/ (.. means return to the previous level)

The result after decompression (.. means return to the previous level)

(1) What is a soft link: A soft link is a commonly used command in Linux. Its function is to establish a synchronous link to a certain file in another location. It is equivalent to a pointer in the C language. The established link directly points to the source file. Address, soft link will not occupy additional resources. When the same file needs to be used in multiple locations, soft link will be used.

(2) The purpose of creating a soft link is to facilitate the use of Flume commands. By creating a soft link, you can directly run the Flume command in any directory without having to switch to the directory where Flume is located every time, which can improve the convenience of use. sex and efficiency.

(3) Create soft link command: ln -s [source file or directory] [target file or directory]
Other parameters: -i interactive mode.
-s soft link (symbolic link). If you do not add the “-s” option, a hard link file will be created
-b delete, overwrite previously established links
-f: Force. If the target file already exists, delete the target file and then create the link file.

(4) Cancel the soft link: There are two methods, you can use the rm command or the unlink command.

Method 1: Use the rm command to cancel the soft link. Run command: rm soft link file path

Example: If the soft link file is flume and the path is /opt/server/flume, then the command to cancel the soft link is:

rm /opt/server/flume

Method 2: Use the unlink command to cancel the soft link. Run command: unlink soft link file path

Example: If the soft link file is flume and the path is /opt/server/flume, then the command to cancel the soft link is:

unlink /opt/server/flume

(5) Delete soft link: Just delete the link name directly rm -rf soft link name

Example: If the soft link file is flume, we first go to the path where flume is located and run the command:

Note: If you do not add f, you will be asked whether to delete the file. If you add f, you will be deleted directly without asking.

rm -rf flume

Notice:

① Directories can only create soft links
② The directory creation link must use an absolute path. The relative path creation will fail and an error message will be displayed: There are too many levels of symbolic connections.
③ Modified files in the link target directory will be synchronously changed in the source file directory.

Achieve results:

3. Configure environment variables

1. Modify configuration file

vim /etc/profile can be modified in any directory

Press the i key on your keyboard to edit

export FLUME_HOME=/opt/server/flume
export PATH=$PATH:$FLUME_HOME/bin

Save and exit Press the Esc key on the keyboard to exit editing, enter :wq and press Enter to save and exit.

Notice:

After configuring, be sure to restart the environment variable source /etc/profile

Otherwise, you will not be able to find the flume version later.

4. Modify the configuration file

1. Switch to the flume folder to configure the file
cd conf/

2. Copy and change the configuration file name
cp flume-env.sh.template flume-env.sh

Check the results after execution

3. Modify configuration file

Note that the path is under cd /opt/server/flume/conf

vim flume-env.sh

4. Add java environment variables
export JAVA_HOME=/opt/server/jdk

Save and exit Press the Esc key on the keyboard to exit editing, enter :wq and press Enter to save and exit.

5. View flume version
flume-ng version

In this way, Flume is installed! If it helps you, please give me a follow~

The knowledge points of the article match the official knowledge files, and you can further learn related knowledge. Java Skill TreeHomepageOverview 135353 people are learning the system