Python error: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+

Article directory

  • Python error: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1 +
    • 1. Problem description
    • 2. Problem analysis
    • 3. Solutions
      • After upgrading openssl, still import urllib3 error
        • Idea 1: Recompile python
        • Idea 2: Specify the Python interpreter to link to the new version of OpenSSL without recompiling Python
    • 4. The relationship between python compilation and openssl? Do I need to compile and upgrade python every time openssl leaks?
    • 5. How does python decide whether to link to openssl dynamically or statically?
        • Option to check if static linking was enabled during Python compilation
        • Determine exactly which OpenSSL library Python uses
    • Sixth, compile python3.8 error: Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host()

python error: ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1 +

1. Problem description

python3 installed requests (pip install requests)

2. Problem Analysis

Note: The requests package introduces urllib3, and the new version of urllib3 requires OpenSSL 1.1.1 + or above, otherwise an error will be reported:
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1 + ,
Currently the ‘ssl’ module is compiled with ‘OpenSSL 1.0.2k-fips 26 Jan 2017’. See: https://github.com/urllib3/urllib3/issues/2168

3. Solution

Need to upgrade openssl

Download and compile openssl

wget --no-check-certificate https://www.openssl.org/source/openssl-1.1.1t.tar.gz
tar -zxvf openssl-1.1.1t.tar.gz
cd openssl-1.1.1t/
./config --prefix=/usr/local/my_openssl
make
make install

tar -zxf ./my_openssl.tar.gz -C /usr/local
echo "copy my_openssl done"
sleep 1

mv /usr/bin/openssl /usr/bin/oldopenssl
ln -s /usr/local/my_openssl/bin/openssl /usr/bin/openssl
ln -s /usr/local/my_openssl/lib/libssl.so.1.1 /usr/lib64/
ln -s /usr/local/my_openssl/lib/libcrypto.so.1.1 /usr/lib64/
echo "change openssl done"

After upgrading openssl, import urllib3 still reports an error

If you have confirmed that the OpenSSL on your system shows up as a newer version, but Python is still using the older version, it may be because the Python interpreter is not properly linked to the newer version of OpenSSL.

Idea 1: Recompile python

Recompiling Python is one way to solve the problem that the Python interpreter is not properly linked to the new version of OpenSSL. This ensures that Python uses the new version of the OpenSSL library.

Configure compilation options: Open a command-line terminal, switch to the directory of the Python source code, and run the following command to configure compilation options:

./configure --with-openssl=/path/to/openssl

–with-ssl: Indicates that the OpenSSL library installed in the system is used.
–with-openssl: Indicates that the built-in OpenSSL library is used.

Replace /path/to/openssl with the path where the new version of the OpenSSL library is installed. This option tells the Python compiler to use the new version of OpenSSL during compilation.

Compile and Install: Run the following commands in a command-line terminal to compile and install the reconfigured Python:

make
sudo make install

This will compile the Python source code and install it into the system. Note that you may need to enter the administrator password when executing sudo make install.

Verify the installation results: After the recompilation and installation is complete, you can verify that the Python interpreter is linked against the new version of OpenSSL by running the following command:

python -c "import ssl; print(ssl.OPENSSL_VERSION)"

Use –with-openssl Pro test is available! recommended~

Idea 2: Specify the Python interpreter to link to the new version of OpenSSL without recompiling Python

If you have installed a new version of OpenSSL and want to specify that the Python interpreter is linked to this new version of OpenSSL without recompiling Python, you can set the environment variable LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (macOS) to the new version of OpenSSL The library path is added to the runtime environment of the Python interpreter.

For example, assuming a new version of the OpenSSL library is located in /usr/local/ssl/lib, you can do the following:

export LD_LIBRARY_PATH=/usr/local/ssl/lib:$LD_LIBRARY_PATH

In the Python interactive environment, execute the following code to view the path of the OpenSSL library used in Python:

import ssl
print(ssl. OPENSSL_VERSION)
print(ssl._ssl.__file__)

I am testing here, no matter how I change LD_LIBRARY_PATH, I always refer to the python installation directory: lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so

4. The relationship between python compilation and openssl? Do I need to compile and upgrade python every time openssl leaks?

There is a close relationship between Python and OpenSSL. OpenSSL is an open source cryptography library that provides functions for implementing SSL/TLS protocols and encryption algorithms. Python’s SSL module is built on the OpenSSL library, which provides support for the SSL/TLS protocol, enabling Python programs to perform operations such as secure network communication, encryption and decryption.

Normally, you only need to upgrade OpenSSL itself, and Python’s SSL module will automatically use the latest version of the OpenSSL library. You don’t need to recompile Python, just make sure you have an updated OpenSSL library installed on your system, and Python will automatically link and use that library.

  1. Python’s SSL module realizes its functions by calling the dynamic link library (or shared library) provided by the operating system. These dynamic link libraries contain the implementation of SSL/TLS protocol and encryption algorithm. In most operating systems, the OpenSSL library is considered a system-level library, so Python’s SSL module will link to the OpenSSL library installed in the operating system by default.

  2. When you upgrade the OpenSSL library in your system, the corresponding dynamic link library files are usually updated. This means that Python’s SSL module can automatically load the latest version of OpenSSL functionality from an updated dynamic link library.

  3. Python’s SSL module dynamically loads the required OpenSSL functions and symbols at runtime. It looks in the operating system’s DLL search path to find and load the correct version of the OpenSSL library. If a new version of the OpenSSL library is present in the system’s dynamic library search path, Python’s SSL module will automatically link against and use that library.

It should be noted that the automatic linking and using the latest version of the OpenSSL library described above only works with the Python distribution provided by the operating system. If you are using a self-compiled Python, you need to ensure that you correctly link to the updated OpenSSL library when compiling and installing Python.

If you are using the source code package, you can configure Python’s compilation options by passing the –with-openssl parameter when running the ./configure command and specifying the path where the new version of the OpenSSL library is located. For example:

./configure --with-openssl=/path/to/new/openssl

By setting the LD_LIBRARY_PATH environment variable, you can specify the search path for the OpenSSL library without recompiling Python. But this approach only works for applications that are linked against OpenSSL at runtime, and may not work if Python has been linked against an older version of OpenSSL when statically linked.

If Python was statically linked against an older version of OpenSSL, simply setting the LD_LIBRARY_PATH environment variable may not cause Python to use a newer version of OpenSSL.

In the case of static linking, OpenSSL’s code and functions are embedded into the resulting executable at compile time, rather than dynamically loaded at runtime. This means that regardless of setting the LD_LIBRARY_PATH environment variable, Python will still use features from older versions of OpenSSL that have been embedded.

Go to your Python installation directory and look for files named libpython*.a. For example, for Python 3.8, the filename might be libpython3.8.a.

 find / -name libpython*.a

If you found this file, then Python is statically linked to OpenSSL. If this file is not found, then Python is dynamically linked to OpenSSL at runtime.

If you find that Python is statically linked to OpenSSL, then simply setting the LD_LIBRARY_PATH environment variable may not cause Python to use the new version of OpenSSL. In this case, you may need to recompile Python and link against the new version of OpenSSL to ensure that Python uses the latest OpenSSL features.

5. How does python decide whether to link to openssl dynamically or statically?

In most cases, Python is dynamically linked with OpenSSL. This means that Python loads the required OpenSSL dynamic library from the system at runtime.

However, there are some special cases that may cause Python to statically link to OpenSSL:

Custom compilation: If you use custom compilation options or manually compile Python, it may cause static linking to OpenSSL. In this case, you need to check the options during compilation and make sure that static linking is enabled.

Check if the static linking option is enabled during Python compilation

Enter the Python installation directory, usually the bin directory in /usr/local/ or /usr/, the path is similar to /usr/local/bin/ or /usr/bin/

Execute the following command to view compilation options:

./python3-config --ldflags

This command will display the linker flags used when compiling Python.

If you see -lssl or -lcrypto in the output, it means that Python was compiled with OpenSSL linkage enabled. This means that Python is dynamically linked against OpenSSL instead of statically linked.

If you see -l:libssl.a or -l:libcrypto.a in the output, then Python was compiled with static linking enabled. This means that Python is statically linked against OpenSSL.

Determine which OpenSSL library Python uses

In the Python interactive environment, execute the following code to view the path of the OpenSSL library used in Python:

import ssl
print(ssl. OPENSSL_VERSION)
print(ssl._ssl.__file__)

This will print out the version of OpenSSL used by Python and the path to the library file actually loaded.

There are multiple libssl library versions installed in the system. This can cause the Python interpreter to not properly link to the required libssl library.

You can execute the following command to check whether the libssl library is installed:

 ldconfig -p | grep libssl

6. Compile python3.8 error: Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host()

Compile python3.8 error: Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().
LibreSSL 2.6.4 and earlier do not provide the necessary APIs, https://github.com/libressl-portable/portable/issues/381

problem analysis:
My local openssl has been updated, but this error is still reported, even if I use the python compilation option to specify the path of my opensslssl –with-openssl=/usr/local/my_openssl/lib/

Need to confirm whether the installed OpenSSL has added the enable-ssl3 configuration during compilation

./config enable-ssl3 shared --prefix=/usr/local/openssl
make
sudo make install

The reason for adding enable-ssl3 configuration when compiling OpenSSL is:

After OpenSSL 1.1.0, the SSLv3 protocol is disabled by default, and only TLSv1.0 and above are supported. But Python still needs SSLv3 support when building ssl modules. So when compiling OpenSSL, you need to explicitly add the enable-ssl3 configuration to enable the SSLv3 protocol.

If you don’t add this configuration, you will get the error you encountered when compiling Python:

Could not build the ssl module!
Python requires an OpenSSL 1.0.2 or 1.1 compatible libssl with X509_VERIFY_PARAM_set1_host().

This is because Python tries to call OpenSSL’s X509_VERIFY_PARAM_set1_host() function, but this function was introduced in OpenSSL version 1.0.2. The OpenSSL 1.1.0 and above versions compiled by default disable SSLv3, so Python cannot build the ssl module.

  1. It will enable the SSLv3 protocol, so that Python can call OpenSSL’s SSLv3 related functions to build the ssl module.
  2. It builds an earlier compatible version of OpenSSL so that Python can call functions like X509_VERIFY_PARAM_set1_host().

Note --with-openssl=/usr/local/openssl should be used for python compilation parameters to specify the main directory of OpenSSL, not subdirectories such as lib. The configure script of Python will automatically search the include subdirectory and lib subdirectory under this path.

Pro-test passed!