This error originates from a subprocess Termux matplotlib pandas compilation error and jupyter installation issues

Article directory

• • • Notice
  • Problem Description
  • • • ~~Question complaints~~
    • Solution 3 steps
    • • version record
    • 1 Clear pip cache
    • 2 Successful installation of matplotlib
    • • matplotlib completes dependent commands
    • 3 Successful installation of pandas
    • Supplementary question 1: `pip install jupyter` error:
    • • solution
    • Supplementary question 3: There is a conflict between openblas and libopenblas, causing libopenblas to fail to be installed.
    • • ~~Take a detour~~
      • Solution
      • • refer to

Attention

This is just for memo, so it is not fully reproduced.

If you encounter problems during the reference process, please try to solve them yourself – but you are very welcome to post problems, solutions, key additions to the article, corrections, etc. in the comment area, such as this This article is to supplement the errors encountered during Termux installation and configuration + python environment configuration.

Problem description

Refer to an article Termux installation and configuration + python environment configuration when configuring Termux.

Encountered problem 1:
matplotlib pandas installation failed.
Supplementary question 2:
jupyter installation failed. (see end of article)
Supplementary question 3:
openblas conflicts with libopenblas, causing libopenblas to fail to be installed

The common point between the installation of the two tools is that the error reporting point is in the Build compilation phase.

Installing Building Dependencies ... \

It took a long time to build, and then suddenly there were thousands of lines of errors, the content of which I couldn’t understand, and various dependency issues in the compilation chain.
The basic thing to notice is (part of the tail)

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Complaints

Of course it’s not our problem, it’s just that these two things are too difficult to install. Dependency issues are annoying.
Maybe the official English Installation Guide will not make mistakes, but as long as you read it written by someone else, there will be mistakes in the configuration article, right?

There are articles in China recording their own solutions, and there are many Q&A strings on the Internet about errors during their installation phase.

Many people complain that the available dependencies for pip to install them are very unstable. Basically, it works well this month, but when the component is updated next month, it won’t work again. I don’t know if it’s a problem with the developer team or a problem with the quality of online configuration articles.

Zhihu related articles also explain how termux was successfully installed in 23 years, but some articles only explain how to get the results from the installation in a pure environment. Do you even want python to be clean? ? ? It is impossible for me to uninstall python and lose all the modules that were configured and installed before. I won’t do it!

Solution 3 steps

Version record

Successfully installed version:

python 3.11.6
pip 23.3.1
numpy 1.23.2
matplotlib 3.8.1
\t
Cython 3.0.5
pytz 2023.3.post1
pandas 2.1.2

1 Clear pip cache

rm -rf ~/.cache/pip

2 Successful installation of matplotlib

• direct cause:
  There are differences in matplotlib installation dependencies between 23 and 21.

For the matplotlib compilation error, the solution I found myself can be found in the answer of Alekc74 in the discussion section of https://github.com/termux/termux-app/discussions/3444#discussioncomment-6394303.

The discussion here also contains various opinions, reasons, and others about configuring matplotlib (I didn’t look into it in detail).

This resolved it(wheel error for contourpy ninja dependency)

In addition, I added two lines myself pkg install matplotlib and pkg uninstall matplotlib
Finally, the problem was somehow solved and matplotlib was successfully installed.

matplotlib completes dependent commands

Pre-command: Python package installation and configuration part based on Termux installation and configuration + python environment configuration
Configure the pre-dependency package of matplotlib itself, that is

# Configure pypi source
pip install pip -U
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple
# Dependencies
pip install wheel
pip install setuptools --upgrade

# 2 Python package installation
# Install numpy and scipy
# Add third-party warehouse
curl -LO https://its-pointless.github.io/setup-pointless-repo.sh
bash setup-pointless-repo.sh

# Install numpy scipy from repository
pkg install -y numpy scipy

# Install lxml
# Install lxml dependencies
pkg install -y libxml2 libxslt

# Install lxml
pip install lxml
Install Pillow


#Install pillow's dependencies
pkg install -y libjpeg-turbo libtiff
pkg install -y littlecms tk libwebp libsodium

# Install pillow
pip install pillow

# Install matplotlib
#Install matplotlib dependencies
pkg install -y freetype libpng
pip install kiwisolver cycler
pip install pyparsing python-dateutil

# At this time, an error will be reported when executing the following, that’s right.
# pip install matplotlib

Then use the following command (It may not be complete, because this has relied on other references before but failed)

pip install wheel

pkg i ninja automake cmake binutils patchelf

pkg i libjpeg-turbo

pip install pillow

pip install setuptools==59.8.0

CFLAGS=-Wno-implicit-function-declaration MATHLIB=m pip install numpy==1.23.2

pip install meson

pip install meson-python

pip install pybind11

pip install --no-build-isolation pip install contourpy

pkg install matplotlib
pkg uninstall matplotlib

pip install matplotlib

Afterwards, matplotlib was installed and compiled successfully.

3 Successful installation of pandas

The reference is the original text, but we have already solved the dependency problem in step 2. Right now

matplotlib completes dependent commands
→

# Install pandas
# Install pandas dependencies
pip install -y pytz cython

# Install pandas
export CFLAGS="-Wno-deprecated-declarations -Wno-unreachable-code"
pip install pandas

pandas can also be installed normally.

Supplementary question 1: pip install jupyter Error:

If you also encounter pip install jupyter, the error is reported as follows (the first half of the error is reported. The complete error report is often too long, and the further it goes, the less meaningful it is)

× pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [58 lines of output]
      Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
      Collecting maturin<2.0,>=1.0
        Using cached https://pypi.tuna.tsinghua.edu.cn/packages/d6/a3/42ff26b2b3011ada67c73db32a9ccf9fa18d459cbef8ed6eefc3b283ee60/maturin-1.3.1.tar.gz (176 kB) Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Building wheels for collected packages: maturin
        Building wheel for maturin (pyproject.toml): started
        Building wheel for maturin (pyproject.toml): finished with status 'error' error: subprocess-exited-with-error

        × Building wheel for maturin (pyproject.toml) did not run successfully. │ exit code: 1
        ╰─> [35 lines of output]
            /data/data/com.termux/files/usr/tmp/pip-build-env-66jbi47_/overlay/lib/python3.11/site-packages/setuptools/config/_apply_pyprojecttoml.py:80: SetuptoolsWarning: `install_requires` overwritten in `pyproject.toml` (dependencies)
              corresp(dist, value, root_dir) running bdist_wheel
            running build
            running build_py
            creating build creating build/lib.linux-aarch64-cpython-311

            creating build/lib.linux-aarch64-cpython-311/maturin

            copying maturin/__init__.py -> build/lib.linux-aarch64-cpython-311/maturin

            copying maturin/__main__.py -> build/lib.linux-aarch64-cpython-311/maturin

            copying maturin/import_hook.py -> build/lib.linux-aarch64-cpython-311/maturin

            running egg_info
            creating maturin.egg-info
            writing maturin.egg-info/PKG-INFO
            writing dependency_links to maturin.egg-info/dependency_links.txt

            writing requirements to maturin.egg-info/requires.txt

            writing top-level names to maturin.egg-info/top_level.txt

            writing manifest file 'maturin.egg-info/SOURCES.txt' reading manifest file 'maturin.egg-info/SOURCES.txt'

            reading manifest template 'MANIFEST.in'
            warning: no files found matching '*.json' under directory 'src/python_interpreter'

            writing manifest file 'maturin.egg-info/SOURCES.txt'


            running build_ext running build_rust


            error: can't find Rust compiler

You will find that the first paragraph

 Building wheel for maturin (pyproject.toml): finished with status 'error'
error: subprocess-exited-with-error

This indicates a compilation error of a component called maturin. But we don’t understand this thing, so let’s try to read on.

The second paragraph contains very important information, starting from

× Building wheel for maturin (pyproject.toml) did not run successfully.

...
error: can't find Rust compiler

error: can't find Rust compiler
ok, this sentence tells us, error: Rust compiler not found.
It’s strange, what if we try installing a Rust environment and then running it? After trying it, it worked!

Solution

# Install rust environment
pkg install rust

# Then execute it again
pip install jupyter

# You can check the effect after success
jupyter notebook

Supplementary question 3: There is a conflict between openblas and libopenblas, causing libopenblas to fail to be installed

It happens when installing jupyter front-end libzmq or the like.
This problem happened inexplicably. The situation is as follows:

# First execute the installation command
# Command A
~ pkg install libzmq

# When I encountered an error, the system suggested that I execute `apt --fix-broken install`, so I did so.

# Command B
~ apt --fix-broken install

# An error occurs
...
dpkg: error processing archive /data/data/com.termux/cache/apt/archives/libopenblas_0.3.24_aarch64.deb (--unpack): trying to overwrite '/data/data/com.termux/files/usr/ lib/libopenblas.so', which is also in package openblas 0.3.15
...

The history of termux has been cleared, so the complete error message cannot be found. But the solution comes from this sentence.

When I execute command B, I can see that it probably means that apt wants to install a library with version 0.3.24 of libopenblas for me. When it goes to the unpack installation link when the file it wants to write to /data/data/com.termux/files/usr/lib/libopenblas.so is replaced by an existing version of openblas It is held by the library of 0.3.15.
So dpkg alerts me that it is trying to overwrite this file.

View the origin of openblas

pkg list-all | grep openblas

The result is

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libopenblas/stable 0.3.24 aarch64
openblas-static/termux 0.3.15 aarch64
openblas/termux,now 0.3.15 aarch64 [installed,automatic]

It seems that it is a dependency that was automatically installed at some point. Maybe it comes with termux.

Take a detour

I took a detour and tried to uninstall openblas and then install a new one. The trouble was that the already installed matplotlib and python-numpy (this It seems that it comes with termux, not numpy) and relies on this. Uninstalling them separately will not work, but uninstalling both together will work.

But it’s not necessary
Because I found that errors were still reported, and they were all about the overwrite problem above, so I finally returned to the problem itself.

Solution

Reference

Found the article apt – dpkg error: “trying to overwrite file, which is also in…” – Ask Ubuntu, where

• Solution 1 is relatively simple and crude – Allow dpkg to overwrite existing package files, which may create the risk of system errors;
• Solution 2 is to report a bug report, but this will have to wait, and the difficulty of the problem will become how to submit a bug report correctly in English.

1

2

In the end, I still took the risk and adopted solution 1. It can currently solve my problem, but it is really a helpless move.

Please give me some advice from experienced experts as to what is a more appropriate approach.

The solution is as follows:

# Allow the libopenblas_0.3.24_aarch64.deb installation package to overwrite the same file (/data/data/com.termux/files/usr/lib/libopenblas.so) as the existing package

dpkg -i --force-overwrite /data/data/com.termux/cache/apt/archives/libopenblas_0.3.24_aarch64.deb


# After overwriting, use apt to repair it again
apt --fix-broken install

# At this time, you can install libzmq and other jupyter pre-dependencies again.
pkg install libzmq
...
# (install jupyter and other subsequent steps)
pip install jupyter

View the installation results:

# check
pkg list-all | grep openblas

The output is as follows. It can be seen that libopenblas has been installed at this time.

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libopenblas/stable,now 0.3.24 aarch64 [installed]
openblas-static/termux 0.3.15 aarch64
openblas/termux,now 0.3.15 aarch64 [installed,automatic]

But again, I don’t understand the consequences of doing this. I just have to temporarily solve the problem. As the reference said, there are risks.
Before commenting specifically, please consider referring to the command using dpkg override.