python packaging and running tips

1. Several methods of packaging

Python has several packaging methods for packaging scripts into executable files for running in environments without a Python interpreter. Here are some common Python packaging methods:

  1. Using pyinstaller: pyinstaller is a commonly used third-party tool that can package Python scripts into executable files. It can automatically detect a script’s dependencies and package them together into a standalone executable.

  2. Using py2exe: py2exe is another commonly used third-party tool for packaging Python scripts into executable files. It provides simple configuration options to package all dependencies into a folder and generate an executable file.

  3. Using cx_Freeze: cx_Freeze is a third-party library for packaging Python scripts into executable files. It can package the script and its dependencies into a directory structure and generate an executable file.

  4. Using Nuitka: Nuitka is a tool for compiling Python scripts into C/C++ code. It compiles scripts into standalone executable files without the need for a Python interpreter.

  5. Using py2app: py2app is a tool for packaging Python scripts into executable files on Mac. It can package scripts and their dependencies into an application package.

These tools have their own characteristics and usage, and you can choose one of them to package your Python script according to your own needs. Please note that the packaging process may involve some configuration and dependency processing. For specific usage, please refer to the official documentation of each tool.

Pyinstaller packaging

(1) Install pyinstaller first

pip install -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com pyinstaller

(2) Execute packaging again

# -w can hide console output
pyinstaller --clean -w -F filename.py

The packaged windows executable program will appear in the dist folder of the currently running program.

(3) Reduce the size

First install pipenv

pip install pipenv

After the installation is complete, you can create a virtual environment and enter packaging

#Create a virtual environment
pipenv install
#Enter virtual environment
pipenv shell
#Install module
Modules used in pip install gadget.py
#Packaged modules must also be installed
pip install pyinstaller
#Start packing
pyinstaller -Fw main.py

#Exit environment
exit

#Delete virtual environment
pipenv --rm

I found that UPX compression is not used. Download UPX and then use UPX compression to download the version you want at https://upx.github.io/.

The download is a compressed package. We only need to unzip it and copy the upx. Exe file inside to the same folder as pyinstaller. Exe (usually the Scripts folder). By default, Pyinstaller will search for the upx executable file from the execution directory. If upx is found, the generated executable file will be compressed.

Re-execute the pyinstaller packaging command. If it looks like the picture below, it means that the upx configuration is successful.

Nuitka Packaging

Nuitka is a Python to C++ compiler that achieves performance optimization by converting Python code into C++ code and compiling it into a binary executable file. Nuitka is able to provide higher performance and better performance than the native Python interpreter. volume. Its working principle can be briefly summarized as follows:

  1. Parsing and Transformation: Nuitka first parses the Python source code and transforms it into an intermediate representation. In the process, it converts the dynamic nature of Python into static C++ code.

  2. Code optimization: During the conversion process, Nuitka applies various optimization techniques to improve the performance of the generated C++ code. These optimizations include constant folding, code inlining, loop unrolling, etc.

  3. Generate C++ code: Once optimization is complete, Nuitka will generate C++ code and save it to a temporary directory on disk.

  4. Compile to executable: The generated C++ code is passed to a C++ compiler, such as GCC or Clang, to produce the final binary executable. During this process, the C++ compiler compiles the C++ code into machine code and links it with the necessary libraries.

Nuitka main program installation

Install Nuitka using pip: Nuitka is an optimizing Python compiler that can be installed using the following command:

pip install nuitka
Use Nuitka to package

During the packaging process, you will be prompted to install dependent environments. Just follow the prompts to install.

# --windows-mode closes the console window; --enable-plugin=tk-inter enables tk, otherwise the graphical interface cannot be displayed
nuitka --windows-disable-console --enable-plugin=tk-inter --output-dir=package --standalone your_script.py

Nuitka common commands:
--windows-disable-console #Do not display the console window
--windows-icon=ICON_PATH #Add ico
--nofollow-imports # All imports are not compiled and handed over to python3x.dll for execution.
--follow-import-to=need #need names the py folder you need to compile into C/C++
--mingw64 #mingw (official recommended compiler)
--standalone #Independent environment, this is necessary (otherwise it cannot be copied to others for use)
--windows-disable-console #No CMD control window
--output-dir=out #Generate file path
--show-progress #Show the progress of compilation, very intuitive
--show-memory #Display memory usage
--include-qt-plugins=sensible,styles #The style of PyQt will not change after packaging
--plugin-enable=qt-plugins #PyQt plug-ins that need to be loaded
--plugin-enable=tk-inter #Packaging the tkinter module is just needed
--plugin-enable=numpy #Packaging numpy, pandas, matplotlib modules is just needed
--plugin-enable=torch #Packaging pytorch is just needed
--plugin-enable=tensorflow #Packaging tensorflow is just needed
--windows-uac-admin=Windows #Users can install with administrator privileges
--onefile #Package into a single exe file
Nuitka depends on the installation of Winlibs_mingw environment

The packaging process requires winlibs_mingw, which provides various environments for compilation and packaging such as gcc. If the online download fails, you can go directly to the github agent to download, and then put it in the user directory ‘C:\Users\xxx\AppData\Local \\
uitka\\
uitka’, the packaging here relies on ‘ https://github.com/brechtsanders/winlibs_mingw/releases/download/11.3.0-14.0.3-10.0.0-msvcrt-r3/winlibs- x86_64-posix-seh-gcc-11.3.0-llvm-14.0.3-mingw-w64msvcrt-10.0.0-r3.zip ‘.

There is also dependence on ccache. The version of packaged dependencies here is’ https://github.com/ccache/ccache/releases/download/v4.6/ccache-4.6-windows-32.zip. ccache is a tool used for cache compilation. Tools output by the compiler can improve the speed of compilation. When compiling large projects, it often takes a lot of time to recompile already compiled files. By caching the compiler output, Ccache can directly use the cache when compiling the same file next time, avoiding the recompilation process, thus saving compilation time.

Nuitka depends on the environment Ccache

Ccache can act as a compiler front-end, forwarding compilation commands to the actual compiler. At compile time, ccache checks whether the input file has been compiled and caches the compiler’s output. If the file has already been compiled and has not changed, ccache will use the cached output directly without calling the actual compiler. Only when the file has changed or the cache has expired will ccache call the actual compiler to recompile the file.

Using ccache can significantly speed up compilation, especially when compiling the same file multiple times. It is especially useful for developers working on large projects or those who frequently modify their code.

Nuitka depends on the environment Depends

If you still need to run it independently without relying on the python environment, you also need to download ‘https://dependencywalker.com/depends22_x64.zip’.

File deletion after packaging is completed

Both .cmd and .Build can be deleted after packaging.

.cmd file, a small script used to run a packaged executable file on a Windows system. This script is responsible for setting the necessary environment variables and launching the packaged executable.

When you use Nuitka to package Python code, in addition to generating a standalone executable file, it will also generate a .cmd file for the executable file to make it easier to run the file on Windows systems .

You can execute a packaged executable by double-clicking or running the .cmd file in a command prompt without having to manually set environment variables or write additional scripts. The .cmd file, when run, will ensure that the environment is set up correctly and then launch the packaged executable.

Possible problems
Orderedset warning
Nuitka:WARNING: Using very slow fallback for ordered sets, please install 'orderedset' PyPI package
Nuitka:WARNING: for best Python compile time performance.

This warning message is a reminder from Nuitka. It recommends that you install the orderedset PyPI package to obtain the best Python compile time performance. If Microsoft Visual Studio is not installed, it is recommended not to resolve the warning. , need to install related environment, the file is very large.

orderedset is a Python library for ordered sets. In some cases, Nuitka may use a slower alternative to handling ordered sets instead of using the orderedset library. This may result in longer compilation times.

To solve this problem, first install Microsoft Visual C++ Build Tools: https://visualstudio.microsoft.com/visual-cpp-build-tools/
After installation, make sure you have added it to your system’s environment variables so that the compiler can find the necessary files. Then follow the steps below to install the orderedset library and execute it from the command line with administrator rights:

pip install wheel

pip install orderedset
Tk-inter warning

Nuitka-Plugins: WARNING: Use –enable-plugin=tk-inter’ for: Tkinter needs TCL included.

This warning means that you need to pass --enable-plugin=tk-inter as a parameter to Nuitka to enable the Tkinter plugin. This is because Tkinter requires support from the TCL library.

You can try using the following command to package your program:

nuitka --enable-plugin=tk-inter your_script.py

This will enable the Tkinter plugin and package your program.

If the TCL library is not installed, you can view and install the TCL library. Official website: https://www.tcl.tk/ Download and install it. Generally, python has it by default.

import tkinter as tk

Enter the following code to print the TCL version used by Tkinter:

print(tk.Tcl().eval('info patchlevel'))
# My version is 8.6.12

Remove the console window display when the py script is running

If you want to run the py or .pyc file without displaying the console window, you can rename the file to .pyw. Various packaging tools will have separate console option controls, such as nuitka --windows-disable-console , which will not be described here.

The .pyw file is a console-less version of Python that runs in the background without displaying any window or console output.

You can rename your .pyc file to .pyw using the following steps:

  1. Find the .pyc file you want to rename, for example script.pyc.

  2. Change the file extension .pyc to .pyw, i.e. change the file name to script.pyw.

  3. Double-click the .pyw file and it will run in the background without displaying the console window.

Note that renaming the .pyc file to .pyw simply removes the console window but does not modify or encrypt the code in any way. Any experienced user can still access and modify the code. If more advanced protection is required, consider using other encryption tools or packaging the code as an executable.