While creating a statically linked embeded-Python-executable is relatively easy on Linux (see for example this SO-post) it is much more complicated on Windows. And you probably don't want to do it.
Also the result might not be what one would expect: Due to limitations of dlls compared to Linux' shared objects, the resulting static python-version will not be able to use/load any other c-extensions, as the one backed-in during the compile/link time.
I also would not recommend to switch from vcruntime-dll to its static version - it would only make sense, when everything (exe, c-extensions, other dll which depend on vcruntime) is statically linked into one huge executable.
The first stumbling block: While on Linux python distributions often have a static Python-library already shipped, Windows distributions have only the dll, which cannot be statically linked in.
Thus one needs to build a static library on Windows. A good starting point is this link.
After downloading the source for the right Python version (git clone --depth=1 --branch v3.8.0 https://github.com/python/cpython.git) you can go to cpythonPCBuild and build cpython as explained in documentation (which might vary from version to version).
In my case it was
cd cpython/PCbuild
.uild.bat -e -p x64
No we have a functioning Python3.8 installation, which can be found in cpython/PCbuild/amd64. Create folder cpython/PCbuild/static_amd64 and add the following pyx-file:
#hello.pyx
print("I'm standalone")
copy python38.dll to static_amd64 for the time being.
Now let's build our program with embedded python interpreter:
cython --embed -3 hello.pyx
"C:Program Files (x86)Microsoft Visual Studio 14.0VCvcvarsall.bat" x64
cl /c hello.c /Fohello.obj /nologo /Ox /W3 /GL /DNDEBUG /MD -I<path_to_code>cpythoninclude -I<path_to_code>cpythonPC
link hello.obj python38.lib /OUT:hello_prog.exe /nologo "/LIBPATH:<path_to_code>cpythonPCbuildamd64"
After the start, hello_prog.exe lies to us, as it is not really standalone. The good news is: it finds the Python-installation which is needed as described for example here.
Now let's create a static python38-library. For that we open pcbuild.sln in cpython/PCbuild-folder and change pythoncore-project's setting to produce static library in PCbuildamd64_static-folder. Rebuild it.
Now we can build the embedded-python-exe:
cl /c hello.c /Fohello.obj /D "Py_NO_ENABLE_SHARED" /nologo /Ox /W3 /GL /DNDEBUG /MD -I<path_to_code>cpythoninclude -I<path_to_code>cpythonPC
link hello.obj python38.lib "version.lib" "shlwapi.lib" "ws2_32.lib" "advapi32.lib" "shell32.lib" "ole32.lib" "oleaut32.lib" "kernel32.lib" "user32.lib" "gdi32.lib" "winspool.lib" "comdlg32.lib" "uuid.lib" "odbc32.lib" "odbccp32.lib" /OUT:hello_prog.exe /nologo "/LIBPATH:<path_to_code>cpythonPCbuildstatic_amd64"
Compared to the build against dll we had to change the following:
Py_NO_ENABLE_SHARED (i.e. /D "Py_NO_ENABLE_SHARED") is added to preprocessor definitions, otherwise the linker will look for wrong symbols.- the Windows dependencies (i.e.
version.lib and so on) which were brought by the python-dll need to be passed to the linker explicitly now (this can be looked up in the linker-command-line of the pythoncore-project).
- lib path show to the static folder, i.e.
"/LIBPATH:<path_to_code>cpythonPCbuildstatic_amd64" now.
- there might be other smaller troubles (different optimization levels, link-time-code generation, disabling whole-program-optimization and so on) depending on your exact tool chain.
We can delete the python38.dll from static_amd64 now and the hello_prog.exe still works.
On Linux, this would be "mission accomplished", on Windows we are just at the beginning...
Make sure that cpython-folder has a DLLs-folder with all right pyd-files, otherwise create and copy all pyd-files from PCbuild/amd64-folder.
Let's make our pyx-file a little bit more complicated:
import _decimal
print("I'm standalone")
_decimal is a fast implementation of the decimal-module which is a C-extension and can be found in the DLL-folder.
After cythonizing and building it, running hello_prog.exe leads to the following error message:
import _decimal
ImportError: DLL load failed while importing _decimal: The specified module could not be found.
The problem is easy to find:
dumpbin /DEPENDENTS ../amd64/_decimal.pyd
...
python38.dll
...
The extensions of our installation still depends on the python-dll. Let's rebuild them against the static library - we need to change library path to from amd64 to static_amd64, to add preprocessor define Py_NO_ENABLE_SHARED and all missing windows-libraries (i.e ""version.lib"& Co.) and adding /EXPORT:PyInit__decimal to link options, otherwise, due to Py_NO_ENABLE_SHARED it becomes invisible. The result has no dependency on the python-dll! We copy it to the DLLs-folder and ...
hello_prog.exe
# crash/stopped worked
What is happening? We violated one definition rule (ODR) and ended up with two Python-interpreters: the one from the hello_prog.exe, which is initialized and the one from _decimal.pyd which is not initialized. _decimal.pyd "speaks" to its interpreter which is not initialized and bad things happens.
The difference to Linux is the difference between shared-objects and dlls: while shared-objects can use symbols from the exe (if the exe is built with right options) dll cannot and thus must either depend on a dll (which we don't want) or need to have its own version.
To avoid the violation of ODR we have only one way out: it must be linked directly into our hello_word-executable. So let's change the project for _decimal to static library and rebuild it in static_amd64-folder. Deleting the pyd from "DLLs"-folder and adding /WHOLEARCHIVE:_decimal.lib to the linker-command-line (whole archive while otherwise the linker would just discard _decimal.lib as none of its symbols is referenced somewhere), leads to an executable, which has the following error:
ModuleNotFoundError: No module named '_decimal'
This is expected - we need to tell to the interpreter, that the module _decimal is backed in and should not be searched on the python path.
The usual solution for this problem is to use PyImport_AppendInittab just before Py_Initialize, that means we need to change the c-file generated by cython (there might be workarounds, but due to multi-phase initialization it is not that easy. So probably a saner way to embed Python is the one presented here or here were main isn't written by Cython). The c-file should look as follows:
//decalare init-functions
extern PyObject* PyInit__decimal();
...
int main(int argc, char** argv) {
...
if (argc && argv)
Py_SetProgramName(argv[0]);
PyImport_AppendInittab("_decimal", PyInit__decimal); // HERE WE GO
// BEFORE Py_Initialize
Py_Initialize();
Building everything now leads to an exe which prints
I'm standalone
and this time it is not lying to us!
Now we have to repeat the last steps for all other built-in extension we need.
The above means there are some restrictions for statically built python-interpreter: All built-in modules need to be backed into the executable and we cannot extend the interpreter latter on with libraries like numpy/scipy (but can to do it directly at the compile/link time).
Getting rid of vcruntime-dll is easier: all above steps must be done with /MT option instead of MD-option. However, there might be some problems due to usage of other dlls (e.g. _ctypes needs ffi-dll) which where built with the dll-version (and thus we once again have ODR-violated) - so I would not recommend it.
