Building¶
Linux¶
The host system must be 64-bit. A Python 3.5+ interpreter must be available. The execution environment must have access to a Docker daemon (all build operations are performed in Docker containers for isolation from the host system).
To build a Python distribution for Linux x64:
$ ./build-linux.py
# With profile-guided optimizations (generated code should be faster):
$ ./build-linux.py --options pgo
# Produce a debug build.
$ ./build-linux.py --options debug
# Produce a free-threaded build without extra optimizations
$ ./build-linux.py --options freethreaded+noopt
You can also build another version of Python. e.g.:
$ ./build-linux.py --python cpython-3.8
To build a Python distribution for Linux x64 using musl libc:
$ ./build-linux.py --target x86_64-unknown-linux-musl
Building a 32-bit x86 Python distribution is also possible:
$ ./build-linux.py --target i686-unknown-linux-gnu
As are various other targets:
$ ./build-linux.py --target aarch64-unknown-linux-gnu
$ ./build-linux.py --target armv7-unknown-linux-gnueabi
$ ./build-linux.py --target armv7-unknown-linux-gnueabihf
$ ./build-linux.py --target mips-unknown-linux-gnu
$ ./build-linux.py --target mipsel-unknown-linux-gnu
$ ./build-linux.py --target ppc64le-unknown-linux-gnu
$ ./build-linux.py --target s390x-unknown-linux-gnu
macOS¶
The XCode command line tools must be installed. A Python 3 interpreter
is required to execute the build. /usr/bin/clang
must exist.
macOS SDK headers must be installed. Try running xcode-select --install
to install them if you see errors about e.g. stdio.h
not being found.
Verify they are installed by running xcrun --show-sdk-path
. It
should print something like
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk
on modern versions of macOS.
To build a Python distribution for macOS:
$ ./build-macos.py
macOS uses the same build code as Linux, just without Docker. So similar build configuration options are available.
build-macos.py
accepts a --target-triple
argument to support building
for non-native targets (i.e. cross-compiling). By default, macOS builds target
the currently running architecture. e.g. an Intel Mac will target
x86_64-apple-darwin
and an M1 (ARM) Mac will target aarch64-apple-darwin
.
It should be possible to build an ARM distribution on an Intel Mac and an Intel
distribution on an ARM Mac.
The APPLE_SDK_PATH
environment variable is recognized as the path
to the Apple SDK to use. If not defined, the build will attempt to find
an SDK by running xcrun --show-sdk-path
.
aarch64-apple-darwin
builds require a macOS 11.0+ SDK and building
Python 3.9+. It should be possible to build for aarch64-apple-darwin
from
an Intel 10.15 machine (as long as the 11.0+ SDK is used).
Python 3.8 may not build properly with a macOS 11.0+ SDK: try using the macOS 10.15 SDK instead.
Windows¶
Visual Studio 2017 (or later) is required. A compatible Windows SDK is required (10.0.17763.0 as per CPython 3.7.2).
If building CPython 3.8+, there are the following additional requirements:
A
git.exe
onPATH
(to clonelibffi
from source).An installation of Cywgin with the
autoconf
,automake
,libtool
, andmake
packages installed. (libffi
build dependency.)
To build a dynamically linked Python distribution for Windows x64:
$ py.exe build-windows.py --options noopt
It’s also possible to build with optional PGO optimizations:
$ py.exe build-windows.py --options pgo
If building CPython 3.8+, you will need to specify the path to a
sh.exe
installed from cygwin. e.g.
$ py.exe build-windows.py –python cpython-3.8 –sh c:cygwinbinsh.exe –options noopt
To build a 32-bit x86 binary, simply use an x86 Native Tools
Command Prompt
instead of x64
.
Using sccache to Speed up Builds¶
Builds can take a long time.
python-build-standalone can automatically detect and use the
sccache compiler cache to speed
up subsequent builds on UNIX-like platforms. sccache
can shave dozens
of minutes from fresh builds, even on a 16 core CPU!
If there is an executable sccache
in the source directory, it will
automatically be copied into the build environment and used. For non-container
builds, an sccache
executable is also searched for on PATH
.
The ~/.python-build-standalone-env
file is read if it exists (the format is
key=value
pairs) and variables are added to the build environment.
In addition, environment variables AWS_ACCESS_KEY_ID
,
AWS_SECRET_ACCESS_KEY
, and any variable beginning with SCCACHE_
are
automatically added to the build environment.
The environment variable support enables you to define remote build caches (such as S3 buckets) to provide a persistent, shared cache across builds and machines.
Keep in mind that when performing builds in containers in Linux (the default behavior), the local filesystem is local to the container and does not survive the build of a single package. So sccache is practically meaningless unless configured to use an external store (such as S3).
When using remote stores (such as S3), sccache
can be constrained on
network I/O. We recommend having at least a 100mbps network connection to
a remote store and employing a network store with as little latency as possible
for best results.