The pyoxidizer
Command Line Tool¶
The pyoxidizer
command line tool is a frontend to the various
functionality of PyOxidizer
. See Components for more
on the various components of PyOxidizer
.
Settings¶
Cache Directory¶
pyoxidizer
may need to download resources such as Python distributions
and Rust toolchains from the Internet. These resources are cached in a
per-user directory.
PyOxidizer chooses the first available directory from the following list to use as the cache:
The value of the environment variable
PYOXIDIZER_CACHE_DIR
.$XDG_CACHE_HOME/pyoxidizer
on Linux ifXDG_CACHE_HOME
is set.$HOME/.cache/pyoxidizer
on Linux ifHOME
is set.$HOME/Library/Caches/pyoxidizer
on macOS ifHOME
is set.{FOLDERID_LocalAppData}/pyoxidizer
on Windows.~/.pyoxidizer/cache
The pyoxidizer cache-clear
command can be used to delete the contents
of the cache.
Managed Rust Toolchain¶
PyOxidizer leverages the Rust programming language and its tooling for building binaries embedding Python.
By default, PyOxidizer will automatically download and use Rust toolchains (the Rust compiler, standard library, and Cargo) when their functionality is needed. PyOxidizer will store these Rust toolchains in the configured cache.
If you already have Rust installed on your machine and want PyOxidizer to
use the existing Rust installation, either pass the --system-rust
flag
to pyoxidizer
invocations or define the PYOXIDIZER_SYSTEM_RUST
environment variable to any value. When the system Rust is being used,
pyoxidizer
will automatically use the cargo
executable found
on the current search path (typically the PATH
environment variable).
Creating New Projects with init-config-file
¶
The pyoxidizer init-config-file
command will create a new
pyoxidizer.bzl
configuration file in the target directory:
$ pyoxidizer init-config-file pyapp
This should have printed out details on what happened and what to do next.
Creating New Rust Projects with init-rust-project
¶
The pyoxidizer init-rust-project
command creates a minimal
Rust project configured to build an application that runs an
embedded Python interpreter from a configuration defined in a
pyoxidizer.bzl
configuration file. Run it by specifying the
directory to contain the new project:
$ pyoxidizer init-rust-project pyapp
This should have printed out details on what happened and what to do next.
The explicit creation of Rust projects to use PyOxidizer
is not
required. If your produced binaries only need to perform actions
configurable via PyOxidizer
configuration files (like running
some Python code), an explicit Rust project isn’t required, as
PyOxidizer
can auto-generate a temporary Rust project at build time.
But if you want to supplement the behavior of the binaries built with Rust, an explicit and persisted Rust project can facilitate that. For example, you may want to run custom Rust code before, during, and after a Python interpreter runs in the process.
See PyOxidizer Rust Projects for more on the composition of Rust projects.
Building PyObject Projects with build
¶
The pyoxidizer build
command is probably the most important and used
pyoxidizer
command. This command evaluates a pyoxidizer.bzl
configuration file by resolving targets in it.
By default, the default target in the configuration file is resolved. However, callers can specify a list of explicit targets to resolve. e.g.:
# Resolve the default target.
$ pyoxidizer build
# Resolve the "exe" and "install" targets, in that order.
$ pyoxidizer build exe install
PyOxidizer
configuration files are effectively defining a build
system, hence the name build for the command to resolve targets
within.
Running the Result of Building with run
¶
Target functions in PyOxidizer
configuration files return objects
that may be runnable. For example, a PythonExecutable
returned by a target function that defines a Python executable binary
can be run by executing a new process.
The pyoxidizer run
command is used to attempt to run an object
returned by a build target. It is effectively pyoxidizer build
followed
by running the returned object. e.g.:
# Run the default target.
$ pyoxidizer run
# Run the "install" target.
$ pyoxidizer run --target install
Analyzing Produced Binaries with analyze
¶
The pyoxidizer analyze
command is a generic command for analyzing the
contents of executables and libraries. While it is generic, its output is
specifically tailored for PyOxidizer
.
Run the command with the path to an executable. For example:
$ pyoxidizer analyze build/apps/myapp/x86_64-unknown-linux-gnu/debug/myapp
Behavior is dependent on the format of the file being analyzed. But the
general theme is that the command attempts to identify the run-time
requirements for that binary. For example, for ELF binaries it will
list all shared library dependencies and analyze glibc
symbol
versions and print out which Linux distributions it thinks the binary
is compatible with.
Note
pyoxidizer analyze
is not yet implemented for all executable
file types that PyOxidizer
supports.
Inspecting Python Distributions¶
PyOxidizer
uses special pre-built Python distributions to build
binaries containing Python.
These Python distributions are zstandard compressed tar files. Zstandard
is a modern compression format that is really, really, really good.
(PyOxidizer’s maintainer also maintains
Python bindings to zstandard
and has
written about the benefits of zstandard
on his blog. You should read that blog post so you are enlightened on
how amazing zstandard is.) But because zstandard is relatively new, not
all systems have utilities for decompressing that format yet. So, the
pyoxidizer python-distribution-extract
command can be used to extract
the zstandard compressed tar archive to a local filesystem path.
Python distributions contain software governed by a number of licenses. This of course has implications for application distribution. See Licensing Considerations for more.
The pyoxidizer python-distribution-licenses
command can be used to
inspect a Python distribution archive for information about its licenses.
The command will print information about the licensing of the Python
distribution itself along with a per-extension breakdown of which
libraries are used by which extensions and which licenses apply to what.
This command can be super useful to audit for license usage and only allow
extensions with licenses that you are legally comfortable with.
For example, the entry for the readline
extension shows that the
extension links against the ncurses
and readline
libraries, which
are governed by the X11, and GPL-3.0 licenses:
readline
--------
Dependency: ncurses
Link Type: library
Dependency: readline
Link Type: library
Licenses: GPL-3.0, X11
License Info: https://spdx.org/licenses/GPL-3.0.html
License Info: https://spdx.org/licenses/X11.html
Note
The license annotations in Python distributions are best effort and can be wrong. They do not constitute a legal promise. Paranoid individuals may want to double check the license annotations by verifying with source code distributions, for example.
Debugging Resource Scanning and Identification with find-resources
¶
The pyoxidizer find-resources
command can be used to scan for
resources in a given source and then print information on what’s found.
PyOxidizer’s packaging functionality scans directories and files and classifies them as Python resources which can be operated on. See Resource Types. PyOxidizer’s run-time importer/loader (oxidized_importer Python Extension) works by reading a pre-built index of known resources. This all works in contrast to how Python typically works, which is to put a bunch of files in directories and let the built-in importer/loader figure it out by dynamically probing for various files.
Because PyOxidizer has introduced structure where it doesn’t exist in Python and because there are many subtle nuances with how files are classified, there can be bugs in PyOxidizer’s resource scanning code.
The pyoxidizer find-resources
command exists to facilitate
debugging PyOxidizer’s resource scanning code.
Simply give the command a path to a directory or Python wheel archive and it will tell you what it discovers. e.g.:
$ pyoxidizer find-resources dist/oxidized_importer-0.1-cp38-cp38-manylinux1_x86_64.whl
parsing dist/oxidized_importer-0.1-cp38-cp38-manylinux1_x86_64.whl as a wheel archive
PythonExtensionModule { name: oxidized_importer }
PythonPackageDistributionResource { package: oxidized-importer, version: 0.1, name: LICENSE }
PythonPackageDistributionResource { package: oxidized-importer, version: 0.1, name: WHEEL }
PythonPackageDistributionResource { package: oxidized-importer, version: 0.1, name: top_level.txt }
PythonPackageDistributionResource { package: oxidized-importer, version: 0.1, name: METADATA }
PythonPackageDistributionResource { package: oxidized-importer, version: 0.1, name: RECORD }
Or give it the path to a site-packages
directory:
$ pyoxidizer find-resources ~/.pyenv/versions/3.8.6/lib/python3.8/site-packages
...
This command needs to use a Python distribution so it knows what file
extensions correspond to Python extensions, etc. By default, it will
download one of the
built-in distributions that is
compatible with the current machine and use that. You can specify a
--distributions-dir
to use to cache downloaded distributions:
$ pyoxidizer find-resources --distributions-dir distributions /usr/lib/python3.8
...
Defining Extra Variables in Starlark Environment¶
Various pyoxidizer
commands (like build
and run
) accept arguments
to define extra variables in the Starlark environment in the VARS
global dict. This feature can be used to parameterize and conditionalize the
evaluation of configuration files.
Note
While we could inject global variables into the Starlark environment,
since it is illegal to access an undefined symbol (there’s not even a
way to test if a symbol is defined) and since we have no hook point to
inject variables after the symbol has been defined, we resort to populating
a global VARS
dict with variables.
For example, let’s make the name of the built executable dynamic:
DEFAULT_APP_NAME = "default"
def make_exe(dist):
dist = default_python_distribution()
return dist.to_python_executable(name = VARS.get("app_name", DEFAULT_APP_NAME))
register_target("exe", make_exe)
resolve_targets()
Then let’s build it:
# Uses `default` as the application name.
$ pyoxidizer build
# Uses `my_app` as the application name.
$ pyoxidizer build --var app_name my_app
# Uses `env_name` as the application name via an environment variable.
$ APP_NAME=env_name pyoxidizer build --var-env app_name APP_NAME