OxidizedZipFinder Meta Path Finder¶
oxidized_importer contains a pure Rust implementation of a meta path
finder that can load Python resources from zip files. Its goal is to be
a compatible reimplementation of zipimport.zipimporter from the Python
standard library.
Usage¶
Instances of OxidizedZipFinder are bound to zip archive data.
Instances can be constructed by calling
OxidizedZipFinder.from_zip_data() or
OxidizedZipFinder.from_path().
OxidizedZipFinder is a meta path finder and instances should
be registered on sys.meta_path. e.g.
import os
import sys
import oxidized_importer
HERE = os.dirname(os.path.abspath(__file__))
zip_path = os.path.join(HERE, "archive.zip")
zip_importer = OxidizedZipFinder.from_path(zip_path)
sys.meta_path.insert(0, zip_importer)
Once an instance is registered on sys.meta_path, it will be consulted
when an import is serviced by Python’s importing mechanism.
Behavior¶
OxidizedZipFinder is similar to - but critically different from -
the standard library zipimport.zipimporter.
OxidizedZipFinder is a meta path finder, not a
path entry finder. This means instances are bound to sys.meta_path and not
sys.path_hooks. Support for enabling use as a path hook is planned. The
lack of sys.path_hooks support means this importer can’t be used as
a replacement for zipimport.zipimporter.
All I/O and zip reading in OxidizedZipFinder is implemented in
Rust. Subtle differences in behavior as a result of zip parsing implementations
could occur.
OxidizedZipFinder doesn’t yet implement support for resource
reading (e.g. the importlib.abc.ResourceReader interface). Only loading
of .py and .pyc files is supported.
OxidizedZipFinder doesn’t validate the header of .pyc
files. If it sees a .pyc version of a module, its bytecode will be
used as-is. (zipimport.zipimporter validates that the content in
the .pyc matches expectations.)
Support for opening just sub-directories within zip files is not yet implemented.
Performance¶
OxidizedZipFinder should perform substantially better than
zipimport.zipimporter.
A test importing the ~450 modules that constitute the Python standard library yielded the following results:
Environment |
|
Us (memory) |
Us (file) |
|
Ryzen 5950X Linux |
205.07 ms |
168.70 ms |
184.74 ms |
126.33 ms |
Ryzen 5950X Windows |
235.73 ms |
147.14 ms |
167.10 ms |
140.21 ms |
(The exact set of modules and Python versions were different between the environments so it isn’t fair to compare numbers across environments: only within the same environment.)
Python API¶
See OxidizedZipFinder for the Python API documentation.