My Shifting Open Source Priorities

March 17, 2024 at 09:00 PM | categories: Personal, PyOxidizer

I'm a maintainer of a handful of open source projects, some of which have millions of downloads and/or are used in important workloads, including in production.

I have a full time job as a software engineer and my open source work is effectively a side job. (Albeit one I try very hard to not let intersect with my day job.)

Historically, my biggest contributions to my open source projects have come when I'm not working full time:

  • python-zstandard was started when I was on medical leave, recovering from a surgery.
  • python-build-standalone and PyOxidizer were mainly built when I was between jobs, after leaving Mozilla.
  • apple-codesign was built in the height of COVID when I took a voluntary leave of absence from work to reconstitute my mental and physical health.

When working full time, my time to contribute to open source has been carved out of weekday nights and weekends, especially in the winter months. I believe that code is an art form and programming a form of creative expression. My open source contributions provide a relaxing avenue for me to express my artistic creativity, when able.

My open source contributions reflect my personal priorities of where and what to spend my free time on.

The only constant in life is change.

In the middle of 2022, I switched job roles and found myself reinvigorated by my new role - Infrastructure Performance - which is at the intersection of some of my strongest technical and professional skills. I found myself willingly pouring more energy and time into my day job. That had the side effect of reducing my open source contributions.

In 2023Q1 I got married. In the months leading up to and after, I chose to prioritize spending time with my now wife and all the time commitments that entails. This also reduced the amount of time available for open source contributions.

In 2023Q4 I became a father to a beautiful baby girl. While on my employer's generous-for-the-United-States fourteen week paternity leave, I somehow found some time to contribute to open source. As refreshing as that was, it didn't last. My man cave where my desktop computer resides has been converted into a nursery. And for the past few months it has been occupied by my mother-in-law, who has been generously effectively serving as a live-in nanny. Even when I'm able to sit down at my desktop, it's hard to get into a state of flow due to the added entropy from the additional three people now living with me.

After realizing the new normal in 2024Q1, I purchased a Wahoo KICKR MOVE bicycle trainer and now spend considerable time doing virtual bicycle rides on Zwift because its one of the few leisure activities I can do at home without drawing scrutiny from my wife and mother-in-law (but 98% my mother-in-law because I've observed that my wife is effectively infallible). I now get excited about virtually summiting famous climbs instead of contributing to open source. (Today's was Mont Ventoux - an absolute beast of a climb that reminded me a lot of my real world ride up Pike's Peak in 2020.)

Various changes in the past eighteen or so months have created additional time constraints and prioritization changes that have resulted in my open source contributions withering.

In addition, my technical interests have been shifting.

I've always gravitated to more systems-level areas of computers. My degree is in Computer Engineering and I have a stereotypical engineer mindset: I have an insatiable curiosity about how things work and interact and I want to always be tinkering. I prefer to be closer to hardware instead of abstracted far away from it. I enjoy interacting with the building blocks of software ecosystems: operating systems, filesystems, runtimes, file formats, compilers, etc.

Historically, my open source contributions to my preferred areas of computing were limited. Again, to me open source is an enjoyable form of creative expression. That means I do it for fun. Historically, the systems-level programming space was limited to languages like C and C++, which I consider frustrating and painful to use. If I'm going to subject myself to misery when programming, you are going to have to pay me well to do it.

As part of creating PyOxidizer, I learned Rust.

When I became proficient in Rust, I realized that Rust unlocks all kinds of systems-level problems that were effectively off-limits for my open source contributions. Would I implement Debian packaging primitives in Python? Or a tool to bulk analyze Linux packages and peek inside ELF binaries for insights about what compiler/linker features are used in the wild in Python/C/C++? Not unless you pay me to do it!

As I learned Rust, I also found myself being drawn away from Python, my prior go-to language. As I wrote in Rust is for Professionals, Rust feels surprisingly high level. It isn't as terse as Python but it is a lot closer than I thought it would be. And Rust gives you vastly stronger compile-time guarantees and run-time performance than Python. I felt like Rust's tooling ecosystem was supporting me instead of standing in my way. I felt that when you consider the overall software development lifecycle - not just the edit-build-run loop that people tend to fixate on, likely because it is the easiest to measure - Rust was vastly more productive and a joy to work with than Python. All those countless hours debugging, fixing, and authoring tests for TypeError and ValueError Python exceptions you see in production just don't happen with Rust and that time can be better spent iterating on core functionality, which is what actually matters.

On top of the Rust undercurrents, I've also become somewhat disenchanted with the Python ecosystem. As I wrote in 2020's Mercurial's Journey to and Reflections on Python 3, the Python 3 transition was bungled and resulted in years - if not a full decade - of lost opportunity. As I wrote in 2023's My User Experience Porting Off setup.py, the Python packaging story feels as discombobulated and frustrating as ever. PyOxidizer additionally brushed up against several limitations in how Python is designed and implemented, many of which are not trivially fixable. As a systems-level guy, I am frequently questioning various aspects of the Python ecosystem which I have contrasting opinions on, including the importance of correctness and performance.

Starting in 2021, I started gravitating towards writing more Rust code and solving problems in the systems domain that were previously off-limits to me, like Apple code signing. Initially the work was in support of PyOxidizer: I was going to implement all these packaging primitives in pure Rust and enable people to distribute Python applications without requiring access to a Windows or macOS machine! Over time, this work consumed me. Apple code signing turned into a major time sink because of its complexity and the fact I was having to reverse engineer a lot of its internals. But I was having a ton of fun doing it: more fun than swimming upstream against decades of encrusted technical debts in the Python ecosystem.

By late 2021, I realized I made a series of mistakes with PyOxidizer.

I started PyOxidizer as a science experiment to see if it was possible to achieve a single file executable Python application without requiring a temporary filesystem at run-time. I succeeded. But the cost was compatibility with the larger pre-built Python package ecosystem. I built all this complexity into PyOxidizer to allow people to tweak how Python resources are packaged so they could choose to build a single file application if they wanted. This ballooned into a hot mess and was obviously not user-friendly. It violated various personal principles about optimizing for end-user experience.

Armed with knowledge of all the pitfalls, I realized that there was a 90% use case for Python application packaging that was simple for end users and technically achievable using all the code primitives - like the pyembed Rust crate - that I built out for PyOxidizer.

Thus the PyOxy project was born and released in May 2022.

While I believe PyOxy is already a generally useful primitive to have in the Python ecosystem, I had bigger goals in mind.

My intent with PyOxy was to build in a simplified and opinionated PyOxidizer lite mode. The pyoxy executable is already a chameleon: if you rename it to python it behaves like a python executable. I wanted to extend this so you could do something like pyoxy build-app and it would collect all dependencies, assemble a Python packed resources blob, and embed that in a copy of the pyoxy binary as an ELF, Mach-O, or PE segment. Then at run-time, the variant executable binary would load the application configuration and Python resources metadata from its own binary and execute the application. Essentially, PyOxy would evolve into a self-packaging Python application. I just needed to evolve the Python packed resources format, implement a very crude ELF, Mach-O, and PE linker to append resources data to an executable, and teach pyembed to read resources data from an ELF, Mach-O, or PE segment. All within my sphere of technical competency. And I was excited to build it and forever alter people's perceptions of how easy it could be to produce a distributable Python application.

Then the roller coaster of my personal life took over. I felt newly invigorated with a new job role. I got engaged and married. I became a father.

By early 2023, it was clear my ability to contribute to open source would be vastly diminished for the foreseeable future. PyOxidizer and PyOxy fell into a state of neglect. Weeks went by without me even tinkering on my local computer, much less push commits or publish a release. Weeks turned into months. Months into quarters. At this point, I haven't pushed a commit to indygreg/PyOxidizer since January 2023. And I'm not sure when I next will, if ever.

In my limited open source contribution time, I've prioritized other projects over PyOxidizer.

python-build-standalone has gained a life outside PyOxidizer. It is now used by rye, Bazel's rules_python, briefcase, and a myriad of other consumers. The release assets have been downloaded over 23 million times and the download rate appears to be accelerating. I still actively support python-build-standalone and intend for the project to be actively supported for the indefinite future: it has become too important to abandon. I'm actively recruiting assistance to help maintain the project and I'm not concerned about its future.

Apple code signing still actively draws my engagement. What I love about the project is it either works or it doesn't: there's limited extra features we can add to it since Apple mostly dictates the feature set. And I perceive the current project to be mostly done.

python-zstandard is downloaded ~8 million times per month. The project is long overdue for some modernization. I'm sitting on a pile of commits to improve it, but progress has been slow. I just learned this weekend that the maintainer of the other popular zstandard Python package deleted their GitHub account recently and now users are looking to onboard to my package. Nothing quite like unanticipated distractions!

That's a very long-winded way of saying that PyOxidizer and all the projects under its umbrella are effectively in a zombie state. I'm hesitant to say dead because if I suddenly found myself with lots of free time I'd love to brush off the cobwebs and bring the projects back to life. But who am I kidding: they are effectively dead at the moment because with everything happening in my personal life, I don't see where I find the time to resuscitate the project. And that assumes I even want to: again, I've become somewhat disenchanted by the state of Python. The main thing that draws me to it is the size of the community and the potential for impact. But to realize that impact I feel like I'd be pushing Python in directions it isn't well-equipped to go in. Quite franky - and, yes, selfishly - I don't want to subject myself to the misery unless I'm being well paid to do it. Again, I view my open source contributions as a fun outlet for my creative expression and nudging Python packaging in directions it is obviously ill-equipped to go in just isn't fun.

If anyone reading has an interest in taking ownership or maintenance responsibilities of PyOxidizer, any projects under its umbrella, or any of my other open source projects, I'm receptive to proposals. Send me an email or create an issue or discussion on GitHub if you want to do it publicly.

But I'm going to assume that PyOxidizer is going to wither and die - or at least incur some massive backwards incompatible breaks if it continues to live. I've already filed issues against python-build-standalone - such as removing Windows static builds - to make the project easier to support and less work for future maintainers.

If I have one regret about how this has played out, it is my failure to communicate developments in my open source commitments / expectations in a timely manner. I knew the future was bleak in early 2023 but didn't publicly say anything. I still thought there was a chance that things were going to change and I didn't want to make a hard decision prematurely. Writing this post has been on my mind since the middle of 2023 but I just couldn't bring myself to write it. And - surprise - having a newborn at home is a giant time and mental commitment! I'm writing this now because people are (finally!) noticing my lack of contributions to PyOxidizer and asking questions. And I'm home alone for a few days and actually have time to sit down and compose this post. (Yes, I'm that stretched for time in my personal life.)

In 2023, I struggled with the idea of letting people down by declaring PyOxidizer dead. But when I wake up every morning, walk into the nursery, and cause my daughter to smile and flail her arms and legs with unbridled excitement when she sees me, I'd have it no other way. When it comes to choosing between open source and family, I choose family.

It feels appropriate to end this post with a link to XKCD 2347: Dependency. But I'm not the random person in Nebraska: I'm a husband and father.