Firefox Contribution Process Debt

January 09, 2015 at 04:45 PM | categories: Mozilla

As I was playing around with source-derived documentation, I grasped the reality that any attempt to move documentation out of MDN's wiki and into something derived from source code (despite the argued technical and quality advantages) would likely be met with fierce opposition because the change process for Firefox is much more involved than edit a wiki.

This observation casts light on something important: the very act of contributing any change to Firefox is too damn hard.

I've always believed this statement to be true. I even like to think I'm one of the few people that has consistently tried to do something about it (inventing mach, overhauling the build system, bootstrap scripting, MozReview, etc). But I, like many of the seasoned Firefox developers, often lose sight of this grim reality. (I think it's fair to say that new contributors often complain about the development experience and as we grow accustomed to it, the complaint volume and intensity wanes).

But when you have the simplicity of editing a wiki page on MDN juxtaposed against the Firefox change contribution process of today, the inefficiency of the Firefox process is clearly seen.

The amount of knowledge required to change Firefox is obscenely absurd. Sure, some complex components will always be difficult to change. But I'm talking about any component: the base set of knowledge required to contribute any change to Firefox is vast. This is before we get into any domain-specific knowledge inside Firefox. I have always believed and will continue to believe that this is a grave institutional issue for Mozilla. It should go without saying that I believe this is an issue worth addressing. After all, any piece of knowledge required for contribution is essentially an obstacle to completion. Elimination of required knowledge lowers the barrier to contribution. This, in turn, allows increased contribution via more and faster change. This advances the quality and relevance of Firefox, which enables Mozilla to advance its Mission.

Seasoned contributors have probably internalized most of the knowledge required to contribute to Firefox. Here is a partial list to remind everyone of the sheer scope of it:

  • Before you do anything
    • Am I able to contribute?
    • Do I meet the minimum requirements (hardware, internet access, etc)?
    • Do I need any accounts?
  • Source control
    • What is source control?
    • How do I install Mercurial/Git?
    • How do I use Mercurial/Git?
    • Where can I get the Firefox source code?
    • How do I *optimally* acquire the Firefox source code?
    • Are there any recommended configuration settings or extensions?
  • Building Firefox
    • Do I even need to build Firefox?
    • How do I build Firefox?
    • What prerequisites need to be installed?
    • How do I install prerequisites?
    • How do I configure the Firefox build to be appropriate for my needs?
    • What are mozconfigs?
    • How do I get Firefox to build faster?
    • What do I do after a build?
  • Changing code
    • Is there IDE support?
    • Where can I find macros and aliases to make things easier?
  • Testing
    • How do I run tests?
    • Which tests are relevant for a given change?
    • What are all these different test types?
    • How do I debug tests?
  • Try and Automation
    • What is Try?
    • How do I get an account?
    • What is vouching and different levels of access?
    • What is SSH?
    • How do I configure SSH?
    • When will my tests run?
    • What is Tree Herder?
    • What do all these letters and numbers mean?
    • What are all these colors?
    • What's an *intermittent failure*?
    • How do I know if something is an *intermittent failure*?
    • What amount of *intermittent failure* is acceptable?
    • What do these logs mean?
    • What's buildbot?
    • What's mozharness?
  • Sending patch to Mozilla
    • Do I need to sign a CLA?
    • Where do I send patches?
    • Do I need to get an account on Bugzilla?
    • Do I need to file a bug?
    • What component should I file a bug in?
    • What format should patches be sent in?
    • How should I format commit messages?
    • How do I upload patches to Bugzilla?
    • How does code review work?
      • What's the modules system?
      • What modules does my change map to?
      • Who are the possible reviewers?
      • How do I ask someone for review?
      • When can I expect review?
      • What does r+ vs r- vs f+ vs f- vs cancelling review all mean?
      • How do I submit changes to my initial patch?
      • What do I do after review?
  • Landing patches
    • What repository should a patch land on?
    • How do you rebase?
    • What's a tree closure?
    • What do I do after pushing?
    • How do I know the result of the landing?

Holy #$%@, that's a lot of knowledge. Not only is this list incomplete, it's also not encompassing a lot of the domain-specific knowledge around the content being changed.

Every item on this list represents a point where a potential contributor could throw up their arms out of despair and walk away, giving their time and talents to another project. Every item on this list that takes 10 minutes instead of 5 could be the tipping point. For common actions, things that take 5 seconds instead of 1 could be the difference maker. This list thus represents reasons that people do not contribute to Firefox or contribute ineffectively (in the case of common contributors, like paid Mozilla staff).

I view items on this list as process debt. Process debt is a term I'm coining (perhaps someone has beat me to it - I'm writing this on a plane without Internet access) that is a sibling of technical debt. Process debt is overhead and inefficiency associated with processes. The border between process debt and technical debt in computers is the code itself (although that border may sometimes not be very well-defined, as code and process are oftentimes similar, such as most interactions with version control or code review tools).

When I see this list of process debt, I'm inspired by the opportunity to streamline the processes and bask in the efficiency gains. But I am also simultaneously overwhelmed by the vast scope of things that need improved. When I think about the amount of energy that will need to be exerted to fight the OMG change crowd, the list becomes depressing. But discussing institutional resistance to change, the stop energy associated with it, and Mozilla's historical record of failing to invest in fixing process (and technical) debt is for another post.

When looking at the above list, I can think of the following general ways to make it more approachable:

  1. Remove an item completely. If it isn't on the list, there is nothing to know and no overhead. The best way to solve a problem is to make it not exist.
  2. Automate an item and makes its existence largely transparent. If an item is invisible, does it exist in the mind of a contributor? (This is also known as solving the problem by adding a layer of indirection.)
  3. Change an item so that it is identical to another, more familiar process. If you use a well-defined process, there is no new knowledge that must be learned and the cost of on-boarding someone already familiar with that knowledge is practically zero.

When you start staring at this list of Firefox contribution process debt, you start thinking about priorities, groupings, and strategies. You look around at what others are doing to see if you can borrow good ideas.

I've done a lot of thinking on the subject and have some ideas and recommendations. Stay tuned for some additional posts on the topic.