tl;dr User experience matters and developers are people too. I have proposed a tool to help developers interact with the Firefox build system and source tree.

I don't think I have to make my case when I state that Mozilla's build system end-user experience is lacking. There are lots of hurdles to overcome:

• Determine where to obtain the source code.
• Install a source control system (possibly).
• Wait a long time for the large source repository to download.
• Figure out how to launch the build process (unlike many other build systems, it isn't as simple as configure or make - although it is close).
• Determine which dependencies need to be installed and install them (this can also take a long time).
• Create a configuration file (mozconfig).
• Build the tree (another long process).

If you want to contribute patches, there are additional steps:

• Configure Mercurial with your personal info.
• Configure Mercurial to generate patches in proper format.
• Create a Bugzilla account (made simpler through Persona!).
• Figure out the proper Bugzilla product/component (even I still struggle at this) so you can file a bug.
• Figure out how to attach a patch to a bug and request review (it isn't intuitive if you've never used Bugzilla before).
• Figure out who should review patch.
• Learn how tests work so you can:
• Write new tests.
• Run existing tests to verify your changes.
• Obtain commit access (so at least you can push to Try).
• Learn how to push to Try.
• Learn about TBPL.
• Discover and use some of the amazing tools to help you (MXR, trychooser, mqext, etc).

Granted, not all of these are required. But, they will be for returning contributors. My point is that there are lots of steps here. And, every one of them represents a point where someone could get frustrated and bail -- a point where Mozilla loses a potential contributor.

Ever since I started at Mozilla, I've been thinking of ways this could be done better. While the Developer Guide on MDN has improved drastically in the last year, there are still many ways the process could be improved and streamlined.

In bug 751795, I've put forward the groundwork of a tool to make the developer experience more user friendly. Yes, this is a vague goal, so let me go in to further detail.

What I've submitted in the bug is essentially a framework for performing common actions related to the build system and source tree. These actions are defined as methods in Python code. Hooking it all together is a command-line interface which is launched via a short script in the root directory called mach (mach is German for do). Since actions speak louder than words, here's an example:

$ ./mach

usage: mach command [command arguments]

This program is your main control point for the Mozilla source tree.

To perform an action, specify it as the first argument. Here are some common
actions:

  mach build         Build the source tree.
  mach help          Show full help.
  mach xpcshell-test Run xpcshell test(s).

To see more help for a specific action, run:

  mach <command> --help

e.g. mach build --help

And, going into a sub-command:

$ ./mach xpcshell-test --help

usage: mach xpcshell-test [-h] [--debug] [TEST]

positional arguments:
  TEST         Test to run. Can be specified as a single JS file, an
               xpcshell.ini manifest file, a directory, or omitted. If
               omitted, the entire xpcshell suite is executed.

optional arguments:
  -h, --help   show this help message and exit
  --debug, -d  Run test in debugger.

Now, I've focused effort at this stage on performing actions after the initial build environment is configured. The reason is this is low-hanging fruit and easily allows me to create a proof-of-concept. But, I have many more ideas that I'd eventually like to see implemented.

One of my grand ideas is to have some kind of setup wizard guide you through the first time you use mach. It can start by asking the basics: "Which application do you want to build?" "Release or Debug?" "Clang or GCC?" "Should I install Clang for you?" It could also be more intelligent about installing dependencies. "I see you are using Ubuntu and are missing required packages X and Y. Would you like me to install them?" And, why stop at a command-line interface? There's no reason a graphical frontend (perhaps Tcl/Tk) couldn't be implemented!

The setup wizard could even encompass configuring your source control system for proper patch generation by ensuring your tree-local .hg/hgrc or .git/config files have the proper settings. We could even ask you for Bugzilla credentials so you could interact with Bugzilla directly from the command-line.

Once we have all of the basic configs in place, it's just a matter of hooking up the plumbing. Want to submit a patch for review? We could provide a command for that:

./mach submit-patch

"refactor-foo" is currently on top of your patch queue.

Submit "refactor-foo"?
y/n: y

Enter bug number for patch or leave empty for no existing bug.
Bug number:

OK. A new bug for this patch will be created.

Please enter a one-line summary of the patch:
Summary: Refactor foo subsystem

Is the patch for (r)eview, (f)eedback, or (n)either?
r/f/n: r

I've identified Gregory Szorc (:gps) as a potential reviewer for
this code. If you'd like someone else, please enter their IRC
nickname or e-mail address. Otherwise, press ENTER.
Reviewer:

I'm ready to submit your patch. Press ENTER to continue or CTRL+C to
abort.

Bug 700000 submitted! You can track it at
https://bugzilla.mozilla.org/show_bug.cgi?id=700000

The framework is extremely flexible and extensible for a few reasons. First, it encourages all of the core actions to be implemented as Python modules/methods. Once you have things defined as API calls (not shell scripts), the environment feels like a cohesive library rather than a loose collection of shell scripts. Shell scripts have a place, don't get me wrong. But, they are hard to debug and test (not to mention performance penalties on Windows). Writing code as reusable libraries with shell scripts only being the frontend is a more robust approach to software design.

Second, the command-line driver is implemented as a collection of sub-commands. This is similar to how version control systems like Git, Mercurial, and Subversion work. This makes discovery of features extremely easy: just list the supported commands! Contrast this to our current build system, where the answer is to consult a wiki (with likely out-of-date and fragmented information) or gasp try to read the makefiles in the tree.

My immediate goal for bug 751795 is to get a minimal framework checked in to the tree with a core design people are content with. Once that is done, I'm hoping other people will come along and implement additional features and commands. Specifically, I'd like to see some of the awesome tools like mqext integrated such that their power can be harnessed without requiring people to first discover they exist and second install and configure them. I think it is silly for these obvious productivity wins to go unused by people ignrant of their existence. If they are valuable, let's ship them as part of a batteries included environment.

In the long run, I think there are many more uses for this framework. For starters, it gives us a rallying point around which to organize all of the Python support/tools code in the tree. Currently, we have things spread all over the place. Quite frankly, it is a mess. I'd like to have a unified site-packages tree with all our Python so things are easier to locate and thus improve.

If nothing else, the tool provides a framework for logging and formatting activities in a unified way. There are separate log streams: one for humans, one for machines. Under the hood, they both use the saming logging infrastructure. When messages are logged, the human stream is formatted as simple sentences (complete with terminal encodings and colorization). The machine-destined log stream is newline-delimited JSON containing the fields that were logged. This allows analysis of output without having to parse strings. This is how all log analysis should be done. But, that's for another post. Anyway, what this all means is that the output for humans can be more readable. Colors, progress bars: we can do that now.

Over time, I imagine some may want to move logic out of configure and makefiles and into this tool (because Python is easier to maintain and debug, IMO). I would love to see that too. But, I want to stress that this isn't a focus right now. I believe this framework should be supplemental in the beginning and the official build system should not rely on it. Maybe that changes in the future. Time will tell.

Anyway, this project is currently just my solo effort. This isn't captured on a roadmap or anyone's quarterly goals. There is no project page listing planned features. If you are interested in helping, drop me a line and/or join in on the bug. Hopefully the core framework will land soon. Once it does, I'm hoping for an explosion of new, user-friendly features/commands to make the overall Firefox development experience smoother.