In case you missed it, I gave a presentation on the state of Firefox's build system yesterday.

You can watch it and view the slides online.

If you build Firefox from source regularly, you should definitely at least skim through the slide deck.

I'm not an HTML expert, so my apogolies for bad UI on the interactive slides. You may need to press enter to select items in dropdown menus. Also, the interactive slides are a bit resource intensive. If the slide deck is really slow, turn off those elements. I've also only tested the slides in Firefox 19 and 20. My apologies if they don't work everywhere.

Hacking on Firefox is hard. Compiling the source tree for the first time is a daunting experience. If you manage to do that, you still need to figure out how to submit patches, write and run tests, etc. There is no shortage of points where people can get confused, make mistakes, or just give up because the barrier is too high.

I have strong convictions about making the overall developer experience better (developers are users too, after all). The easier you make the developer experience, the greater the chances of retaining developers. And retaining developers means more contributions: more features and fewer bugs. This translates to a better state for the Mozilla Project. This makes the world a better place.

Since practically my first day working at Mozilla, I've been experimenting with ways to make contributing to Firefox easier by improving the build system or interaction with it.

With a lot of help, I've finally succeeded in landing something into the Firefox source tree that I think will ultimately lead to a much better developer experience.

It's called mach (German for do) and if you pull the latest version of mozilla-central (Firefox's main source code repository), you can run mach today.

Running Mach

You can run Mach by simply executing it from the root directory in the repository:

$ ./mach

Ideally, I shouldn't have to tell you anything else: mach's output should provide you all the instruction you need to use it. If it doesn't, that is a bug and it should be fixed.

Realistically, mach is still in early development and its user experience still leaves a lot to be desired.

Because technical usage docs belong in a medium that is easily discoverable and where the community can improve on them (not a post on a personal blog), you can find additional usage information in the mach article on MDN. The mach README holds more technical information for people wanting to poke at the inner workings.

Mach does require Python 2.7. The build system will likely soon require Python 2.7 as well. So, if you don't have Python 2.7, you should upgrade now before you lose the ability to build the tree. Conveniently, the tree now has a bootstrap script which covers the installation of Python. So, Python 2.7 should just be a simple command away.

Features

Why would you use mach? Good question! Compared to the existing out-of-the-box experience, mach adds:

• Ability to run xpcshell and mochitest tests from the source directory. This means you can tab-complete test filenames in your shell and it just works.
• Parsing of compiler warnings (currently supports Clang 3.1 and MSVC formats) into a unified warnings database (actually a JSON file). After builds you can run ./mach warnings-list or ./mach warnings-summary to get a concise list without having to look at build logs.
• A single command-line interface where you can easily discover new functionality. Just run ./mach help (try doing that with make!).

Naysayers will say this is a paltry list. They are correct. I have bigger plans. But, you have to start somewhere.

Goals and Future Ideas

The overall goal of mach is to improve the developer experience of developing Firefox and other Gecko applications. It does this by providing a convenient, central command in the root directory of the repository that acts as both an oracle to discover new commands and functionality (./mach help) as well as a proxy to execute them. You don't need to worry about environment variables, working directories, or finding some random script hidden in the bowells of the source tree. You just run a single command and the world is brought to you. No build documentation. No outdated wikis. No copying commands into your shell. No having to scour random blogs for useful tools. You just clone the repository, run a command, see what you can do, and get stuff done. Your shell-literate grandmother could do it.

Mach should be your springboard to in-tree developer tools and increased productivity. You shouldn't need anything except a copy of the source tree and mach.

Like Git and Mercurial, mach is powered by the concept of sub-commands/actions. So, one simply needs to register a new sub-command with the mach driver. This involves writing a Python class. Once you do that, people can discover and use that functionality. All batteries are included with a copy of mozilla-central.

As stated above, the current set of commands is rather small. But, the sky is the limit. Here are some of my ideas:

• Ability to upload, download, and apply patches from Bugzilla (Burak Yiğit Kaya, Jeff Hammel, Clint Talbert and I have already talked about this -- progress tracked in bug 774141).
• Automatically configure Mercurial with optimal settings (ensure user info is set, proper lines of diff context, enable mqext, etc). Tracked bug 794580.
• Submit Try builds. The trychooser Mercurial extension could easily live as a mach subcommand! Tracked in bug 774137.
• Identify Bugzilla components and/or reviewers from files touched by patch. It's all in the history of the touched files and the history of the old bugs referenced by those commits! Bug 774145.
• Interaction with the self-serve build API. That thing on TBPL to control individual builds - we could make a CLI interface for it. (Steve Fink and/or Jeff Hammel already have code for this - it just needs to be integrated). Bug 774147.

If you have an idea for a feature, please file a bug. Please note there are many features on file already. However, some obvious ones such as integration with missing test suites have yet to be filed (at least at the time I wrote this post).

If you wrote an awesome developer tool and would like others to use it (without having to rely on people discovering it by reading a corner of the Internet), add it to mach! Use mach as a wedge to get more exposure and users. File a bug. I will happily r+ patches that add useful developer functionality to the tree.

What this Means / Longer Term Future

While there is no timetable, mach will eventually replace client.mk. client.mk, like mach, is essentially a CLI driver for the build system. Having the driver implemented in Python rather than make has many compelling advantages. I could probably write a whole post on it, but I'll spare the world from my ranting.

Anyway, this all means that you may want to start re-training your muscle memory now. Stop typing make and start typing mach. If you need to type make because mach does not provide a feature, this is a missing feature from mach. File a bug and request a new mach feature!

I want to condition people to stop typing make, especially in the object directory. There are drastic changes to the build system in the works (bug 784841 is the tip of the iceburg). These changes will require the build system to be treated as a black box. So, things like invoking make from the object directory will eventually break. You'll need to go through an intelligent tool to invoke the build system. Mach will be that tool.

Thanks

I would like to single out the following individuals for helping to land mach:

• Jeff Hammel for doing the bulk of the reviewing. He shares my vision for mach and how it will make the overall developer experience much more pleasant and how this will translate to better things for The Project. In my mind, Jeff deserves as much credit for landing mach as I do.
• Mike Hommey and Ms2ger for review help. Mike Hommey helped identify a lot of issues with build system integration. Ms2ger provided lots of general feedback on Python issues and API design.
• Mike Connor (my manager) for allowing me to work on this. It's not related to my job description in any way so he could say I shouldn't be spending time on this. But, he realizes the positive impact this can have and has been supportive of it.

I hope you find mach useful!

If you've looked at the build instructions for Firefox lately, you may have noticed something new: support for system bootstrapping!

Now checked in to mozilla-central is a framework for ensuring your system is capable of building mozilla-central and Firefox. You just need to download and run a single Python script and it performs magic.

Kudos go out to a community contributor, kmm (name wasn't revealed) for doing the legwork for tracking down and verifying things worked on all the Linux distros. Richard Newman and Jeff Hammel also helped with code reviews. Just hours after it landed, Landry Breuil contributed support for OpenBSD!

Currently, bootstrapping works for the following:

• Ubuntu Linux
• Mint
• CentOS 6.x
• Fedora
• OS X 10.6, 10.7, and 10.8
• OpenBSD

If you want to add support for an additional OS, please file a Core : Build Config bug. Likewise, if you encounter an issue, please file a bug so others don't get tripped up by it!

Bootstrap support is definitely in its infancy. It still needs features like better prompting and opportunities for user choice (e.g. support MacPorts on OS X - currently it only works with Homebrew). But, I think it is much better than what existed previously, especially on OS X.

I consider this bootstrapping component just one piece in a larger mission to make developing and building Firefox (and other Gecko applications) easier. This should (hopefully) lead to more development involvement. The next component to land will likely be mach. It's (finally) been getting some review love (thanks Jeff Hammel!), so I'm optimistic it will land in the next few weeks.

I have very alpha support for Visual Studio project generation for mozilla-central that daring people can dogfood.

I want to emphasize that this is extremely alpha. Normally, I wouldn't release things as fragile as they are. But, I know Windows developers sorely miss Visual Studio, especially IntelliSense. The current Visual Studio projects support IntelliSense, so I want to get this in the hands of Windows developers ASAP.

The current directions for making this all work are a bit hacky. Things will change once things have matured. For now, please excuse the mess.

First, you will need to grab the code. If you use Git, set up a remote to my repository:

git remote add indygreg git://github.com/indygreg/mozilla-central.git
git fetch indygreg

The branch of interest is build-splendid. I periodically rebase this branch on top of master. You have been warned.

You can switch to this branch:

git checkout -b build-splendid indygreg/build-splendid

Alternatively, you can squash it down to a single commit and merge it into your local branch. Once you've done that, you can record the SHA-1 of the commit and cherry-pick that wherever you like!

git merge --squash indygreg/build-splendid
git commit

In the current state, you need to build the tree or the Visual Studio projects will complain about missing files. It doesn't matter if you build the tree before or after Visual Studio projects are generated. But, we might as well get it out of the way. From your MozillaBuild environment, run:

./mach build

That should just work. If it doesn't, you may need to configure mach.ini. See my previous post on how to configure mach.ini. As a reference, my Windows config is:

[build]

configure_extra = --disable-webgl

[compiler]

[paths]
source_directory = c:\dev\src\mozilla-central-git
object_directory = c:\dev\src\mozilla-central-git\objdir

Now, to generate Visual Studio project files:

./mach backendconfig visualstudio

That should take about a minute to finish. When it's done, it should have created objdir/msvc/mozilla.sln. You should be able to load that in Visual Studio!

You will need to regenerate Visual Studio project files when the build config changes. As a rule of thumb, do this every time you pull source. You don't need to perform a full build before you generate Visual Studio files (you do need to perform configure, however). However, if you have not performed a full build, Visual Studio may not be able to find some files, like headers generated from IDLs.

Please close the solution before regenerating the project files. If you don't, Visual Studio puts up a modal dialog for each project file that changed and you have to click through over a hundred of these. It's extremely frustrating. I'm investigating workarounds.

Current State

Currently, it only generates projects for C/C++ compilation (libraries). I still need to add support for IDL, headers, etc. However, each project has proper compiler flags, header search paths, etc. So, IntelliSense is happy and some things do manage to compile!

Many parts are broken and sub-par.

I've only tested on Visual Studio 2008. If you are running Visual Studio \2010, you can try to upgrade the solution. This may work. The backend supports generating solutions for different versions. But, I haven't tested things work on non-2008 and I don't want to expose untested behavior.

Compiling within Visual Studio works for some things. On my system, I get a lot of nullptr not defined errors. I'm not sure why. This will hopefully be worked out soon.

If you do manager to compile within Visual Studio, the output files don't go in the right places. So, if you do a build from the command-line, it will have to re-compile to pick up changes.

Project names are based on the name of the library they produce. I'm not sure if this is the best solution.

Project dependencies are not set up. They will be added later.

Projects for linking libxul or building firefox.exe are not yet provided. Along the same vein, debugging support is not built-in. I'm working on it.

Basically, IntelliSense works. You can now use Visual Studio as a rich editor. Hopefully this is a step in the right direction.

I'm anxious to hear if this works for other people. Please leave comments!

Would you like to build Firefox faster? If so, do the following:

hg qimport http://people.mozilla.org/~gszorc/build-splendid.patch
hg qpush
rm .mozconfig* (you may want to create a backup first)
./mach build

This should just work on OS X, Linux, and other Unix-style systems. Windows support is currently broken, sorry.

mach can do much more than build. Run the following to see what:

./mach --help

Important Info

mach replaces client.mk. mach has its own configuration file. The first time you run mach, it will create the file mach.ini in the same directory as the mach script. This is your new mozconfig file.

The default mach.ini places the object directory into the directory objdir under the top source directory. It also builds an optimized binary without debug info.

Run the following to see which config settings you can add to mach.ini:

./mach settings-create
./mach settings-list

This may fail because I'm still working out the kinks with gettext. If it doesn't work, open python/mozbuild-bs/mozbuild/base.py and search for _register_settings. Open python/mozbuild-bs/mozbuild/locale/en-US/LC_MESSAGES/mozbuild.po for the help messages.

As a point of reference, my mach.ini looks like the following:

[build]
application = browser

configure_extra = --enable-dtrace --enable-tests

[compiler]
cc = /usr/local/llvm/bin/clang
cxx = /usr/local/llvm/bin/clang++

cflags = -fcolor-diagnostics
cxxflags = -fcolor-diagnostics

[paths]
source_directory = /Users/gps/src/mozilla-central-git
object_directory = /Users/gps/src/mozilla-central-git/objdir

I am on OS X and am using a locally-built version of LLVM/Clang, which I have installed to /usr/local/llvm.

You'll notice there are no options to configure make. The patch automatically selects optimal settings for your platform!

Known Issues and Caveats

This is alpha. It works in scenarios in which I have tested it, mainly building the browser application on OS X and Linux. There are many features missing and likely many bugs.

I have been using this as part of my day-to-day development for weeks. However, your mileage may vary.

As stated above, Windows support is lacking. It will appear to work, but things will blow up during building. Don't even try to use it on Windows.

There are likely many bugs. Please don't file Bugzilla bugs, as this isn't part of the build system just yet.

This patch takes over the build system. Do not attempt to use client.mk or run make directly with this patch applied.

If you encounter an issue, your methods of recourse are:

1. Post a comment on this blog post
2. Ping me on irc.mozilla.org. My nick is gps. Try the #buildfaster channel.
3. Send an email to gps@mozilla.com

I am particularly interested in exceptions and build failures.

If you encounter an issue building with this, just reverse the patch and build like you have always done (don't forget to restore your mozconfig file).

If mach.ini does not support everything you were doing in your mozconfig, please send me a copy of your mozconfig so I can implement whatever you need.

Other Info

I will likely write a follow-up post detailing what's going on. If you are curious, the code lives in python/mozbuild-bs. The backend and frontend sub-packages are where the magic is at. Once the backend has been configured, check out hybridmake.mk and all of the splendid.mk files in the object directory.

I am particularly interested in the real-world impact of this patch on people's build times. In this early version of the patch, you likely won't see drastic speed increases. On my MacBook Pro with an SSD, I see end-to-end clobber build times decrease by over a minute. With a little more work, I should be able to shave another minute or two off of that.

I will try to keep the patch up-to-date as I improve the build system. Refresh early and often.