Starting today, a Mozilla LDAP account with Mercurial SSH access is no longer required to hg push into MozReview to initiate code review with Mozilla projects.

The instructions for configuring your client to use MozReview have been updated to reflect how you can now push to MozReview over HTTP using a Bugzilla API Key for authentication.

This change effectively enables first-time contributors to use MozReview for code review. Before, you had to obtain an LDAP account and configure your SSH client, both of which could be time consuming processes and therefore discourage people from contributing. (Or you could just use Bugzilla/Splinter and not get the benefits of MozReview, which many did.)

I encourage others to update contribution docs to start nudging people towards MozReview over Bugzilla/patch-based workflows (such as bzexport).

Bug 1195856 tracked this feature.

For the past few months, Mozilla has been serving Mercurial clones from Amazon S3. We upload snapshots (called bundles) of large and/or high-traffic repositories to S3. We have a custom Mercurial extension on the client and server that knows how to exchange the URLs for these snapshots and to transparently use them to bootstrap a clone. The end result is drastically reduced Mercurial server load and faster clone times. The benefits are seriously ridiculous when you operate version control at scale.

Amazon CloudFront is a CDN. You can easily configure it up to be backed by an S3 bucket. So we did.

https://hg.cdn.mozilla.net/ is Mozilla's CDN for hosting Mercurial data. Currently it's just bundles to be used for cloning.

As of today, if you install the bundleclone Mercurial extension and hg clone a repository on hg.mozilla.org such as mozilla-central (hg clone https://hg.mozilla.org/mozilla-central), the CDN URLs will be preferred by default. (Previously we preferred S3 URLs that hit servers in Oregon, USA.)

This should result in clone time reductions for Mozillians not close to Oregon, USA, as the CloudFront CDN has servers all across the globe and your Mercurial clone should be bootstrapped from the closest and hopefully therefore fastest server to you.

Unfortunately, you do need the the aforementioned bundleclone extension installed for this to work. But, this should only be temporary: I've proposed integrating this feature into the core of Mercurial so if a client talks to a server advertising pre-generated bundles the clone offload just works. I already have tentative buy-in from one Mercurial maintainer. So hopefully I can land this feature in Mercurial 3.6, which will be released November 1. After that, I imagine some high-traffic Mercurial servers (such as Bitbucket) will be very keen to deploy this so CPU load on their servers is drastically reduced.

I added a feature to Mercurial 3.4 that exposes JSON from Mercurial's various web APIs. Unfortunately, due to the presence of legacy code on hg.mozilla.org providing similar functionality, we weren't able to deploy this feature to hg.mozilla.org when we deployed Mercurial 3.4 several weeks ago.

I'm pleased to announce that as of today, JSON is now exposed from hg.mozilla.org!

To access JSON output, simply add json- to the command name in URLs. e.g. instead of https://hg.mozilla.org/mozilla-central/rev/de7aa6b08234 use https://hg.mozilla.org/mozilla-central/json-rev/de7aa6b08234. The full list of web commands, URL patterns, and their parameters are documented in the hgweb help topic.

Not all web commands support JSON output yet. Not all web commands expose all data available to them. If there is data you need but isn't exposed, please file a bug and I'll see what I can do.

Thanks go to Steven MacLeod for reviewing the rather large series it took to make this happen.

Sometime last week we enabled a new API on hg.mozilla.org: json-mozbuildinfo. This endpoint will return JSON describing moz.build-derived metadata about the files that changed in a commit.

Example. Docs.

We plan to eventually leverage this API to do cool things like have MozReview automatically file bugs in the appropriate component and assign appropriate reviewers given the set of changed files in a commit.

The API is currently only available on mozilla-central. And, we have very conservative resource limits in place. So large commits may cause it to error out. As such, the API is considered experimental. Also, performance is not as optimal as it could be. You have to start somewhere.

I'd like to thank Guillaume Destuynder (kang) for his help with the security side of things. When I started on this project, I didn't think I'd be writing C code for spawning secure processes, but here we are. In the not so distant future, I'll likely be adding seccomp(2) into the mix, which will make the execution environment as or more secure than the Firefox content process sandbox, depending on how it is implemented. The rabbit holes we find ourselves in to implement proper security...

Just a few minutes ago, hg.mozilla.org reached an important milestone: deployments are now performed via Ansible from our open source version-control-tools repository instead of via Puppet from Mozilla's private sysadmins repository. This is important for a few reasons.

First, the code behind the operation of hg.mozilla.org is now open source and available for the public to see and change. I strive for my work at Mozilla to be open by default. With hg.mozilla.org's private Puppet repository, people weren't able to see what was going on under the covers. Nor were they empowered to change anything. This may come as a shock, but even I don't have commit privileges to the internal Puppet repository that was previously powering hg.mozilla.org! I did have read access. But any change I wanted to make involved me proxying it through one of two people. It was tedious, made me feel uncomfortable for having to nag people to do my work, and slowed everyone down. We no longer have this problem, thankfully.

Second, having the Ansible code in version-control-tools enables us to use the same operational configuration in production as we do in our Docker test environment. I can now spin up a cluster of Docker containers that behave very similarly to the production servers (which aren't running Docker). This enables us to write end-to-end tests of complex systems running across multiple Docker containers and have relatively high confidence that our production and testing environments behave very similarly. In other words, I can test complex interactions between multiple systems all from my local machine - even from a plane! For example, we can and do test that SSH connections to a simulated production environment running in Docker behave as expected, complete with an OpenSSH server speaking to an OpenLDAP server for SSH public key lookup. While we still have many tests to write, we had no such tests a year ago and every production deployment was a cross-your-fingers type moment. Having comprehensive tests gives us confidence to move fast and not break things.

One year ago, hg.mozilla.org's infrastructure was opaque, didn't have automated tests, and was deployed too seldomly. There was the often correct perception that changing this critical-to-Mozilla service was difficult and slow. Today, things couldn't be more different. The hg.mozilla.org infrastructure is open, we have tests, and we can and do deploy multiple times per day without forward notice and without breaking things. I love this brave new world of open infrastructure and moving fast.