Mercurial 3.5 was released today (following Mercurial's time-based schedule of releasing a new version every 3 months).

There were roughly 1000 commits between 3.4 and 3.5, making this a busy version. Although, 1000 commits per release has become the new norm, as development on Mercurial has accelerated in the past few years.

In my mind, the major highlight of Mercurial 3.5 is that the new bundle2 wire protocol for transferring data during hg push and hg pull is now enabled by default on the client. Previously, it was enabled by default only on the server. hg.mozilla.org is running Mercurial 3.4, so clients that upgrade to 3.5 today will be speaking to it using the new wire protocol.

The bundle2 wire protocol succeeds the existing protocol (which has been in place for years) and corrects many of its deficiencies. Before bundle2, pull and push operations were not atomic because Mercurial was performing a separate API call for each piece of data. It would start by transferring changeset data and then have subsequent transfers of metadata like bookmarks and phases. As you can imagine, there were race conditions and scenarios where pushes could be incomplete (not atomic). bundle2 transfers all this data in one large chunk, so there are much stronger guarantees for data consistency and for atomic operations.

Another benefit of bundle2 is it is a fully extensible data exchange format. Peers can add additional parts to the payload. For extensions that wish to transfer additional metadata (like Mozilla's pushlog data), they can simply add this directly into the data stream without requiring additional requests over the wire protocol. This translates to fewer network round trips and faster push and pull operations.

The progress extension has been merged into Mercurial's core and is enabled by default. It is now safe to remove the extensions.progress config option from your hgrc.

Mercurial 3.5 also (finally) drops support for Python 2.4 and 2.5. Hopefully nobody reading this is still running these ancient and unsupported versions of Python. This is a win for Mercurial developers, as we were constantly having to work around deficiencies with these old Python releases. There were dozens of commits removing hacks and workarounds for Python 2.4 and 2.5. Dropping 2.4 and 2.5 also means Python 3 porting can begin in earnest. However, this isn't a high priority for anyone, so don't hold your breath.

There were a number of performance improvements in 3.5:

• operations involving obsolescence markers are faster (for users of changeset evolution)
• various revsets were optimized
• parts of phases calculation are now performed in C. The not public() revset should be much faster.
• hg status and things walking the filesystem are faster (Mozillians should be using hgwatchman to make hg status insanely fast)

A ui.allowemptycommit config option was introduced to control whether empty commits are allowed. Mozillians manually creating trychooser commits may run into problems creating empty commits without this option (a better solution is to use mach push-to-try).

Work is progressing on per-directory manifests. Currently, Mercurial stores the mapping of files to content in a giant list called the manifest. For repositories with tens or hundreds of thousands of files, decoding and reading large manifests is very CPU intensive. Work is being done to enable Mercurial to split manifests by directory. So instead of a single manifest, there are several. This is a prequisite to narrow clone, which is the ability to clone history for a subset of files (like how Subversion works). This work will eventually enable repositories with millions of files to exist without significant performance loss. It will also allow monolithic repositories to exist without the common critique that they are too unwieldy to use because they are so large.

hgignore files now have an include: and subinclude: syntax that can be used to include other files containing ignore rules. This feature is useful for a number of reasons. First, it makes sense for ignore rules to live in the directory hierarchy next to paths they impact. Second, for people working with monolithic repositories, it means you can export a sub-directory of your monorepo (to e.g. a Git repository) and its ignore rules - being defined in local directories - can still work. (I'm pretty sure Facebook is using this approach to make its syncing of directories/projects from its Mercurial monorepo to GitHub easier to manage.)

Significant work has been done on the template parser. If you have written custom templates, you may find that Mercurial 3.5 is more strict about parsing certain syntax.

Revsets with chained or no longer result in stack exhaustion. Before, programmatically generated revsets like 1 or 2 or 3 or 4 or 5 or 6... would likely fail.

Interactions with servers over SSH should now display server output in real time. Before, server output was buffered and only displayed at the end of the operation. (You may not see this on hg.mozilla.org until the server is upgraded to 3.5, which is planned for early September.)

There are now static analysis checks in place to ensure that Mercurial config options have corresponding documentation in hg help config. As a result, a lot of formerly undocumented options are now documented.

I contributed various improvements. These include:

• auto sharing repository data during clone
• clone and pull performance improvements
• hg help scripting

There were tons of other changes, of course. See the official release notes and the upgrade notes for more.

The Mercurial for Mozillians Installing Mercurial article provides a Mozilla tailored yet generally applicable guide for installing or upgrading Mercurial to 3.5. As always, conservative software users may want to wait until September 1 for the 3.5.1 point release to fix any issues or regressions from 3.5.