Firefox/Python 3 Migration

Why This is Important

In mozilla-central there are over 3500 Python files (excluding third party files), comprising roughly 230k lines of code. Most of it is Python 2.[1]

Python 2 will reach it's End Of Life on January 1st 2020. This page collects links and resources to assist the migration of the Firefox ecosystem off of Python 2 and onto Python 3.

How We Get There

As of today (May 2019) we require both the Python 2 and Python 3 interpreters to be installed to build and develop Firefox.

Next we:

1. Make Python 3 porting and development safe by adding a Python 3 option to the developer toolchain. (mach lint, coverage.py, parts of the test suite)
2. Make key support libraries run in both Python 2 and Python 3. (mach-core)
3. Build interpreter switching mechanisms into the toolchain so we can run sub-components in just 2 or just 3. (mach sub-commands, py_action)
4. Port Python 2-only components to Python 3 piecemeal.
5. Remove Python 2 compatibility. (When that might happen needs discussion. Possibly sometime in 2020.)

Getting Involved

Find the others

Most discussion happens in the #python channel on Slack (NDA'd Mozillians only).

Find a place to start hacking

Bug 1496527 tracks the migration work across Firefox.

Help port mach-core to Python 3 (bug 1473498)

1. Edit python/mach/mach/test/python.ini to remove the skip-if
2. Run: ./mach python-test --python 3 python/mach
3. Fix the errors until all tests pass

Remove more of the "excluded" paths from the py2 and py3 linters

Remove a line from https://searchfox.org/mozilla-central/source/tools/lint/py3.yml or https://searchfox.org/mozilla-central/source/tools/lint/py2.yml. Run the linter, see if the module passes.

Run all python source tests with Python 3 and see what breaks

Make sure each of the tasks listed in https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/python.yml has Python 3 specified, and do a try push, enabling all the python source tests.

See what breaks, then fix it!

How to Port the Code

Before starting

If you haven't ported modules before (or need a refresher), read the Porting Guide on python.org. It has a good outline of the porting process.

Porting modules in the Firefox source tree

• We've already standardized on Python 2.7 across the tree.
• Make sure you can test the code you're working on with python3!
  • If using `tox`, then adding `py35`, and `py36` to the list of environments should be sufficient.
  • If using |mach python-test|, make sure the relevant manifest doesn't skip-if Python 3 and run |mach python-test --python 3 <path>|.
• `futurize` Stage 1 fixes can be used to transform code into code that's compatible with both python2.7 and python3.
• The `six` module is available in-tree, feel free to use it as a dependency.
• Black can format Python 2 code and Python 3 code (but it requires Py3.6+ to run).

Porting Resources

Porting Guide on python.org

Cheat Sheet: Writing Python 2-3 compatible code. Note: misses some advanced changes, like porting __eq__() to __hash__() and changes to the slice interface.

References

Bug 1496527 tracks the migration

https://ahal.ca/blog/2019/python-3-at-mozilla

Python 3 tree migration notes

mach Python 3 migration roadmap