Performance/MemShrink

MemShrink is a project that aims to reduce Firefox's memory consumption. There are three potential benefits:

1. Speed: less cache pressure, and less paging. The latter is crucial, as it can destroy performance.
2. Stability: fewer OOMs, whether due to address space exhaustion or otherwise. This results in fewer crashes (due to mishandling of OOM) or aborts.
3. Perception: fewer people will complain about Firefox being a memory hog.

These factors are doubly important for Firefox Mobile.

There are two main facets to this:

1. "Slimmer" memory usage, e.g. more space-efficient data structures.
2. Avoiding "leaks". This loose use of the term (which is used throughout this document) includes:

• True leaks, where memory is lost forever.
• Lifetime issues, where memory is not reclaimed until you close the page/tab/window/process.
• Collection heuristic issues (e.g. GC is too infrequent in certain cases).
• Bad cache algorithms and poorly tuned caches.
• Fragmentation.

Leaks are generally more important, as they are more likely to lead to horrible performance.

Meetings

Meetings will happen every Tuesday at 1pm Pacific time.

• Dial-in: Audio-only conference# 95312
  • People with Mozilla phones or softphones please dial x4000 Conf# 95312
  • US/Toll-free: +1 800 707 2533, (pin 4000) Conf# 95312
  • US/California/Mountain View: +1 650 903 0800, x4000 Conf# 95312
  • US/California/San Francisco: +1 415 762 5700, x4000 Conf# 95312
  • US/Oregon/Portland: +1 971 544 8000, x4000 Conf# 95312
  • CA/British Columbia/Vancouver: +1 778 785 1540, x4000 Conf# 95312
  • CA/Ontario/Toronto: +1 416 848 3114, x4000 Conf# 95312
  • UK/London: +44 (0)207 855 3000, x4000 Conf# 95312
  • FR/Paris: +33 1 84 88 37 37, x4000 Conf# 95312
  • Gmail Chat (requires Flash and the Google Talk plugin): paste +1 650 903 0800 into the Gmail Chat box that doesn't look like it accepts phone numbers
  • SkypeOut is free if you use the 800 number
• Vidyo: Warp Core
• IRC: #memshrink
• Etherpad: memshrink (copied to wiki after the meeting)

Ideally, these weekly meetings would only be needed for a medium length of time, say 1 or 2 quarters. Hopefully after that, things will have improved enough that meetings could be less frequent (bi-weekly or monthly) or not needed at all.

Minutes and Progress Reports

* 2011-10-11 minutes; (no weekly report)

• 2011-10-04 minutes; (no weekly report)
• 2011-09-20 minutes; (no weekly report)
• 2011-08-30 minutes; Week 11 report
• 2011-08-23 minutes; Week 10 report
• 2011-08-16 minutes; Week 9 report
• 2011-08-09 minutes; Week 8 report
• 2011-08-02 (no minutes taken); Week 7 report
• 2011-07-26 minutes; Week 6 report
• 2011-07-19 minutes; Week 5 report
• 2011-07-12 minutes; Week 4 report
• 2011-07-05 minutes; Week 3 report
• 2011-06-28 (no minutes taken); Week 2 report
• 2011-06-21 (no minutes taken); Week 1 report
• 2011-06-14 (no minutes taken)

Bug Tracking

Bugs tracked by the MemShrink project are prioritized by adding one of "MemShrink:P1", "MemShrink:P2" or "MemShrink:P3" to the whiteboard.

• Unprioritized MemShrink bugs. These are triaged regularly in meetings.
• MemShrink:P1 bugs. These are discussed regularly in meetings.
• MemShrink:P2 bugs. These are discussed occasionally in meetings. P2 is the default priority.
• MemShrink:P3 bugs. These are discussed rarely in meetings.

Some other interesting bug lists.

• All MemShrink bugs.
• Unconfirmed MemShrink bugs. These are problems reported by users and generally require some kind of additional work to confirm.
• MemShrink bugs with a "mentor" annotation. These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
• Unassigned MemShrink bugs. These need someone to work on them.

There's also a tracking bug for bugs relating to profiling, tools and infrastructure. (These bugs overlap several of the above lists.)

Goals

The real goal is to minimize bad performance (primarily paging) caused by poor memory usage. But that's hard to track.

A more trackable goal is to get the number of MemShrink P1 bugs down to zero. That will mean that all the bad leaks will have been fixed, and also the important auxiliary work (e.g. infrastructure to detect regressions) will be in place.

Current Infrastructure

The main automated infrastructure we currently have is the Tp4 and Tp5 Talos tests. They load a bunch of popular (Alexa top 100) pages sequentially, take memory measurements (eg. private bytes on Windows) periodically, and somehow average those measurements into a single number. If that number increases significantly, certain people receive a warning email.

Problems with this:

• Sometimes increases in the measurements fall through the cracks. I'm not sure what to do about this. The variation in measurements between runs doesn't help here.

• They open only one tab at a time. This doesn't reflect how people browse, and doesn't stress the browser. In comparison, Sayre's Membuster benchmark opens a heap of new tabs.

• The "average of a sample taken every 20 seconds" is a curious measure.

• The metrics are odd and inconsistent across platforms. Using the "explicit" and "resident" memory reporters (see about:memory for details) would be better. Some statistics about page faults would also be good.

Even with these flaws, they're a good start, but there's room to improve.

Additional Infrastructure

• Automated testing using other existing tools (Valgrind, trace-malloc) should be set up. (Valgrind is covered by bug 631811.)

• jst has some ideas about using static analysis in place to automatically detect some kinds of leaks (bug 423032).

• Telemetry will help a lot. There would be 2 kinds of telemetry: built-in idle-daily stats and an addon. Builtin idle-daily would report a sanitized version of about:memory, number of open tabs, among other non-privacy-invasive stats. We would also distribute a telemetry addon allowing users to opt into more invasive telemetry. This addon would be able to run benchmarks, and allow the users to explicitly trigger submission of detailed privacy/perf-invasive information (such as open websites). (bug 663423)

• Mozmill endurance tests can track memory usage over time, which is much more informative than snapshot measurements or averaged measurements. It's unclear how much overhead they introduce, however. Leveraging them somehow could avoid a lot of duplicated work.

areweslimyet.com

jmuizelaar set up AWSY, inspired by AWFY.

Why was AWFY so successful? Some key characteristics.

• The benchmarks were easy to choose: everybody already used SS and V8, and then Mozilla released Kraken. They run quickly, too, which is nice for devs, and allows expensive tools (like Cachegrind) to be used.
• The metrics were easy to choose. SS time, V8 time, Kraken time. They can be measured easily, precisely, and fairly repeatably. You can break the total time down into per-benchmark times, which really helps with understanding improvements and regressions. They're easy for devs to run on their own machine.
• Understanding where the time went was fairly easy, because there are good time-oriented profilers: Shark, Cachegrind, etc.
• There was a well-defined goal: match or beat the other browsers.

In comparison, for AWSY there is no good and/or standard benchmark suite; metrics are less clear and/or harder to measure; the memory profiling tools aren't as good; but most importantly, there's no well-defined goal w.r.t. any benchmarks.

If we were focusing mostly on memory slimming a graph like AWFY's could work well. But given the primary goal above (zero MemShrink P1 bugs), areweslimyet.com just points to the list of open MemShrink P1 bugs.