Performance/MemShrink: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
No edit summary
m (added zoom number)
 
(81 intermediate revisions by 12 users not shown)
Line 1: Line 1:
MemShrink is a project that aims to reduce Firefox's memory consumption.  There are three potential benefits:
MemShrink is a project that aims to reduce the memory consumption of Firefox (on desktop and mobile).  There are three potential benefits.


# Speed.  Less cache pressure, less paging, and fewer/smaller GC and CC pauses.
# Speed.  Firefox will be faster due to less cache pressure, less paging, and fewer/smaller GC and CC pauses.  Changes that reduce memory consumption but make Firefox slower are not desirable.
# Stability.  Fewer aborts/crashes due to physical or virtual memory exhaustion.  The latter is mostly a problem on 32-bit Windows builds, the former is mostly a problem on mobile.
# Stability.  Firefox will suffer fewer aborts/crashes due to virtual or physical memory exhaustion.  The former is mostly a problem on 32-bit Windows builds with a 2GB or 4GB virtual memory limit, the latter is mostly a problem on mobile devices that lack swap space.
# Reputation.  Fewer people will complain about Firefox being a memory hog, and fewer people will complain that Mozilla ignores memory usage.
# Reputation.  Fewer people will complain that Firefox is a memory hog and that Mozilla ignores memory usage.


These factors are doubly important for Firefox Mobile.
[http://blog.mozilla.com/nnethercote/2012/02/08/the-benefits-of-reducing-memory-consumption-2/ This blog post] describes these benefits in more detail.


There are two main facets to this:
There are two main ways to reduce memory consumption.


# "Slimmer" memory usage, e.g. more space-efficient data structures.
# "Slim down" memory usage, e.g. make data structures more space-efficient.
# Avoiding "leaks".  This loose use of the term (which is used throughout this document) includes:
# Avoid "leaks".  This loose use of the term (which is used throughout this document) includes:
* True leaks, where memory is lost forever.
#* True leaks, where memory is lost forever.
* Lifetime issues, where memory is not reclaimed until you close the page/tab/window/process.
#* Lifetime issues, where memory is not reclaimed until you close the page/tab/window/process.
* Collection heuristic issues (e.g. GC is too infrequent in certain cases).
#* Collection heuristic issues (e.g. GC is too infrequent).
* Bad cache algorithms and poorly tuned caches.
#* Bad cache algorithms and poorly tuned caches.
* Fragmentation.
#* Fragmentation.


Leaks are generally more important, as they are more likely to lead to horrible performance.
Leaks are generally more important, because they are more likely to lead to horrible performance.


== Meetings ==
== Meetings ==
Meetings will happen every Tuesday at 2pm Pacific time.
Meetings happen every second Wednesday at 11am Pacific time ([http://arewemeetingyet.com/Los%20Angeles/2014-10-14/16:00/b/MemShrink%20Meeting countdown in your timezone])


{{conf|95346}}
{{conf|95346}}
* Vidyo: PB&J
* zoom: 310 455 675
* IRC: [irc://irc.mozilla.org/memshrink #memshrink]
* IRC: [irc://irc.mozilla.org/memshrink #memshrink]
* Etherpad: [https://etherpad.mozilla.org/memshrink memshrink] (copied to wiki after the meeting)


Ideally, these weekly meetings would only be needed for a medium length of time, say 1 or 2 quarters.  Hopefully after that, things will have improved enough that meetings could be less frequent (bi-weekly or monthly) or not needed at all.
== Bug Tracking ==
 
== Minutes and Progress Reports  ==
 
*2011-11-15 (no minutes taken);  [http://blog.mozilla.com/nnethercote/2011/11/16/memshrink-progress-report-week-22/ Week 22 report]
*[[Performance/MemShrink/Meetings/2011-11-08|2011-11-08 minutes]]; [http://blog.mozilla.com/nnethercote/2011/11/09/memshrink-progress-week-21/ Week 21 report]
*[[Performance/MemShrink/Meetings/2011-11-01|2011-11-01 minutes]]; [http://blog.mozilla.com/nnethercote/2011/11/02/memshrink-progress-week-20/ Week 20 report]
*[[Performance/MemShrink/Meetings/2011-10-25|2011-10-25 minutes]]; [http://blog.mozilla.com/nnethercote/2011/10/26/memshrink-progress-week-19/ Week 19 report]
*[[Performance/MemShrink/Meetings/2011-10-18|2011-10-18 minutes]]; [http://blog.mozilla.com/nnethercote/2011/10/19/memshrink-progress-weeks-13-18/ Weeks 13--18 report]
*[[Performance/MemShrink/Meetings/2011-10-11|2011-10-11 minutes]]; (no weekly report)
*[[Performance/MemShrink/Meetings/2011-10-04|2011-10-04 minutes]]; (no weekly report)
*2011-09-27 (no minutes taken); (no weekly report)
*[[Performance/MemShrink/Meetings/2011-09-20|2011-09-20 minutes]]; (no weekly report)
*2011-09-13 (no minutes taken); (no weekly report)
*[[Performance/MemShrink/Meetings/2011-09-06|2011-09-06 minutes]]; [http://blog.mozilla.com/nnethercote/2011/09/07/memshrink-progress-week-12/ Week 12 report]
*[[Performance/MemShrink/Meetings/2011-08-30|2011-08-30 minutes]]; [http://blog.mozilla.com/nnethercote/2011/08/31/memshrink-progress-week-11/ Week 11 report]
*[[Performance/MemShrink/Meetings/2011-08-23|2011-08-23 minutes]]; [http://blog.mozilla.com/nnethercote/2011/08/24/memshrink-progress-week-10/ Week 10 report]
*[[Performance/MemShrink/Meetings/2011-08-16|2011-08-16 minutes]]; [http://blog.mozilla.com/nnethercote/2011/08/17/memshrink-progress-week-9/ Week 9 report]
*[[Performance/MemShrink/Meetings/2011-08-09|2011-08-09 minutes]]; [http://blog.mozilla.com/nnethercote/2011/08/10/memshrink-progress-week-8/ Week 8 report]
*2011-08-02 (no minutes taken); [http://blog.mozilla.com/nnethercote/2011/08/03/memshrink-progress-week-7/ Week 7 report]
*[[Performance/MemShrink/Meetings/2011-07-26|2011-07-26 minutes]]; [http://blog.mozilla.com/nnethercote/2011/07/27/memshrink-progress-week-6/ Week 6 report]
*[[Performance/MemShrink/Meetings/2011-07-19|2011-07-19 minutes]]; [http://blog.mozilla.com/nnethercote/2011/07/20/memshrink-progress-week-5/ Week 5 report]
*[[Performance/MemShrink/Meetings/2011-07-12|2011-07-12 minutes]]; [http://blog.mozilla.com/nnethercote/2011/07/13/memshrink-progress-week-4/ Week 4 report]
*[[Performance/MemShrink/Meetings/2011-07-05|2011-07-05 minutes]]; [http://blog.mozilla.com/nnethercote/2011/07/06/memshrink-progress-week-3/ Week 3 report]
*2011-06-28 (no minutes taken); [http://blog.mozilla.com/nnethercote/2011/06/29/memshrink-progress-week-2/ Week 2 report]
*2011-06-21 (no minutes taken); [http://blog.mozilla.com/nnethercote/2011/06/22/memshrink-progress-week-1/ Week 1 report]
*2011-06-14 (no minutes taken)


== Bug Tracking ==
Bugs are prioritized during MemShrink meetings.


Bugs tracked by the MemShrink project are prioritized by adding one of "MemShrink:P1", "MemShrink:P2" or "MemShrink:P3" to the whiteboard.
If you want a bug that is related to memory usage to be triaged in a MemShrink meeting, add "[MemShrink]" to the bug whiteboard. After prioritization, this whiteboard marking will be changed to one of "[MemShrink:P1]", "[MemShrink:P2]", "[MemShrink:P3]" or "[MemShrink:meta]". The meaning of these markings, and links to all such bugs, is as follows.  


* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=MemShrink%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized MemShrink bugs].  These are triaged regularly in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=MemShrink%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized MemShrink bugs].  These are prioritized regularly in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P1 bugs]. These are discussed regularly in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P1 bugs]. Once prioritized, these will be revisited and discussed occasionally in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P2 bugs]. These are discussed occasionally in meetings. P2 is the default priority.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P2 bugs]. Once prioritized, these will be revisited and discussed rarely in meetings. P2 is the default priority.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P3 bugs]. These are discussed rarely in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P3 bugs]. Once prioritized, these are unlikely to be revisited and discussed again in meetings.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3Ameta&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:meta bugs]. Once prioritized, these will be revisited and discussed occasionally in meetings.


Some other interesting bug lists.
Some other interesting bug lists that overlap with the ones above.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id All MemShrink bugs].
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id All MemShrink bugs].
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252875&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&resolution=---&resolution=DUPLICATE Unconfirmed MemShrink bugs]. These are problems reported by users and generally require some kind of additional work to confirm.
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252875&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&resolution=---&resolution=DUPLICATE Unconfirmed MemShrink bugs]. These are problems reported by users and generally require some kind of additional work to confirm.
* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252905&status_whiteboard=MemShrink%20mentor&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE MemShrink bugs with a "mentor" annotation]. These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
* [https://bugzilla.mozilla.org/buglist.cgi?f1=bug_mentor&list_id=12543462&o1=isnotempty&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED MemShrink bugs with a "mentor" annotation]. These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=1252950&resolution=---&resolution=DUPLICATE&emailtype1=regexp&status_whiteboard_type=allwordssubstr&emailassigned_to1=1&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=nobody%40%7Cgeneral%40js.bugs Unassigned MemShrink bugs]. These need someone to work on them.
* [https://bugzilla.mozilla.org/buglist.cgi?list_id=1252950&resolution=---&resolution=DUPLICATE&emailtype1=regexp&status_whiteboard_type=allwordssubstr&emailassigned_to1=1&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=nobody%40%7Cgeneral%40js.bugs Unassigned MemShrink bugs]. These need someone to work on them.
 
There's also a [https://bugzilla.mozilla.org/show_bug.cgi?id=640791 tracking bug] for bugs relating to profiling, tools and infrastructure.  (These bugs overlap several of the above lists.)
 
== Current Infrastructure ==
 
The main automated infrastructure we currently have is the Tp4 and Tp5 Talos tests.  They load a bunch of popular (Alexa top 100) pages sequentially, take memory measurements (eg. private bytes on Windows) periodically, and somehow average those measurements into a single number.  If that number increases significantly, certain people receive a warning email.
 
Problems with this:
* Sometimes increases in the measurements fall through the cracks.  I'm not sure what to do about this.  The variation in measurements between runs doesn't help here.
 
* They open only one tab at a time.  This doesn't reflect how people browse, and doesn't stress the browser.  In comparison, Sayre's [https://bugzilla.mozilla.org/show_bug.cgi?id=631705 Membuster] benchmark opens a heap of new tabs.
 
* The "average of a sample taken every 20 seconds" is a curious measure.
 
* The metrics are odd and inconsistent across platforms.  Using the "explicit" and "resident" memory reporters (see about:memory for details) would be better.  Some statistics about page faults would also be good.
 
Even with these flaws, they're a good start, but there's room to improve.
 
== Additional Infrastructure ==


* Automated testing using other existing tools (Valgrind, trace-malloc) should be set up. (Valgrind is covered by {{bug|631811}}.)
== Tools ==


* jst has some ideas about using static analysis in place to automatically detect some kinds of leaks ({{bug|423032}}).
All the tools that we use to measure and improve memory usage are described [https://developer.mozilla.org/en-US/docs/Mozilla/Performance on MDN], under "Memory profiling and leak detection tools".


* Telemetry will help a lot.  There would be 2 kinds of telemetry: built-in idle-daily stats and an addon.  Builtin idle-daily would report a sanitized version of about:memory, number of open tabs, among other non-privacy-invasive stats.  We would also distribute a telemetry addon allowing users to opt into more invasive telemetry. This addon would be able to run benchmarks, and allow the users to explicitly trigger submission of detailed privacy/perf-invasive information (such as open websites).  ({{bug|663423}})
== Meeting Minutes, Progress Reports and Presentations ==


* Mozmill endurance tests can track memory usage over time, which is much more informative than snapshot measurements or averaged measurements. It's unclear how much overhead they introduce, however. Leveraging them somehow could avoid a lot of duplicated work.
We no longer take meeting minutes nor make progress reports, but the meetings still happen. Brief MemShrink updates are given in the minutes of the weekly Platform meeting ([https://wiki.mozilla.org/Platform#Meeting_Notes wiki], [https://blog.mozilla.org/meeting-notes/archives/tag/mozillaplatform blog]). And nnethercote still [https://blog.mozilla.org/nnethercote/category/memshrink/ blogs] about major MemShrink occurrences.


== areweslimyet.com ==
Here is the historical record from when we did take minutes and make progress reports.


jmuizelaar set up [http://areweslimyet.com/ AWSY], inspired by [http://arewefastyet.com AWFY].
{| border="1" cellpadding="2"
! Date !! Minutes !! Progress report
|-
| 2014-06-16 || - || [https://blog.mozilla.org/nnethercote/2014/06/16/memshrinks-3rd-birthday/ MemShrink's 3rd Birthday] (includes the fourth "Big Ticket Items" list)
|-
| 2013-11-26 || - || [https://blog.mozilla.org/nnethercote/2013/11/27/memshrink-progress-final/ The final progress report]
|-
| 2013-10-29 || - || [https://blog.mozilla.org/nnethercote/2013/10/30/memshrink-progress-week-121-124/ Week 121--124]
|-
| 2013-09-30 || - || [https://blog.mozilla.org/nnethercote/2013/10/01/memshrink-progress-week-117-120/ Week 117--120]
|-
| 2013-09-03 || - || [https://blog.mozilla.org/nnethercote/2013/09/04/memshrink-progress-week-113-116/ Week 113--116]
|-
| 2013-08-06 || - || [https://blog.mozilla.org/nnethercote/2013/08/07/memshrink-progress-week-109-112/ Week 109--112]
|-
| 2013-07-09 || - || [https://blog.mozilla.org/nnethercote/2013/07/10/memshrink-progress-week-105-108/ Week 105--108]
|-
| 2013-06-14 || - || [https://blog.mozilla.org/nnethercote/2013/06/15/memshrinks-2nd-birthday/ MemShrink's 2nd Birthday] (includes the third "Big Ticket Items" list)
|-
| 2013-06-11 || - || [https://blog.mozilla.org/nnethercote/2013/06/12/memshrink-progress-week-103-104/ Week 103--104]
|-
| 2013-05-28 || - || [http://jlebar.com/2013/5/8/MemShrink_process%2C_weeks_98-102.html Week 99--102] (guest post)
|-
| 2013-04-30 || - || [http://jlebar.com/2013/5/8/MemShrink_process%2C_weeks_97-98.html Week 97--98] (guest post)
|-
| 2013-04-16 || - || [https://blog.mozilla.org/nnethercote/2013/04/17/memshrink-progress-week-95-96/ Week 95--96]
|-
| 2013-04-02 || - || [https://blog.mozilla.org/nnethercote/2013/04/03/memshrink-progress-week-93-94/ Week 93--94]
|-
| 2013-03-19 || - || [https://blog.mozilla.org/nnethercote/2013/03/20/memshrink-progress-week-91-92/ Week 91--92]
|-
| 2013-03-05 || - || [https://blog.mozilla.org/nnethercote/2013/03/07/memshrink-progress-week-89-90/ Week 89--90]
|-
| 2013-02-19 || - || [https://blog.mozilla.org/nnethercote/2013/02/20/memshrink-progress-week-87-88/ Week 87--88]
|-
| 2013-02-05 || - || [https://blog.mozilla.org/nnethercote/2013/02/06/memshrink-progress-week-85-86/ Week 85--86]
|-
| 2013-01-22 || - || [https://blog.mozilla.org/nnethercote/2013/01/23/memshrink-progress-week-83-84/ Week 83--84]
|-
| 2013-01-08 || - || [https://blog.mozilla.org/nnethercote/2013/01/09/memshrink-progress-week-79-82/ Week 79--82]
|-
| 2012-12-11 || - || [https://blog.mozilla.org/nnethercote/2012/12/12/memshrink-progress-week-77-78/ Week 77--78]
|-
| 2012-11-27 || - || [https://blog.mozilla.org/nnethercote/2012/11/28/memshrink-progress-week-75-76/ Week 75--76]
|-
| 2012-11-13 || - || [https://blog.mozilla.org/nnethercote/2012/11/14/memshrink-progress-week-73-74/ Week 73--74]
|-
| 2012-10-30 || - || [https://blog.mozilla.org/nnethercote/2012/10/31/memshrink-progress-week-71-72/ Week 71--72]
|-
| 2012-10-16 || - || [https://blog.mozilla.org/nnethercote/2012/10/17/memshrink-progress-week-69-70/ Week 69--70]
|-
| 2012-10-02 || - || [https://blog.mozilla.org/nnethercote/2012/10/03/memshrink-progress-week-67-68/ Week 67--68]
|-
| 2012-09-18 || - || [https://blog.mozilla.org/nnethercote/2012/09/19/memshrink-progress-week-65-66/ Week 65--66]
|-
| 2012-09-04 || - || [https://blog.mozilla.org/nnethercote/2012/09/05/memshrink-progress-week-63-64/ Week 63--64]
|-
| 2012-08-21 || - || [https://blog.mozilla.org/nnethercote/2012/08/23/memshrink-progress-week-61-62/ Week 61--62]
|-
| 2012-08-07 || - || [https://blog.mozilla.org/nnethercote/2012/08/08/memshrink-progress-week-57-60/ Week 57--60]
|-
| 2012-07-10 || - || [https://blog.mozilla.org/nnethercote/2012/07/11/memshrink-progress-week-55-56/ Week 55--56] (includes the second "Big Ticket Items" list)
|-
| 2012-06-26 || - || [http://blog.mozilla.org/nnethercote/2012/06/27/memshrink-progress-week-53-54/ Week 53--54]
|-
| 2012-06-14 || - || [http://blog.mozilla.org/nnethercote/2012/06/15/memshrinks-1st-birthday/ MemShrink's 1st Birthday]
|-
| 2012-06-12 || - || [http://blog.mozilla.org/nnethercote/2012/06/13/memshrink-progress-week-51-52/ Week 51--52]
|-
| 2012-05-29 || - || [http://blog.mozilla.org/nnethercote/2012/05/30/memshrink-progress-week-49-50/ Week 49--50]
|-
| 2012-05-15 || - || [http://blog.mozilla.org/nnethercote/2012/05/16/memshrink-progress-week-47-48/ Week 47--48]
|-
| 2012-05-01 || - || [http://blog.mozilla.org/nnethercote/2012/05/02/memshrink-progress-week-45-46/ Week 45--46]
|-
| 2012-04-17 || - || [http://blog.mozilla.org/nnethercote/2012/04/18/memshrink-progress-week-43-44/ Week 43--44]
|-
| 2012-04-03 || - || [http://blog.mozilla.org/nnethercote/2012/04/04/memshrink-progress-week-42/ Week 42]
|-
| 2012-03-27 || - || [http://blog.mozilla.com/nnethercote/2012/03/28/memshrink-progress-week-41/ Week 41]
|-
| 2012-03-20 || - || [http://blog.mozilla.com/nnethercote/2012/03/21/memshrink-progress-week-40/ Week 40]
|-
| 2012-03-13 || - || [http://blog.mozilla.com/nnethercote/2012/03/14/memshrink-progress-week-39/ Week 39]
|-
| 2012-03-06 || - || [http://blog.mozilla.com/nnethercote/2012/03/07/memshrink-progress-week-38/ Week 38]
|-
| 2012-02-28 || - || [http://blog.mozilla.com/nnethercote/2012/02/29/memshrink-progress-week-37/ Week 37]
|-
| 2012-02-21 || - || [http://blog.mozilla.com/nnethercote/2012/02/22/memshrink-progress-week-36/ Week 36]
|-
| 2012-02-14 || - || [http://blog.mozilla.com/nnethercote/2012/02/15/memshrink-progress-week-35/ Week 35]
|-
| 2012-02-07 || - || [http://blog.mozilla.com/nnethercote/2012/02/08/memshrink-progress-week-34/ Week 34]
|-
| 2012-01-31 || - || [http://blog.mozilla.com/nnethercote/2012/02/01/memshrink-progress-week-33/ Week 33]
|-
| 2012-01-24 || - || [http://blog.mozilla.com/nnethercote/2012/01/25/memshrink-progress-week-32/ Week 32] (includes the first "Big Ticket Items" list)
|-
| 2012-01-17 || - || [http://blog.mozilla.com/nnethercote/2012/01/18/memshrink-progress-week-31/ Week 31]
|-
| 2012-01-10 || - || [http://blog.mozilla.com/nnethercote/2012/01/11/memshrink-progress-week-30/ Week 30]
|-
| 2012-01-03 || - || [http://blog.mozilla.com/nnethercote/2012/01/04/memshrink-progress-weeks-28-29/ Weeks 28--29]
|-
| 2011-12-20 || - || [http://blog.mozilla.com/nnethercote/2011/12/21/memshrink-progress-week-27/ Week 27]
|-
| 2011-12-13 || [[Performance/MemShrink/Meetings/2011-12-13|minutes]] || [http://blog.mozilla.com/nnethercote/2011/12/14/memshrink-progress-week-26/ Week 26]
|-
| 2011-12-06 || [[Performance/MemShrink/Meetings/2011-12-06|minutes]] ||  [http://blog.mozilla.com/nnethercote/2011/12/07/memshrink-progress-week-25/ Week 25]
|-
| 2011-11-29 || [[Performance/MemShrink/Meetings/2011-11-29|minutes]] ||  [http://blog.mozilla.com/nnethercote/2011/11/30/memshrink-progress-report-week-24/ Week 24]
|-
| 2011-11-22 || - ||  [http://blog.mozilla.com/nnethercote/2011/11/23/memshrink-progress-report-week-23/ Week 23]
|-
| 2011-11-15 || - ||  [http://blog.mozilla.com/nnethercote/2011/11/16/memshrink-progress-report-week-22/ Week 22]
|-
| 2011-11-08 || [[Performance/MemShrink/Meetings/2011-11-08|minutes]] || [http://blog.mozilla.com/nnethercote/2011/11/09/memshrink-progress-week-21/ Week 21]
|-
| 2011-11-01 || [[Performance/MemShrink/Meetings/2011-11-01|minutes]] || [http://blog.mozilla.com/nnethercote/2011/11/02/memshrink-progress-week-20/ Week 20]
|-
| 2011-10-25 || [[Performance/MemShrink/Meetings/2011-10-25|minutes]] || [http://blog.mozilla.com/nnethercote/2011/10/26/memshrink-progress-week-19/ Week 19]
|-
| 2011-10-18 || [[Performance/MemShrink/Meetings/2011-10-18|minutes]] || [http://blog.mozilla.com/nnethercote/2011/10/19/memshrink-progress-weeks-13-18/ Weeks 13--18]
|-
| 2011-10-11 || [[Performance/MemShrink/Meetings/2011-10-11|minutes]] || -
|-
| 2011-10-04 || [[Performance/MemShrink/Meetings/2011-10-04|minutes]] || -
|-
| 2011-09-27 || - || -
|-
| 2011-09-20 || [[Performance/MemShrink/Meetings/2011-09-20|minutes]] || -
|-
| 2011-09-13 || - || -
|-
| 2011-09-06 || [[Performance/MemShrink/Meetings/2011-09-06|minutes]] || [http://blog.mozilla.com/nnethercote/2011/09/07/memshrink-progress-week-12/ Week 12]
|-
| 2011-08-30 || [[Performance/MemShrink/Meetings/2011-08-30|minutes]] || [http://blog.mozilla.com/nnethercote/2011/08/31/memshrink-progress-week-11/ Week 11]
|-
| 2011-08-23 || [[Performance/MemShrink/Meetings/2011-08-23|minutes]] || [http://blog.mozilla.com/nnethercote/2011/08/24/memshrink-progress-week-10/ Week 10]
|-
| 2011-08-16 || [[Performance/MemShrink/Meetings/2011-08-16|minutes]] || [http://blog.mozilla.com/nnethercote/2011/08/17/memshrink-progress-week-9/ Week 9]
|-
| 2011-08-09 || [[Performance/MemShrink/Meetings/2011-08-09|minutes]] || [http://blog.mozilla.com/nnethercote/2011/08/10/memshrink-progress-week-8/ Week 8]
|-
| 2011-08-02 || - || [http://blog.mozilla.com/nnethercote/2011/08/03/memshrink-progress-week-7/ Week 7]
|-
| 2011-07-26 || [[Performance/MemShrink/Meetings/2011-07-26|minutes]] || [http://blog.mozilla.com/nnethercote/2011/07/27/memshrink-progress-week-6/ Week 6]
|-
| 2011-07-19 || [[Performance/MemShrink/Meetings/2011-07-19|minutes]] || [http://blog.mozilla.com/nnethercote/2011/07/20/memshrink-progress-week-5/ Week 5]
|-
| 2011-07-12 || [[Performance/MemShrink/Meetings/2011-07-12|minutes]] || [http://blog.mozilla.com/nnethercote/2011/07/13/memshrink-progress-week-4/ Week 4]
|-
| 2011-07-05 || [[Performance/MemShrink/Meetings/2011-07-05|minutes]] || [http://blog.mozilla.com/nnethercote/2011/07/06/memshrink-progress-week-3/ Week 3]
|-
| 2011-06-28 || - || [http://blog.mozilla.com/nnethercote/2011/06/29/memshrink-progress-week-2/ Week 2]
|-
| 2011-06-21 || - || [http://blog.mozilla.com/nnethercote/2011/06/22/memshrink-progress-week-1/ Week 1]
|-
| 2011-06-14 || - || -
|}


Why was AWFY so successful?  Some key characteristics.
* The benchmarks were easy to choose:  everybody already used SS and V8, and then Mozilla released Kraken.  They run quickly, too, which is nice for devs, and allows expensive tools (like Cachegrind) to be used.
* The metrics were easy to choose.  SS time, V8 time, Kraken time.  They can be measured easily, precisely, and fairly repeatably.  You can break the total time down into per-benchmark times, which really helps with understanding improvements and regressions.  They're easy for devs to run on their own machine.
* Understanding where the time went was fairly easy, because there are good time-oriented profilers:  Shark, Cachegrind, etc.
* There was a well-defined goal:  match or beat the other browsers.


In comparison, for AWSY there is no good and/or standard benchmark suite; metrics are less clear and/or harder to measure; the memory profiling tools aren't as good;  but most importantly, there's no well-defined goal w.r.t. any benchmarks.
Presentations.


A more trackable goal is to get the number of MemShrink P1 bugs down to zero. That will mean that all the bad leaks will have been fixed, and also the important auxiliary work (e.g. infrastructure to detect regressions) will be in place.  Therefore, areweslimyet.com just points to the list of open MemShrink P1 bugs.
*2012-01-16 [[media:LCA2012.pdf|"Notes on Reducing Firefox's Memory Consumption"]], presented at linux.conf.au Browser MiniConf

Latest revision as of 20:17, 10 July 2019

MemShrink is a project that aims to reduce the memory consumption of Firefox (on desktop and mobile). There are three potential benefits.

  1. Speed. Firefox will be faster due to less cache pressure, less paging, and fewer/smaller GC and CC pauses. Changes that reduce memory consumption but make Firefox slower are not desirable.
  2. Stability. Firefox will suffer fewer aborts/crashes due to virtual or physical memory exhaustion. The former is mostly a problem on 32-bit Windows builds with a 2GB or 4GB virtual memory limit, the latter is mostly a problem on mobile devices that lack swap space.
  3. Reputation. Fewer people will complain that Firefox is a memory hog and that Mozilla ignores memory usage.

This blog post describes these benefits in more detail.

There are two main ways to reduce memory consumption.

  1. "Slim down" memory usage, e.g. make data structures more space-efficient.
  2. Avoid "leaks". This loose use of the term (which is used throughout this document) includes:
    • True leaks, where memory is lost forever.
    • Lifetime issues, where memory is not reclaimed until you close the page/tab/window/process.
    • Collection heuristic issues (e.g. GC is too infrequent).
    • Bad cache algorithms and poorly tuned caches.
    • Fragmentation.

Leaks are generally more important, because they are more likely to lead to horrible performance.

Meetings

Meetings happen every second Wednesday at 11am Pacific time (countdown in your timezone)


  • Dial-in: Audio-only conference# 95346
    • People with Mozilla phones or softphones please dial x4000 Conf# 95346
    • US/Toll-free: +1 800 707 2533, (pin 4000) Conf# 95346
    • US/California/Mountain View: +1 650 903 0800, x4000 Conf# 95346
    • US/California/San Francisco: +1 415 762 5700, x4000 Conf# 95346
    • US/Oregon/Portland: +1 971 544 8000, x4000 Conf# 95346
    • CA/British Columbia/Vancouver: +1 778 785 1540, x4000 Conf# 95346
    • CA/Ontario/Toronto: +1 416 848 3114, x4000 Conf# 95346
    • UK/London: +44 (0)207 855 3000, x4000 Conf# 95346
    • FR/Paris: +33 1 84 88 37 37, x4000 Conf# 95346
    • Gmail Chat (requires Flash and the Google Talk plugin): paste +1 650 903 0800 into the Gmail Chat box that doesn't look like it accepts phone numbers
    • SkypeOut is free if you use the 800 number
  • zoom: 310 455 675
  • IRC: #memshrink

Bug Tracking

Bugs are prioritized during MemShrink meetings.

If you want a bug that is related to memory usage to be triaged in a MemShrink meeting, add "[MemShrink]" to the bug whiteboard. After prioritization, this whiteboard marking will be changed to one of "[MemShrink:P1]", "[MemShrink:P2]", "[MemShrink:P3]" or "[MemShrink:meta]". The meaning of these markings, and links to all such bugs, is as follows.

  • Unprioritized MemShrink bugs. These are prioritized regularly in meetings.
  • MemShrink:P1 bugs. Once prioritized, these will be revisited and discussed occasionally in meetings.
  • MemShrink:P2 bugs. Once prioritized, these will be revisited and discussed rarely in meetings. P2 is the default priority.
  • MemShrink:P3 bugs. Once prioritized, these are unlikely to be revisited and discussed again in meetings.
  • MemShrink:meta bugs. Once prioritized, these will be revisited and discussed occasionally in meetings.

Some other interesting bug lists that overlap with the ones above.

Tools

All the tools that we use to measure and improve memory usage are described on MDN, under "Memory profiling and leak detection tools".

Meeting Minutes, Progress Reports and Presentations

We no longer take meeting minutes nor make progress reports, but the meetings still happen. Brief MemShrink updates are given in the minutes of the weekly Platform meeting (wiki, blog). And nnethercote still blogs about major MemShrink occurrences.

Here is the historical record from when we did take minutes and make progress reports.

Date Minutes Progress report
2014-06-16 - MemShrink's 3rd Birthday (includes the fourth "Big Ticket Items" list)
2013-11-26 - The final progress report
2013-10-29 - Week 121--124
2013-09-30 - Week 117--120
2013-09-03 - Week 113--116
2013-08-06 - Week 109--112
2013-07-09 - Week 105--108
2013-06-14 - MemShrink's 2nd Birthday (includes the third "Big Ticket Items" list)
2013-06-11 - Week 103--104
2013-05-28 - Week 99--102 (guest post)
2013-04-30 - Week 97--98 (guest post)
2013-04-16 - Week 95--96
2013-04-02 - Week 93--94
2013-03-19 - Week 91--92
2013-03-05 - Week 89--90
2013-02-19 - Week 87--88
2013-02-05 - Week 85--86
2013-01-22 - Week 83--84
2013-01-08 - Week 79--82
2012-12-11 - Week 77--78
2012-11-27 - Week 75--76
2012-11-13 - Week 73--74
2012-10-30 - Week 71--72
2012-10-16 - Week 69--70
2012-10-02 - Week 67--68
2012-09-18 - Week 65--66
2012-09-04 - Week 63--64
2012-08-21 - Week 61--62
2012-08-07 - Week 57--60
2012-07-10 - Week 55--56 (includes the second "Big Ticket Items" list)
2012-06-26 - Week 53--54
2012-06-14 - MemShrink's 1st Birthday
2012-06-12 - Week 51--52
2012-05-29 - Week 49--50
2012-05-15 - Week 47--48
2012-05-01 - Week 45--46
2012-04-17 - Week 43--44
2012-04-03 - Week 42
2012-03-27 - Week 41
2012-03-20 - Week 40
2012-03-13 - Week 39
2012-03-06 - Week 38
2012-02-28 - Week 37
2012-02-21 - Week 36
2012-02-14 - Week 35
2012-02-07 - Week 34
2012-01-31 - Week 33
2012-01-24 - Week 32 (includes the first "Big Ticket Items" list)
2012-01-17 - Week 31
2012-01-10 - Week 30
2012-01-03 - Weeks 28--29
2011-12-20 - Week 27
2011-12-13 minutes Week 26
2011-12-06 minutes Week 25
2011-11-29 minutes Week 24
2011-11-22 - Week 23
2011-11-15 - Week 22
2011-11-08 minutes Week 21
2011-11-01 minutes Week 20
2011-10-25 minutes Week 19
2011-10-18 minutes Weeks 13--18
2011-10-11 minutes -
2011-10-04 minutes -
2011-09-27 - -
2011-09-20 minutes -
2011-09-13 - -
2011-09-06 minutes Week 12
2011-08-30 minutes Week 11
2011-08-23 minutes Week 10
2011-08-16 minutes Week 9
2011-08-09 minutes Week 8
2011-08-02 - Week 7
2011-07-26 minutes Week 6
2011-07-19 minutes Week 5
2011-07-12 minutes Week 4
2011-07-05 minutes Week 3
2011-06-28 - Week 2
2011-06-21 - Week 1
2011-06-14 - -


Presentations.

Retrieved from "https://dev.wikimo.nonprod.webservices.mozgcp.net/index.php?title=Performance/MemShrink&oldid=1214902"