Performance/MemShrink: Difference between revisions

Latest revision as of 20:17, 10 July 2019

MemShrink is a project that aims to reduce the memory consumption of Firefox (on desktop and mobile). There are three potential benefits.

Speed. Firefox will be faster due to less cache pressure, less paging, and fewer/smaller GC and CC pauses. Changes that reduce memory consumption but make Firefox slower are not desirable.
Stability. Firefox will suffer fewer aborts/crashes due to virtual or physical memory exhaustion. The former is mostly a problem on 32-bit Windows builds with a 2GB or 4GB virtual memory limit, the latter is mostly a problem on mobile devices that lack swap space.
Reputation. Fewer people will complain that Firefox is a memory hog and that Mozilla ignores memory usage.

This blog post describes these benefits in more detail.

There are two main ways to reduce memory consumption.

"Slim down" memory usage, e.g. make data structures more space-efficient.
Avoid "leaks". This loose use of the term (which is used throughout this document) includes:
- True leaks, where memory is lost forever.
- Lifetime issues, where memory is not reclaimed until you close the page/tab/window/process.
- Collection heuristic issues (e.g. GC is too infrequent).
- Bad cache algorithms and poorly tuned caches.
- Fragmentation.

Leaks are generally more important, because they are more likely to lead to horrible performance.

Meetings

Meetings happen every second Wednesday at 11am Pacific time (countdown in your timezone)

Dial-in: Audio-only conference# 95346
- People with Mozilla phones or softphones please dial x4000 Conf# 95346
- US/Toll-free: +1 800 707 2533, (pin 4000) Conf# 95346
- US/California/Mountain View: +1 650 903 0800, x4000 Conf# 95346
- US/California/San Francisco: +1 415 762 5700, x4000 Conf# 95346
- US/Oregon/Portland: +1 971 544 8000, x4000 Conf# 95346
- CA/British Columbia/Vancouver: +1 778 785 1540, x4000 Conf# 95346
- CA/Ontario/Toronto: +1 416 848 3114, x4000 Conf# 95346
- UK/London: +44 (0)207 855 3000, x4000 Conf# 95346
- FR/Paris: +33 1 84 88 37 37, x4000 Conf# 95346
- Gmail Chat (requires Flash and the Google Talk plugin): paste +1 650 903 0800 into the Gmail Chat box that doesn't look like it accepts phone numbers
- SkypeOut is free if you use the 800 number
zoom: 310 455 675
IRC: #memshrink

Bug Tracking

Bugs are prioritized during MemShrink meetings.

If you want a bug that is related to memory usage to be triaged in a MemShrink meeting, add "[MemShrink]" to the bug whiteboard. After prioritization, this whiteboard marking will be changed to one of "[MemShrink:P1]", "[MemShrink:P2]", "[MemShrink:P3]" or "[MemShrink:meta]". The meaning of these markings, and links to all such bugs, is as follows.

Unprioritized MemShrink bugs. These are prioritized regularly in meetings.
MemShrink:P1 bugs. Once prioritized, these will be revisited and discussed occasionally in meetings.
MemShrink:P2 bugs. Once prioritized, these will be revisited and discussed rarely in meetings. P2 is the default priority.
MemShrink:P3 bugs. Once prioritized, these are unlikely to be revisited and discussed again in meetings.
MemShrink:meta bugs. Once prioritized, these will be revisited and discussed occasionally in meetings.

Some other interesting bug lists that overlap with the ones above.

All MemShrink bugs.
Unconfirmed MemShrink bugs. These are problems reported by users and generally require some kind of additional work to confirm.
MemShrink bugs with a "mentor" annotation. These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
Unassigned MemShrink bugs. These need someone to work on them.

Tools

All the tools that we use to measure and improve memory usage are described on MDN, under "Memory profiling and leak detection tools".

Meeting Minutes, Progress Reports and Presentations

We no longer take meeting minutes nor make progress reports, but the meetings still happen. Brief MemShrink updates are given in the minutes of the weekly Platform meeting (wiki, blog). And nnethercote still blogs about major MemShrink occurrences.

Here is the historical record from when we did take minutes and make progress reports.

Date	Minutes	Progress report
2014-06-16	-	MemShrink's 3rd Birthday (includes the fourth "Big Ticket Items" list)
2013-11-26	-	The final progress report
2013-10-29	-	Week 121--124
2013-09-30	-	Week 117--120
2013-09-03	-	Week 113--116
2013-08-06	-	Week 109--112
2013-07-09	-	Week 105--108
2013-06-14	-	MemShrink's 2nd Birthday (includes the third "Big Ticket Items" list)
2013-06-11	-	Week 103--104
2013-05-28	-	Week 99--102 (guest post)
2013-04-30	-	Week 97--98 (guest post)
2013-04-16	-	Week 95--96
2013-04-02	-	Week 93--94
2013-03-19	-	Week 91--92
2013-03-05	-	Week 89--90
2013-02-19	-	Week 87--88
2013-02-05	-	Week 85--86
2013-01-22	-	Week 83--84
2013-01-08	-	Week 79--82
2012-12-11	-	Week 77--78
2012-11-27	-	Week 75--76
2012-11-13	-	Week 73--74
2012-10-30	-	Week 71--72
2012-10-16	-	Week 69--70
2012-10-02	-	Week 67--68
2012-09-18	-	Week 65--66
2012-09-04	-	Week 63--64
2012-08-21	-	Week 61--62
2012-08-07	-	Week 57--60
2012-07-10	-	Week 55--56 (includes the second "Big Ticket Items" list)
2012-06-26	-	Week 53--54
2012-06-14	-	MemShrink's 1st Birthday
2012-06-12	-	Week 51--52
2012-05-29	-	Week 49--50
2012-05-15	-	Week 47--48
2012-05-01	-	Week 45--46
2012-04-17	-	Week 43--44
2012-04-03	-	Week 42
2012-03-27	-	Week 41
2012-03-20	-	Week 40
2012-03-13	-	Week 39
2012-03-06	-	Week 38
2012-02-28	-	Week 37
2012-02-21	-	Week 36
2012-02-14	-	Week 35
2012-02-07	-	Week 34
2012-01-31	-	Week 33
2012-01-24	-	Week 32 (includes the first "Big Ticket Items" list)
2012-01-17	-	Week 31
2012-01-10	-	Week 30
2012-01-03	-	Weeks 28--29
2011-12-20	-	Week 27
2011-12-13	minutes	Week 26
2011-12-06	minutes	Week 25
2011-11-29	minutes	Week 24
2011-11-22	-	Week 23
2011-11-15	-	Week 22
2011-11-08	minutes	Week 21
2011-11-01	minutes	Week 20
2011-10-25	minutes	Week 19
2011-10-18	minutes	Weeks 13--18
2011-10-11	minutes	-
2011-10-04	minutes	-
2011-09-27	-	-
2011-09-20	minutes	-
2011-09-13	-	-
2011-09-06	minutes	Week 12
2011-08-30	minutes	Week 11
2011-08-23	minutes	Week 10
2011-08-16	minutes	Week 9
2011-08-09	minutes	Week 8
2011-08-02	-	Week 7
2011-07-26	minutes	Week 6
2011-07-19	minutes	Week 5
2011-07-12	minutes	Week 4
2011-07-05	minutes	Week 3
2011-06-28	-	Week 2
2011-06-21	-	Week 1
2011-06-14	-	-

Presentations.

2012-01-16 "Notes on Reducing Firefox's Memory Consumption", presented at linux.conf.au Browser MiniConf

@@ Line 1: / Line 1: @@
-MemShrink is a project that aims to reduce Firefox's memory consumption.  There are three potential benefits.
+MemShrink is a project that aims to reduce the memory consumption of Firefox (on desktop and mobile).  There are three potential benefits.
 # Speed.  Firefox will be faster due to less cache pressure, less paging, and fewer/smaller GC and CC pauses.  Changes that reduce memory consumption but make Firefox slower are not desirable.
@@ Line 20: / Line 20: @@
 == Meetings ==
-Meetings happen every second Tuesday at 4pm Pacific time.
+Meetings happen every second Wednesday at 11am Pacific time ([http://arewemeetingyet.com/Los%20Angeles/2014-10-14/16:00/b/MemShrink%20Meeting countdown in your timezone])
 {{conf|95346}}
-* Vidyo: PB&J
+* zoom: 310 455 675
 * IRC: [irc://irc.mozilla.org/memshrink #memshrink]
-* Etherpad: [https://etherpad.mozilla.org/memshrink memshrink] (copied to wiki after the meeting)
+== Bug Tracking ==
+Bugs are prioritized during MemShrink meetings.
+If you want a bug that is related to memory usage to be triaged in a MemShrink meeting, add "[MemShrink]" to the bug whiteboard. After prioritization, this whiteboard marking will be changed to one of "[MemShrink:P1]", "[MemShrink:P2]", "[MemShrink:P3]" or "[MemShrink:meta]". The meaning of these markings, and links to all such bugs, is as follows.
+* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=MemShrink%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized MemShrink bugs].  These are prioritized regularly in meetings.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P1 bugs]. Once prioritized, these will be revisited and discussed occasionally in meetings.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P2 bugs]. Once prioritized, these will be revisited and discussed rarely in meetings. P2 is the default priority.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P3 bugs]. Once prioritized, these are unlikely to be revisited and discussed again in meetings.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3Ameta&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:meta bugs]. Once prioritized, these will be revisited and discussed occasionally in meetings.
+Some other interesting bug lists that overlap with the ones above.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id All MemShrink bugs].
+* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252875&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&resolution=---&resolution=DUPLICATE Unconfirmed MemShrink bugs]. These are problems reported by users and generally require some kind of additional work to confirm.
+* [https://bugzilla.mozilla.org/buglist.cgi?f1=bug_mentor&list_id=12543462&o1=isnotempty&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED MemShrink bugs with a "mentor" annotation]. These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
+* [https://bugzilla.mozilla.org/buglist.cgi?list_id=1252950&resolution=---&resolution=DUPLICATE&emailtype1=regexp&status_whiteboard_type=allwordssubstr&emailassigned_to1=1&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=nobody%40%7Cgeneral%40js.bugs Unassigned MemShrink bugs]. These need someone to work on them.
+== Tools ==
+All the tools that we use to measure and improve memory usage are described [https://developer.mozilla.org/en-US/docs/Mozilla/Performance on MDN], under "Memory profiling and leak detection tools".
 == Meeting Minutes, Progress Reports and Presentations ==
-Meeting dates, meeting minutes, and progress reports.
+We no longer take meeting minutes nor make progress reports, but the meetings still happen. Brief MemShrink updates are given in the minutes of the weekly Platform meeting ([https://wiki.mozilla.org/Platform#Meeting_Notes wiki], [https://blog.mozilla.org/meeting-notes/archives/tag/mozillaplatform blog]). And nnethercote still [https://blog.mozilla.org/nnethercote/category/memshrink/ blogs] about major MemShrink occurrences.
+Here is the historical record from when we did take minutes and make progress reports.
 {| border="1" cellpadding="2"
 ! Date !! Minutes !! Progress report
 |-
-| 2012-10-03 || - || [https://blog.mozilla.org/nnethercote/2012/10/03/memshrink-progress-week-67-68/ Week 67--68]
+| 2014-06-16 || - || [https://blog.mozilla.org/nnethercote/2014/06/16/memshrinks-3rd-birthday/ MemShrink's 3rd Birthday] (includes the fourth "Big Ticket Items" list)
+|-
+| 2013-11-26 || - || [https://blog.mozilla.org/nnethercote/2013/11/27/memshrink-progress-final/ The final progress report]
+|-
+| 2013-10-29 || - || [https://blog.mozilla.org/nnethercote/2013/10/30/memshrink-progress-week-121-124/ Week 121--124]
+|-
+| 2013-09-30 || - || [https://blog.mozilla.org/nnethercote/2013/10/01/memshrink-progress-week-117-120/ Week 117--120]
+|-
+| 2013-09-03 || - || [https://blog.mozilla.org/nnethercote/2013/09/04/memshrink-progress-week-113-116/ Week 113--116]
+|-
+| 2013-08-06 || - || [https://blog.mozilla.org/nnethercote/2013/08/07/memshrink-progress-week-109-112/ Week 109--112]
+|-
+| 2013-07-09 || - || [https://blog.mozilla.org/nnethercote/2013/07/10/memshrink-progress-week-105-108/ Week 105--108]
+|-
+| 2013-06-14 || - || [https://blog.mozilla.org/nnethercote/2013/06/15/memshrinks-2nd-birthday/ MemShrink's 2nd Birthday] (includes the third "Big Ticket Items" list)
+|-
+| 2013-06-11 || - || [https://blog.mozilla.org/nnethercote/2013/06/12/memshrink-progress-week-103-104/ Week 103--104]
+|-
+| 2013-05-28 || - || [http://jlebar.com/2013/5/8/MemShrink_process%2C_weeks_98-102.html Week 99--102] (guest post)
+|-
+| 2013-04-30 || - || [http://jlebar.com/2013/5/8/MemShrink_process%2C_weeks_97-98.html Week 97--98] (guest post)
+|-
+| 2013-04-16 || - || [https://blog.mozilla.org/nnethercote/2013/04/17/memshrink-progress-week-95-96/ Week 95--96]
 |-
-| 2012-09-19 || - || [https://blog.mozilla.org/nnethercote/2012/09/19/memshrink-progress-week-65-66/ Week 65--66]
+| 2013-04-02 || - || [https://blog.mozilla.org/nnethercote/2013/04/03/memshrink-progress-week-93-94/ Week 93--94]
 |-
-| 2012-09-05 || - || [https://blog.mozilla.org/nnethercote/2012/09/05/memshrink-progress-week-63-64/ Week 63--64]
+| 2013-03-19 || - || [https://blog.mozilla.org/nnethercote/2013/03/20/memshrink-progress-week-91-92/ Week 91--92]
+|-
+| 2013-03-05 || - || [https://blog.mozilla.org/nnethercote/2013/03/07/memshrink-progress-week-89-90/ Week 89--90]
+|-
+| 2013-02-19 || - || [https://blog.mozilla.org/nnethercote/2013/02/20/memshrink-progress-week-87-88/ Week 87--88]
+|-
+| 2013-02-05 || - || [https://blog.mozilla.org/nnethercote/2013/02/06/memshrink-progress-week-85-86/ Week 85--86]
+|-
+| 2013-01-22 || - || [https://blog.mozilla.org/nnethercote/2013/01/23/memshrink-progress-week-83-84/ Week 83--84]
+|-
+| 2013-01-08 || - || [https://blog.mozilla.org/nnethercote/2013/01/09/memshrink-progress-week-79-82/ Week 79--82]
+|-
+| 2012-12-11 || - || [https://blog.mozilla.org/nnethercote/2012/12/12/memshrink-progress-week-77-78/ Week 77--78]
+|-
+| 2012-11-27 || - || [https://blog.mozilla.org/nnethercote/2012/11/28/memshrink-progress-week-75-76/ Week 75--76]
+|-
+| 2012-11-13 || - || [https://blog.mozilla.org/nnethercote/2012/11/14/memshrink-progress-week-73-74/ Week 73--74]
+|-
+| 2012-10-30 || - || [https://blog.mozilla.org/nnethercote/2012/10/31/memshrink-progress-week-71-72/ Week 71--72]
+|-
+| 2012-10-16 || - || [https://blog.mozilla.org/nnethercote/2012/10/17/memshrink-progress-week-69-70/ Week 69--70]
+|-
+| 2012-10-02 || - || [https://blog.mozilla.org/nnethercote/2012/10/03/memshrink-progress-week-67-68/ Week 67--68]
+|-
+| 2012-09-18 || - || [https://blog.mozilla.org/nnethercote/2012/09/19/memshrink-progress-week-65-66/ Week 65--66]
+|-
+| 2012-09-04 || - || [https://blog.mozilla.org/nnethercote/2012/09/05/memshrink-progress-week-63-64/ Week 63--64]
 |-
 | 2012-08-21 || - || [https://blog.mozilla.org/nnethercote/2012/08/23/memshrink-progress-week-61-62/ Week 61--62]
 |-
 | 2012-08-07 || - || [https://blog.mozilla.org/nnethercote/2012/08/08/memshrink-progress-week-57-60/ Week 57--60]
-|-
-| 2012-07-24 || - || -
 |-
 | 2012-07-10 || - || [https://blog.mozilla.org/nnethercote/2012/07/11/memshrink-progress-week-55-56/ Week 55--56] (includes the second "Big Ticket Items" list)
@@ Line 151: / Line 220: @@
 *2012-01-16 [[media:LCA2012.pdf|"Notes on Reducing Firefox's Memory Consumption"]], presented at linux.conf.au Browser MiniConf
-== Bug Tracking ==
-Bugs tracked by the MemShrink project are prioritized by adding one of "MemShrink:P1", "MemShrink:P2" or "MemShrink:P3" to the whiteboard.
-* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=regexp&query_format=advanced&list_id=577353&status_whiteboard=MemShrink%5b%5e%3A%5d&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE Unprioritized MemShrink bugs].  These are triaged regularly in meetings.
-* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP1&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P1 bugs].  These are discussed regularly in meetings.
-* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP2&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P2 bugs].  These are discussed occasionally in meetings.  P2 is the default priority.
-* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink%3AP3&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id MemShrink:P3 bugs].  These are discussed rarely in meetings.
-Some other interesting bug lists that overlap with the ones above.
-* [https://bugzilla.mozilla.org/buglist.cgi?list_id=463753&resolution=---&resolution=DUPLICATE&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&order=bugs.bug_id All MemShrink bugs].
-* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252875&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&resolution=---&resolution=DUPLICATE Unconfirmed MemShrink bugs].  These are problems reported by users and generally require some kind of additional work to confirm.
-* [https://bugzilla.mozilla.org/buglist.cgi?status_whiteboard_type=allwordssubstr&query_format=advanced&list_id=1252905&status_whiteboard=MemShrink%20mentor&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&resolution=---&resolution=DUPLICATE MemShrink bugs with a "mentor" annotation].  These are bugs that someone has identified as reasonable easy, and that person is willing to help a newcomer fix the bug.
-* [https://bugzilla.mozilla.org/buglist.cgi?list_id=1252950&resolution=---&resolution=DUPLICATE&emailtype1=regexp&status_whiteboard_type=allwordssubstr&emailassigned_to1=1&query_format=advanced&status_whiteboard=MemShrink&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&email1=nobody%40%7Cgeneral%40js.bugs Unassigned MemShrink bugs].  These need someone to work on them.
-== areweslimyet.com ==
-http://arewefastyet.com is a very successful site used to track the performance of Firefox's JavaScript engine.  Why was arewefastyet.com so successful?  Some key characteristics.
-* The benchmarks were easy to choose:  everybody already used SS and V8, and then Mozilla released Kraken.  They run quickly, too, which is nice for devs, and allows expensive tools (like Cachegrind) to be used.
-* The metrics were easy to choose.  SS time, V8 time, Kraken time.  They can be measured easily, precisely, and fairly repeatably.  You can break the total time down into per-benchmark times, which really helps with understanding improvements and regressions.  They're easy for devs to run on their own machine.
-* There was a well-defined goal:  match or beat the other browsers.
-In comparison, for memory consumption there is no good and/or standard benchmark suite; metrics are less clear and harder to measure (especially across different browsers);  and most importantly, there's no well-defined goal w.r.t. any benchmarks.
-Nonetheless, we've created a memory benchmark and are tracking Firefox's performance on it over time, to provide a sense of MemShrink progress and detect regressions.  It is available at http://areweslimyet.com/.

Performance/MemShrink: Difference between revisions

Latest revision as of 20:17, 10 July 2019

Contents

Meetings

Bug Tracking

Tools

Meeting Minutes, Progress Reports and Presentations

Navigation menu

Performance/MemShrink: Difference between revisions

Latest revision as of 20:17, 10 July 2019

Meetings

Bug Tracking

Tools

Meeting Minutes, Progress Reports and Presentations

Navigation menu

Search