Here's the thing: they're not! The reason those users where crashing was because something, somewhere (possibly in the graphics stack) was reserving ton of space without using it. We had crashes on file with 20+ GiB of free physical memory which was the reason why I started looking into why a machine with so much free memory could suffer a crash.
I hope I described how this whole things work because Windows memory management is not well known and some things about it are counter-intuitive; especially if you're coming from Linux.
I can confirm this is the case. Firefox reserves tens of gigabytes and then only actually uses a few. But since those tens of unused gigabytes are still committed, nothing else on the system can use it, and Firefox (+ tons of other apps) will OOM when the total committed memory (of all applications) reaches the total amount of RAM installed.
I have 17 crashes on file (in about:crashes) that are OOMs caused by Firefox committing too much memory and not using it. My solution is to restart the browser when committed memory reaches 38GB or so, but that clears all private windows which annoys me. I shouldn't have to restart the browser so often.
Could you open a bug on our tracker, and point to those crash reports? I'm very interested in analyzing real-world scenarios where this happens to figure out where those committed areas are coming from. Our best guess is that they come from the graphics stack because we account all the memory we commit in Firefox and there's nothing missing, so wherever this is coming from it's outside of our direct control.
From the looks of it you have a minuscule page file (16MiB), that's why you're running out of commit space. I suggest setting it to auto-resize (the Windows default) which should provide a much better experience even with other applications, unless you need to keep it small because of storage concerns. Generally speaking Windows behaves very poorly with small swap files because of its inability to overcommit memory.
Firefox should not be relying on overcommit in the first place. The issue isn't that I "only" have 40GB of commit space - it's that Firefox keeps taking more and more to the point of crashing (itself and everything else I have running).
And yes, my page file had grown to 64GiB before, when I stupidly left it on automatic. There's a reason it's at 16MiB now - it's because completely turning it off disables memory compression. Not like that helps much due to the lack of overcommit.
You're assuming that Firefox is eating all your commit space, but it's not, far from it, I can see it from the crash reports. You've got other applications running and they are also fighting for it, leaving large chunks unused. Without some swap space available Windows is leaving 10+ GiB of physical memory free which it cannot use for anything else.
This is how Windows behaves by design, there's nothing we can do about it. Windows without a swap file sucks.
Funny because I just switched to Chromium and left it on for around 36 hours and my commit space is not being eaten up. Coincidence? ;)
Before, when I killed Firefox (before it crashes naturally, of course), my committed memory would drop from 38GiB to around 18GiB, so uhh, unless some other app is intentionally playing with me and syncing up its memory leaks with Firefox's existence...
(Plus, Process Hacker confirmed that all of Firefox's "Private bytes" added up to about how much committed memory got freed when I killed it. Explain how that is caused by memory fragmentation!)
This is my third conversation with a Mozilla engineer about this! Always looking forward to be proven wrong, but it looks like you just can't seem to find the problem yet. I hope one day you can. :)
It seems unlikely given this exchange, as you seem to have decided to be as unhelpful as possible.
> Before, when I killed Firefox (before it crashes naturally, of course), my committed memory would drop from 38GiB to around 18GiB, so uhh, unless some other app is intentionally playing with me and syncing up its memory leaks with Firefox's existence...
From the article:
> However, we have no control over Windows system libraries and in particular graphics drivers. One thing we noticed is that graphics drivers commit memory to make room for textures in system memory.
Guess what's going to create textures which might trigger such an issue? The browser's hardware acceleration. Guess what happens to browser's textures when the browser is killed? They're freed.
Might be a bug in Firefox, might be a bug in the crash reporter, might be a bug in the driver, might be that chromium is not using hardware acceleration.
But here you have a mozilla engineer specifically working on crashes, and instead of trying to work with them (possibly off-board) and with the data they have to diagnose the issue — because I'd assume when gsvelto talks about commit space contents that's what they see in the crash reports - you're just being a snarky asshole.
> It seems unlikely given this exchange, as you seem to have decided to be as unhelpful as possible.
I'm not being unhelpful on purpose. They are just giving me answers that do not solve my problem. My problem is "Firefox requires overcommit to run for long periods of time ... because it has a memory leak". The solution to this problem is to figure out where the memory leak is coming from, and fix it, right? Not just to add more memory, even if it is only virtual memory.
> Guess what's going to create textures which might trigger such an issue? The browser's hardware acceleration. Guess what happens to browser's textures when the browser is killed? They're freed.
You just described a memory leak, yes, the exact issue that I am experiencing.
> Might be a bug in Firefox, might be a bug in the crash reporter, might be a bug in the driver, might be that chromium is not using hardware acceleration.
Well, I don't think the crash reporter or Chromium are at fault. And I know Chromium is using hardware acceleration because if I choose the d3d11 backend then Netflix goes black in screenshots due to DRM. So, if it is a graphics issue, it's something that Firefox does and Chromium does not. Which leaves Firefox bugs, or bugs in the driver that only Firefox triggers.
> But here you have a mozilla engineer specifically working on crashes, and instead of trying to work with them (possibly off-board) and with the data they have to diagnose the issue — because I'd assume when gsvelto talks about commit space contents that's what they see in the crash reports - you're just being a snarky asshole.
I'm sorry that I came off as a snarky asshole and that wasn't really my intention. I've been dealing with this issue for months, and as a result of that there is a lack of effort from me and that probably shows. I am sorry.
I would like to perform some more experiments to figure out what the issue is, but the problem is that when Firefox runs out of memory it also crashes half the other things on my system, which makes it dangerous.
For me, Firefox slows down dramatically as it uses more memory (e.g. more tabs and windows opened), and reaches a less-than-usable state well before it has exhausted the available memory. Tab discarding - automatic or by literally closing the tabs - does seem to recover the memory used, but does not recover much of the performance (if any). My AMD 5800 and tens of gigs of memory runs into the same crushing performance blockers as my old 8GB FX-8300 machine, with virtually the same "workload" and usage profile.
Kinda the opposite of what I'd want; I usually have over 16GB more that Firefox could use if it needed, and that's once it has reached critical mass with maybe hundreds of tabs.
Its memory measurements usually look sane, so I feel like there's some data structure or algorithm that is doing something insane in the background - which is already confirmed to be the case with the History menu, particularly if you select and delete thousands of items at a time.
You don't need a lot to eventually get a crash on Firefox. All you really need is to hibernate/sleep instead of restarting, and so never restart your Firefox, and have some Twitch stream opened - the memory leaks will eventually use up all your memory, in my experience at least.
Firefox has been solid for me since the dawn of time, and I have or currently run it on Windows, Mac, Linux, and FreeBSD.
For a while it used to be kind of slow on Mac and Linux, but I think that was slow graphics calls, which points to a possible issue with the graphics driver I was using. But the last time I checked (many months ago), it was much better.
I recall a particularly strange Firefox bug on Windows where the browser would die with an out of memory error even though there were upwards of 16 GB free to use. As it turns out something was consuming large amounts of swap on the system, and this was where the out of memory condition was happening.
