#2993 Change: Increase vm.max_map_count value
Closed: Accepted 2 years ago by churchyard. Opened 2 years ago by bcotton.

This change aims at increasing the default value of the vm.max_map_count sysctl

Owners, do not implement this work until the FESCo vote has explicitly ended.
The Fedora Program Manager will create a tracking bug in Bugzilla for this Change, which is your indication to proceed.
See the FESCo ticket policy and the Changes policy for more information.

REMINDER: This ticket is for FESCo members to vote on the proposal. Further discussion should happen in the devel list thread linked above.


I think we should try and work with upstream here...

Hmm, there were important questions raised in the fedora-devel thread that didn't receive any real answers. ("Yes, this is an interesting question" is not a real answer ;) ).

In particular @fweimer wrote [1]:

You can play with the following program to overload the kernel with many
mappings:
...
In my experiments, the kernel OOM handler does not terminate this mapping-creating process, but random desktop services first.

[1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/YGTDBA2XENG7GSMHFYZQNIIGCGS3LQCD/

and Roberto Ragusa wrote:

The right question is: is there a value that is high enough for apps (including
Steam) and low enough to not be a security danger?
I mean from 65530 to 2billion there is a lot of middle ground.

I think it's the role of the Change Owners to answer those questions. This would require some research, or maybe even a lot of research. Before that happens, the change seems very risky.

-1

This is a big hammer for a narrow use case. There's no mention of why this is required for Steam or any potential negative impact for the system.

-1

I have to agree here. I would be open to raising the limit, but not just copying the huge value that SteamOS sets. The "HOWTO"s that are linked for various games use different values (1,048,576, 1,000,000, and 16,777,216, respectively), but all are much lower than the proposed limit of 2,147,483,642 (1-16 million vs. 2 billion).

-1 for setting the limit to vm.max_map_count=2147483642.

Hmm, there were important questions raised in the fedora-devel thread that didn't receive any real answers. ("Yes, this is an interesting question" is not a real answer ;) ).

Agreed, the concerns in the devel thread were not addressed, and not enough justification is given for the proposed limit value. The limit could be, and perhaps should be, increased from its current value, but the work to determine and justify an appropriate new value has not been done.

-1

Tagging this for next FESCo meeting for further discussion

Metadata Update from @amoloney:
- Issue tagged with: meeting

2 years ago

This will be discussed at today's meeting at 17:00 UTC.

This was discussed in today's FESCo meeting:

Metadata Update from @zbyszek:
- Issue untagged with: meeting

2 years ago

Speaking of limits and safety.

Fully updated Fedora 38 can be killed using the simplest fork bomb:

fork() {
    fork | fork &
}
fork

So, if people are concerned about DoS attacks, maybe people should start getting concerned that Fedora out of the box is not foolproof at all.

As noted above the highest value "suggested" by the internets is 16,777,216 on Counter Strike 2, which I could run locally and verify that (at least in a match against bots) doesn't request more than 80,000 mappings.
If we can assume that the other games I couldn't test work correctly with their respectively suggested values, then I suggest changing the proposed value to 1,048,576 (2^20), which gets nowhere near killing a 2GB memory machine when running the infamous reproducer.

Can anyone who have the game test Hogwarts Legacy with the settings from previous comment?

Metadata Update from @sgallagh:
- Issue tagged with: meeting

2 years ago

Fully updated Fedora 38 can be killed using the simplest fork bomb:
...

Are you sure. I just tested this, and while the user cannot fork processes, the machine is stable and other users can log in and/or continue their sessions. Maybe describe your setup and what you mean by "killed".

Fully updated Fedora 38 can be killed using the simplest fork bomb:
...

Are you sure. I just tested this, and while the user cannot fork processes, the machine is stable and other users can log in and/or continue their sessions. Maybe describe your setup and what you mean by "killed".

I don't know whether it's a sick joke or what.

I ran this code simple bash code on two of my PCs and both completely froze in less than 10 seconds. Both are running fully updated Fedora 38 installations with default settings with the only exception - no SWAP - I do not use it. I couldn't even start a new shell in a virtual console, as a typed "root", pressed "Enter" and I couldn't enter the password. CPUs were roaring. Ctrl + Alt + Del didn't work either. They were dead.

Should I record a video for you? Including a complete installation process using an official Fedora ISO?

No need to be so defensive. You could instead mention what environment you are using. Gnome?

You could instead mention what environment you are using. Gnome?

This is reproducible even without X.org running. This is reproducible at runlevel 1.

Here's Fedora 38 (Fedora-Xfce-Live-x86_64-38-1.6.iso) VM completely dying within 30 seconds of running this script:

https://mega.nz/file/S9shCAjQ#QPDAO40Opm_mY_plzB84uxZYp9Mb6EEkC0lUFbvqOnE

  • 4 virtual CPU cores.
  • 8GB of RAM.
  • No SWAP,

Takes less time on real hardware as VirtualBox struggles a little bit.

  • 2993 Change: Increase vm.max_map_count value (sgallagh, 17:06:33)

  • AGREED: Set the limit to 1048576 in Rawhide. We will revert the
    change if serious, practical exploits are discovered. (+7, 1, -0)
    (sgallagh, 17:29:16)

Should I edit the wiki page to reflect the agreed-upon value, and submit a pull request to src.fedoraproject.org? BTW I'm not sure which package should provide this config

Should I edit the wiki page to reflect the agreed-upon value

Yes.

and submit a pull request to src.fedoraproject.org?

Yes.

BTW I'm not sure which package should provide this config

systemd.

Metadata Update from @churchyard:
- Issue close_status updated to: Accepted
- Issue status updated to: Closed (was: Open)

2 years ago

Metadata Update from @amoloney:
- Issue untagged with: meeting
- Issue tagged with: pending announcement

2 years ago

As a small followup, I investigated one of the world's most bloated, poorly written and memory-hungry games: Star Citizen.

People have been recommending a completely nonsensical "16 million VMAs" value for it, while I've personally been playing it for years with 1 million VMA limit.

Here's how to do the test for any game:

  • To find the system's total VMA memory map usage: sudo cat /proc/*/maps | wc -l (this is an incorrect method if multiple processes can share VMAs though)
  • To find the game's process ID: ps aux | grep -i starcitizen
  • To find the game's VMA memory map usage: cat /proc/PROCESS_ID/maps | wc -l (where you replace PROCESS_ID with the game's PID from the previous command).

The Star Citizen VMA usage is as follows (multiple values for a stage means waiting for a few minutes and then sampling again):

  • While launching (before a window appears): 2381
  • At the game's main "server selection" menu: 9934, 9916
  • While the game is connecting to a server and loading the world: 9851, 9880, 18230
  • When awakening in the game's bedroom immediately after login: 42796
  • After traveling to personal hangar and retrieving a ship: 46367
  • After getting into the ship: 46493
  • After flying out into space: 47070
  • After quantum travel to asteroids for a mission: 44448
  • After getting out of my ship and doing a space walk to the mission target: 46203
  • During quantum travel while returning home after the mission: 46195
  • After quantum travel, idling outside my space station: 48363
  • After landing inside space station again: 49350
  • While unloading cargo from the ship into my hangar: 49944
  • Idling in my hangar: 50124, 50113, 50062
  • After retrieving a different ship and getting into it in my hangar: 51512
  • Flying out of the station, in space again: 51139
  • After quantum travel to a random, small moon: 47994
  • After flying closer to the moon: 49785
  • After getting into my ship's bed in space, right before logging out: 49813
  • After having logged back out to the main menu/server selection interface: 30977

Before closing the game, I also wanted to check the system's total VMA usage. As noted earlier, the method I used for that check is most likely extremely inaccurate (I suspect that multiple processes are referring to the same VMA areas). It doesn't make sense that this would be my system's idle VMA usage, since all kernels default to 65530 VMA limit as explained in the kernel here:

https://github.com/torvalds/linux/blob/v5.18/include/linux/mm.h#L178

But either way, no matter what this number represents, here's my "total" VMA usage:

  • While Star Citizen was running: 299099
  • After exiting Star Citizen: 249284 (a reduction of 49815)

So, time for conclusions:

  • Even a poorly programmed game like Star Citizen is clearly allocating and de-allocating memory map handles as-needed when game resources are streaming in/out of existence. There is no sign of any growth over time.
  • The "16 million" number is an idiotic myth and should not be taken seriously.
  • The "1 million" number chosen by Valve is overkill (especially since I highly doubt that my system actually uses 250k VMA pages, although I do have a ton of developer tools and very bloated applications open; 16 apps in total with lots of windows and tabs, several of which are Chromium and Electron based; so it's not completely impossible).
  • Even though we'd clearly be completely fine with something like half a million pages, I agree that "1" million is a safe bet with a huge margin, and it's not causing people any computer crashing or denial-of-service issues that we know of.

This Star Citizen test result also aligns well with @aleasto testing Counter-Strike 2 and finding that it never exceeeded 80,000 handles.

Another infamously heavy game is Hogwarts Legacy, but while I haven't personally measured it, that game has already been thoroughly confirmed to work perfectly with a 1 million VMA limit.

So this is settled. 1 Million is the highest we should ever go.

Log in to comment on this ticket.

Metadata