#8635 Memory availability detection does not work with cgroupsv2 environment
Closed: fixed 3 years ago by frenaud. Opened 3 years ago by abbra.

The code in check_available_memory() helper fails in the case of a container on cgroupv2-enabled system. This is reproducible with ubi8-init container, for example. I tried with CentOS 8 Stream on top of ubi8-init, running on Fedora 33.

[root@m1 ~]# ipa-server-install -a Secret123 -p Secret123 --setup-dns --auto-forwarders -r IPA.TEST -U --force
Option --force-ntpd has been deprecated and will be removed in a future release.
Unable to determine the amount of available RAM
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information
[root@m1 ~]# ls /sys/fs/cgroup/memory/memory.limit_in_bytes
ls: cannot access '/sys/fs/cgroup/memory/memory.limit_in_bytes': No such file or directory
[root@m1 ~]# ls /sys/fs/cgroup/memory/
ls: cannot access '/sys/fs/cgroup/memory/': No such file or directory
[root@m1 ~]# ls /sys/fs/cgroup/
cgroup.controllers  cgroup.freeze     cgroup.max.descendants  cgroup.stat         cgroup.threads  cpu.pressure  init.scope   memory.current  memory.events.local  memory.low  memory.min        memory.pressure  memory.swap.current  memory.swap.high  pids.current  pids.max
cgroup.events       cgroup.max.depth  cgroup.procs        cgroup.subtree_control  cgroup.type     cpu.stat      io.pressure  memory.events   memory.high          memory.max  memory.oom.group  memory.stat      memory.swap.events   memory.swap.max   pids.events   system.slice
    if in_container():
        if os.path.exists(
            '/sys/fs/cgroup/memory/memory.limit_in_bytes'
        ) and os.path.exists('/sys/fs/cgroup/memory/memory.usage_in_bytes'):
            with open('/sys/fs/cgroup/memory/memory.limit_in_bytes') as fd:
                limit = int(fd.readline())
            with open('/sys/fs/cgroup/memory/memory.usage_in_bytes') as fd:
                used = int(fd.readline())
            available = limit - used
        else:
            raise ScriptError(
                "Unable to determine the amount of available RAM"
            )

In cgroupv2 environments memory container interface provides none of the proc files we check, thus causing a ScriptError exception. We do not support cgroupv2 at all but the failure is only visible in the container.

[root@m1 ~]# python3
Python 3.6.8 (default, Dec  3 2020, 18:11:24) 
[GCC 8.4.1 20200928 (Red Hat 8.4.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil
>>> psutil.virtual_memory()
svmem(total=67450454016, available=56323100672, percent=16.5, used=9726017536, free=31081684992, active=11837841408, inactive=19466330112, buffers=1631563776, cached=25011187712, shared=681390080)
>>> with open('/sys/fs/cgroup/memory.current') as fd:
...     current = int(fd.readline())
... 
>>> current
4083568640
>>> from ipaserver.install import installutils
>>> installutils.in_container()
True

More details on how cgroupv2 is organized can be found at https://www.kernel.org/doc/Documentation/cgroup-v2.txt


Metadata Update from @rcritten:
- Issue assigned to rcritten

3 years ago

master:

  • 176fe83 Add support for cgroup v2 to the installer memory checker
  • 573d66e ipatests: test the cgroup v2 memory restrictions

ipa-4-9:

  • 1dd4501 Add support for cgroup v2 to the installer memory checker
  • 85d944c ipatests: test the cgroup v2 memory restrictions

Metadata Update from @frenaud:
- Issue close_status updated to: fixed
- Issue status updated to: Closed (was: Open)

3 years ago

Metadata Update from @abbra:
- Custom field changelog adjusted to Support IPA deployment in cgroup v2 containerized environments

3 years ago

Metadata Update from @abbra:
- Custom field changelog adjusted to Containerized environments on Linux with cgroup v2 are now recognized and supported. (was: Support IPA deployment in cgroup v2 containerized environments)

3 years ago

Login to comment on this ticket.

Metadata