#51010 Ticket 51008 - dbhome in containers
Closed 3 years ago by spichugi. Opened 4 years ago by firstyear.
firstyear/389-ds-base 51008-dbhome-container  into  master

@@ -181,6 +181,7 @@ 

          '/data/ldif',

          '/data/run',

          '/data/run/lock',

+         '/data/run/dbhome',

          '/data/logs'

      ]:

          if not os.path.exists(d):
@@ -222,6 +223,14 @@ 

          s2b.set('local_state_dir', '/data')

          s2b.set('inst_dir', '/data')

          s2b.set('db_dir', '/data/db')

+         # 51008, ds changed dbhome for bdb to /dev/shm, however in docker this

+         # defaults to 64M and DS requires at least 130M. It may not be possible to

+         # Ask people to change this in their deployments, and in the interest of

+         # simplicity, we just put this into a /data/run directory instead to make

+         # containers as painless as possible. If people want it to be a ram disk

+         # they can easily use something like:

+         #    docker run --tmpfs /data/run/dbhome:rw

+         s2b.set('db_home_dir', '/data/run/dbhome')

How this setting helps regarding the constraint of /data/run being large enough for db environment ?

          # Why is this bak? Some dsctl commands use INST_DIR/bak, not "backup_dir"

          # due to some legacy handling of paths in lib389's population of instances.

          s2b.set('backup_dir', '/data/bak')

@@ -720,10 +720,6 @@ 

                  dse += line.replace('%', '{', 1).replace('%', '}', 1)

  

          with open(os.path.join(slapd['config_dir'], 'dse.ldif'), 'w') as file_dse:

-             db_home_dir = slapd['db_home_dir']

-             if self.containerised:

-                 # don't set db_home_dir to tmpfs in containers

-                 db_home_dir = slapd['db_dir']

              dse_fmt = dse.format(

                  schema_dir=slapd['schema_dir'],

                  lock_dir=slapd['lock_dir'],
@@ -745,7 +741,7 @@ 

                  ds_suffix=ds_suffix,

                  config_dir=slapd['config_dir'],

                  db_dir=slapd['db_dir'],

-                 db_home_dir=db_home_dir

+                 db_home_dir=slapd['db_home_dir']

              )

              file_dse.write(dse_fmt)

  

Bug Description: When starting 389 in containers, the shm may be too
small. Mark fixed this in #51007, but it removed the ability to have
the dbhome in a tmpfs/shm.

Fix Description: Move the request for dbhome/container logic into
the dscontainer entry point instead to keep the setup.py simpler,
and make the dbhome in /data/run/dbhome allowing people to use
an shm for dbhome with '--tmpfs /data/run/dbhome:rw' to a docker
run or create command.

https://pagure.io/389-ds-base/issue/51008

Author: William Brown william@blackhats.net.au

Review by: ???

How this setting helps regarding the constraint of /data/run being large enough for db environment ?

tmpfs is by default bigger than defaul /dev/shm

sh# docker run --tmpfs=/tmp -ti --rm fedora:31 bash
[root@bf2f6bc5e849 /]# df -h /tmp/
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           7.7G     0  7.7G   0% /tmp
[root@bf2f6bc5e849 /]# df -h /dev/shm/
Filesystem      Size  Used Avail Use% Mounted on
shm              64M     0   64M   0% /dev/shm

But you can change size of them from command line.

                    --tmpfs=/run/user:rw,nosuid,nodev,relatime,size=1603356k \
                    --shm-size=512m

Yes I'm aware of that @lslebodn.

The point is to make it so that:

docker run 389ds/dirsrv:latest

Works out of the box, not requiring extra magic or arguments.

How this setting helps regarding the constraint of /data/run being large enough for db environment ?

I believe this means the size of the storage and its type becomes a problem of the host/runner; this is not supposed to fix the fact it might be small, rather defer this problem to the host who can fix this potential issue with no need to change the arguments to docker run. Firstyear, please correct me if I am wrong here. If not then LGTM. :)

docker by default has /dev/shm set to 64m. DS requires at least 130M.

This means:

docker run 389ds/dirsrv:latest

Would fail unless you add '--shm-size=256M'.

@mreynolds had a fix for this in his patch, but that set dbhome to the same location as the db, IE /data/db. Putting the dbhome into a ramdisk has some performance benefits and the way he changed it eliminated that option. So my change makes it possible to have this by setting it to /data/run/dbhome instead.

And yes, @mhonek you are correct :)

rebased onto d7810ff

4 years ago

Pull-Request has been merged by firstyear

4 years ago

We're seeing more issues with containers and db-home-dir when the dscontainer is NOT used.

Would this change still work for your patch's purpose? It won't override it correct?

@@ -720,6 +720,11 @@ class SetupDs(object):
                 dse += line.replace('%', '{', 1).replace('%', '}', 1)

         with open(os.path.join(slapd['config_dir'], 'dse.ldif'), 'w') as file_dse:
+            db_home_dir = slapd['db_home_dir']
+            if self.containerised and slapd['db_home_dir'] == ds_paths.db_home_dir:
+                # In a container, but we can not use the default location.  Change it db_dir
+                db_home_dir = slapd['db_dir']
+
             dse_fmt = dse.format(
                 schema_dir=slapd['schema_dir'],
                 lock_dir=slapd['lock_dir'],
@@ -741,7 +746,7 @@ class SetupDs(object):
                 ds_suffix=ds_suffix,
                 config_dir=slapd['config_dir'],
                 db_dir=slapd['db_dir'],
-                db_home_dir=slapd['db_home_dir']
+                db_home_dir=db_home_dir
             )

yeah, that wourd work for me. :)

389-ds-base is moving from Pagure to Github. This means that new issues and pull requests
will be accepted only in 389-ds-base's github repository.

This pull request has been cloned to Github as issue and is available here:
- https://github.com/389ds/389-ds-base/issues/4063

If you want to continue to work on the PR, please navigate to the github issue,
download the patch from the attachments and file a new pull request.

Thank you for understanding. We apologize for all inconvenience.

Pull-Request has been closed by spichugi

3 years ago