PR#50411: Issue 49875 - Move SystemD service config to a drop-in file - 389-ds-base

389-ds-base

#50411 Issue 49875 - Move SystemD service config to a drop-in file

Closed 3 years ago by spichugi. Opened 4 years ago by mhonek.

mhonek/389-ds-base systemd-dropin into master

Issue 49875 - Move SystemD service config to a drop-in file

Matus Honek • 4 years ago

10bffac

Makefile.am

file modified

+17 -6

		`@@ -300,6 +300,7 @@`
		`serverplugindir = $(libdir)@serverplugindir@`
		`taskdir = $(datadir)@scripttemplatedir@`
		`systemdsystemunitdir = @with_systemdsystemunitdir@`
		`+ systemdsystemunitdropindir = @with_systemdsystemunitdir@/$(PACKAGE_NAME)@.service.d`
		`systemdsystemconfdir = @with_systemdsystemconfdir@`
		`systemdgroupname = @with_systemdgroupname@`
		`initdir = @initdir@`
		`@@ -880,6 +881,11 @@`
		`systemdsystemunit_DATA = wrappers/$(PACKAGE_NAME)@.service \`
		`wrappers/$(systemdgroupname) \`
		`wrappers/$(PACKAGE_NAME)-snmp.service`
		`+`
		`+ systemdsystemunitdropin_DATA = wrappers/$(PACKAGE_NAME)@.service.d/custom.conf`
		`+ if with_sanitizer`
		`+ systemdsystemunitdropin_DATA += wrappers/$(PACKAGE_NAME)@.service.d/xsan.conf`
		`+ endif`
		`else`
		`if INITDDIR`
		`init_SCRIPTS = wrappers/$(PACKAGE_NAME) \`
		`@@ -2312,12 +2318,17 @@`
		`# yes, that is an @ in the filename . . .`
		`%/$(PACKAGE_NAME)@.service: %/systemd.template.service.in`
		`if [ ! -d $(dir $@) ] ; then mkdir -p $(dir $@) ; fi`
		`- if [ ! -z ${SANITIZER} ] ; then \`
		`- service_template=$(shell echo $^ \| sed 's/template/template.xsan/g'); \`
		`- else \`
		`- service_template=$^; \`
		`- fi; \`
		`- $(fixupcmd) $$service_template > $@`
		`+ $(fixupcmd) $^ > $@`
		`+`
		`+ %/$(PACKAGE_NAME)@.service.d/custom.conf: %/systemd.template.service.custom.conf.in`
		`+ if [ ! -d $(dir $@) ] ; then mkdir -p $(dir $@) ; fi`
		`+ $(fixupcmd) $^ > $@`
		`+`
		`+ if with_sanitizer`
		`+ %/$(PACKAGE_NAME)@.service.d/xsan.conf: %/systemd.template.service.xsan.conf.in`
		`+ if [ ! -d $(dir $@) ] ; then mkdir -p $(dir $@) ; fi`
		`+ $(fixupcmd) $^ > $@`
		`+ endif`

		`%/$(systemdgroupname): %/systemd.group.in`
		`if [ ! -d $(dir $@) ] ; then mkdir -p $(dir $@) ; fi`

configure.ac

file modified

		`@@ -196,6 +196,8 @@`
		`AC_SUBST([ubsan_rust_defs])`
		`AM_CONDITIONAL(enable_ubsan,test "$enable_ubsan" = "yes")`

		`+ AM_CONDITIONAL(with_sanitizer,test "$enable_asan" = "yes" -o "$enable_msan" = "yes" -o "$enable_tsan" = "yes" -o "$enable_ubsan" = "yes")`
		`+`
		`# Enable CLANG`
		`AC_MSG_CHECKING(for --enable-clang)`
		`AC_ARG_ENABLE(clang, AS_HELP_STRING([--enable-clang], [Enable clang (default: no)]),`

wrappers/systemd.template.service.custom.conf.in

file added

+52

		`@@ -0,0 +1,52 @@`
		`+ # To change any of the below values, please use a drop-in file in which`
		`+ # you can declare overrides according to systemd.unit(5), either of:`
		`+ # - applying to all instances:`
		`+ # /etc/systemd/system/dirsrv@.service.d/custom.conf`
		`+ # - applying to a single instance (overriding the above):`
		`+ # /etc/systemd/system/dirsrv@<instance>.service.d/custom.conf`
		`+ #`
		`+ # Some of the most interesting coniguration options are mentioned below.`
		`+ # See systemd.service(5) and systemd.exec(5) for the respective documentation.`
		`+ #`
		`+ # After updating the service configuration, do not forget to apply the changes:`
		`+ # - reload systemd configuration: systemctl daemon-reload`
		`+ # - restart the service: systemctl restart @package_name@@<instance>.service`
		`+`
		`+ [Service]`
		`+ TimeoutStartSec=0`
		`+ TimeoutStopSec=600`
		`+`
		`+ # These are from man systemd.exec and man systemd.resource-control`
		`+`
		`+ # This controls the resources to the direct child of systemd, in`
		`+ # this case ns-slapd. Because we are type notify we recieve these`
		`+ # limits correctly.`
		`+`
		`+ # This controls the number of file handles avaliable. File handles`
		`+ # correlate to sockets for the process, and our access to logs and`
		`+ # databases. Note, the configuration setting in Directory Server,`
		`+ # "nsslapd-maxdescriptors", can override this limit.`
		`+ #LimitNOFILE=`
		`+`
		`+ # You can limit the memory in the cgroup with these, and ns-slapd`
		`+ # will account for them in it's autotuning.`
		`+ # Memory account may be controlled by DefaultMemoryAccounting= in systemd-system.conf`
		`+ #MemoryAccounting=yes`
		`+ #MemoryLimit=<bytes>`
		`+`
		`+ # Limits on the size of coredump that may be produced by the process. It's not`
		`+ # specified how this interacts with coredumpd.`
		`+ # 0 means not to produce cores.`
		`+ #LimitCORE=<bytes>`
		`+`
		`+ # Limit number of processes (threads) we may spawn. We don't advise you change`
		`+ # this as DS will autodetect your threads / cpus and adjust as needed.`
		`+ #LimitNPROC=`
		`+`
		`+ # Possible hardening options:`
		`+ #PrivateDevices=yes`
		`+ #ProtectSystem=yes`
		`+ #ProtectHome=yes`
		`+ #PrivateTmp=yes`
		`+`
		`+`

wrappers/systemd.template.service.in

file modified

+3 -54

		`@@ -1,17 +1,6 @@`
		`- # you usually do not want to edit this file - instead, edit the`
		`- # @initconfigdir@/@package_name@.systemd file instead - otherwise,`
		`- # do not edit this file in /lib/systemd/system - instead, do the following:`
		`- # cp /lib/systemd/system/dirsrv\@.service /etc/systemd/system/dirsrv\@.service`
		`- # mkdir -p /etc/systemd/system/@systemdgroupname@.wants`
		`- # edit /etc/systemd/system/dirsrv\@.service - uncomment the LimitNOFILE=8192 line`
		`- # where %i is the name of the instance`
		`- # you may already have a symlink in`
		`- # /etc/systemd/system/@systemdgroupname@.wants/dirsrv@%i.service pointing to`
		`- # /lib/systemd/system/dirsrv\@.service - you will have to change it to link`
		`- # to /etc/systemd/system/dirsrv\@.service instead`
		`- # ln -s /etc/systemd/system/dirsrv\@.service /etc/systemd/system/@systemdgroupname@.wants/dirsrv@%i.service`
		`- # systemctl daemon-reload`
		`- # systemctl (re)start @systemdgroupname@`
		`+ # You should not need to edit this file. Instead, use a drop-in file as described in:`
firstyear commented 4 years ago Is it possible to show an "example" drop in, or link to the systemd.template.service.custom.conf.in somehow to guide admins to the "how" to achivee this?
		`+ # /usr/lib/systemd/system/@package_name@@.service.d/custom.conf`
		`+`
		`[Unit]`
		`Description=@capbrand@ Directory Server %i.`
		`PartOf=@systemdgroupname@`
		`@@ -21,51 +10,11 @@`
		`[Service]`
		`Type=notify`
		`NotifyAccess=all`
		`- TimeoutStartSec=0`
		`- TimeoutStopSec=600`
		`EnvironmentFile=-@initconfigdir@/@package_name@`
		`EnvironmentFile=-@initconfigdir@/@package_name@-%i`
		`PIDFile=@localstatedir@/run/@package_name@/slapd-%i.pid`
		`ExecStartPre=@libexecdir@/ds_systemd_ask_password_acl @instconfigdir@/slapd-%i/dse.ldif`
		`ExecStart=@sbindir@/ns-slapd -D @instconfigdir@/slapd-%i -i @localstatedir@/run/@package_name@/slapd-%i.pid`

		`- #### To change any of these values or directives, you should use a drop in file`
		`- # such as: /etc/systemd/system/dirsrv@<instance>.d/custom.conf`
		`-`
		`- # These are from man systemd.exec and man systemd.resource-control`
		`-`
		`- # This controls the resources to the direct child of systemd, in`
		`- # this case ns-slapd. Because we are type notify we recieve these`
		`- # limits correctly.`
		`-`
		`- # This controls the number of file handles avaliable. File handles`
		`- # correlate to sockets for the process, and our access to logs and`
		`- # databases. Note, the configuration setting in Directory Server,`
		`- # "nsslapd-maxdescriptors", can override this limit.`
		`- LimitNOFILE=16384`
		`-`
		`- # You can limit the memory in the cgroup with these, and ns-slapd`
		`- # will account for them in it's autotuning.`
		`- # Memory account may be controlled by DefaultMemoryAccounting= in systemd-system.conf`
		`- # MemoryAccounting=true`
		`- # MemoryLimit=bytes`
		`-`
		`- # Limits on the size of coredump that may be produced by the process. It's not`
		`- # specified how this interacts with coredumpd.`
		`- # 0 means not to produce cores.`
		`- # This value is 64G`
		`- LimitCORE=68719476736`
		`-`
		`- # Limit number of processes (threads) we may spawn. We don't advise you change`
		`- # this as DS will autodetect your threads / cpus and adjust as needed.`
		`- # LimitNPROC=`
		`-`
		`- # Hardening options:`
		`- # PrivateDevices=true`
		`- # ProtectSystem=true`
		`- # ProtectHome=true`
		`- # PrivateTmp=true`
		`-`
		`[Install]`
		`WantedBy=multi-user.target`
		`-`

wrappers/systemd.template.service.xsan.conf.in

file added

+11

		`@@ -0,0 +1,11 @@`
		`+ # This file is present because the server has been built with a sanitizer.`
		`+ # It is not meant for a production usage.`
		`+ [Unit]`
		`+ Description=@capbrand@ Directory Server with @SANITIZER@ %i.`
		`+`
		`+ [Service]`
		`+ # We can't symbolize here, as llvm symbolize crashes when it goes near systemd.`
		`+ Environment=ASAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.asan:print_stacktrace=1`
		`+ Environment=TSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.tsan:print_stacktrace=1:second_deadlock_stack=1:history_size=7`
		`+ Environment=MSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.msan:print_stacktrace=1`
		`+ Environment=UBSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.ubsan:print_stacktrace=1`

wrappers/systemd.template.xsan.service.in

file removed

-77

		`@@ -1,77 +0,0 @@`
		`- # you usually do not want to edit this file - instead, edit the`
		`- # @initconfigdir@/@package_name@.systemd file instead - otherwise,`
		`- # do not edit this file in /lib/systemd/system - instead, do the following:`
		`- # cp /lib/systemd/system/dirsrv\@.service /etc/systemd/system/dirsrv\@.service`
		`- # mkdir -p /etc/systemd/system/@systemdgroupname@.wants`
		`- # edit /etc/systemd/system/dirsrv\@.service - uncomment the LimitNOFILE=8192 line`
		`- # where %i is the name of the instance`
		`- # you may already have a symlink in`
		`- # /etc/systemd/system/@systemdgroupname@.wants/dirsrv@%i.service pointing to`
		`- # /lib/systemd/system/dirsrv\@.service - you will have to change it to link`
		`- # to /etc/systemd/system/dirsrv\@.service instead`
		`- # ln -s /etc/systemd/system/dirsrv\@.service /etc/systemd/system/@systemdgroupname@.wants/dirsrv@%i.service`
		`- # systemctl daemon-reload`
		`- # systemctl (re)start @systemdgroupname@`
		`- [Unit]`
		`- Description=@capbrand@ Directory Server with @SANITIZER@ %i.`
		`- PartOf=@systemdgroupname@`
		`- After=chronyd.service ntpd.service network-online.target syslog.target`
		`- Before=radiusd.service`
		`-`
		`- [Service]`
		`- Type=notify`
		`- NotifyAccess=all`
		`- TimeoutStartSec=0`
		`- TimeoutStopSec=600`
		`- EnvironmentFile=@initconfigdir@/@package_name@`
		`- EnvironmentFile=@initconfigdir@/@package_name@-%i`
		`- PIDFile=@localstatedir@/run/@package_name@/slapd-%i.pid`
		`- # We can't symbolize here, as llvm symbolize crashes when it goes near systemd.`
		`- Environment=ASAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.asan:print_stacktrace=1`
		`- Environment=TSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.tsan:print_stacktrace=1:second_deadlock_stack=1:history_size=7`
		`- Environment=MSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.msan:print_stacktrace=1`
		`- Environment=UBSAN_OPTIONS=log_path=@localstatedir@/run/@package_name@/ns-slapd-%i.ubsan:print_stacktrace=1`
		`- LimitCORE=infinity`
		`- ExecStartPre=@libexecdir@/ds_systemd_ask_password_acl @instconfigdir@/slapd-%i/dse.ldif`
		`- ExecStart=@sbindir@/ns-slapd -D @instconfigdir@/slapd-%i -i @localstatedir@/run/@package_name@/slapd-%i.pid`
		`-`
		`- #### To change any of these values or directives, you should use a drop in file`
		`- # such as: /etc/systemd/system/dirsrv@<instance>.d/custom.conf`
		`-`
		`- # These are from man systemd.exec and man systemd.resource-control`
		`-`
		`- # This controls the resources to the direct child of systemd, in`
		`- # this case ns-slapd. Because we are type notify we recieve these`
		`- # limits correctly.`
		`-`
		`- # This controls the number of file handles avaliable. File handles`
		`- # correlate to sockets for the process, and our access to logs and`
		`- # databases.`
		`- LimitNOFILE=16384`
		`-`
		`- # You can limit the memory in the cgroup with these, and ns-slapd`
		`- # will account for them in it's autotuning.`
		`- # Memory account may be controlled by DefaultMemoryAccounting= in systemd-system.conf`
		`- # MemoryAccounting=true`
		`- # MemoryLimit=bytes`
		`-`
		`- # Limits on the size of coredump that may be produced by the process. It's not`
		`- # specified how this interacts with coredumpd.`
		`- # 0 means not to produce cores.`
		`- # This value is 64G`
		`- LimitCORE=68719476736`
		`-`
		`- # Limit number of processes (threads) we may spawn. We don't advise you change`
		`- # this as DS will autodetect your threads / cpus and adjust as needed.`
		`- # LimitNPROC=`
		`-`
		`- # Hardening options:`
		`- # PrivateDevices=true`
		`- # ProtectSystem=true`
		`- # ProtectHome=true`
		`- # PrivateTmp=true`
		`-`
		`-`
		`- [Install]`
		`- WantedBy=multi-user.target`
		`-`

mhonek commented 4 years ago

Bug Description:
Runtime configuration options are mixed into the service specification
which should seldom be changed by users.

Fix Description:
Move the runtime configuration options into a drop-in file. These options
are then automatically pull in by SystemD.

Additional Info:
Erasing the default values of the mentioned options to implicitly pull in
system defaults which are more sane nowadays.

Author: Matus Honek mhonek@redhat.com

Review by: ???

vashirov commented 4 years ago

I was thinking of dropping systemd.template.xsan.service.in completely and use a drop-in file for Environment options for ASAN/MSAN/TSAN instead of maintaining two virtually identical files. WDYT?

firstyear commented 4 years ago

It's not possible as you can't make certain sections "drop in" IIRC. I experimented with this and chose the split file method to avoid this. I could be mistaken here or it's changed since, but previous attempts were not valid unit files :(

firstyear commented on line 16 of wrappers/systemd.template.service.in 4 years ago

Is it possible to show an "example" drop in, or link to the systemd.template.service.custom.conf.in somehow to guide admins to the "how" to achivee this?

vashirov commented 4 years ago

It's not possible as you can't make certain sections "drop in" IIRC. I experimented with this and chose the split file method to avoid this. I could be mistaken here or it's changed since, but previous attempts were not valid unit files :(

I'm sorry to disappoint, but this is exactly what drop-in files are for: to override certain parameters in certain sections. This functionality is in systemd for quite a while.

Here's an example:

[vashirov@server ~]$ cat .config/systemd/user/hello.service
[Unit]
Description=Service that prints a greeting

[Service]
ExecStart=/usr/bin/echo "Hello ${WORLD}"
Environment=WORLD=World

[Install]
WantedBy=default.target

[vashirov@server ~]$ systemctl --user daemon-reload
[vashirov@server ~]$ systemctl --user start hello
[vashirov@server ~]$ journalctl --user -u hello
-- Logs begin at Thu 2019-05-30 00:35:33 EDT, end at Thu 2019-05-30 00:37:39 EDT. --
May 30 00:37:39 server.example.com systemd[1039]: Started Service that prints a greeting.
May 30 00:37:39 server.example.com echo[1085]: Hello World
May 30 00:37:39 server.example.com systemd[1039]: hello.service: Succeeded.

Now let's create an override:

[vashirov@server ~]$ systemctl --user edit hello

This will open an $EDITOR, but you can create drop-in directory by hand an place a .conf file there yourself.

[vashirov@server ~]$ cat .config/systemd/user/hello.service.d/override.conf
[Unit]
Description=Service with an overrided description

[Service]
Environment=WORLD="Down Under"

[vashirov@server ~]$ systemctl --user start hello
[vashirov@server ~]$ journalctl --user -u hello
-- Logs begin at Thu 2019-05-30 00:35:33 EDT, end at Thu 2019-05-30 00:39:00 EDT. --
May 30 00:37:39 server.example.com systemd[1039]: Started Service that prints a greeting.
May 30 00:37:39 server.example.com echo[1085]: Hello World
May 30 00:37:39 server.example.com systemd[1039]: hello.service: Succeeded.
May 30 00:39:00 server.example.com systemd[1039]: Started Service with an overrided description.
May 30 00:39:00 server.example.com echo[1096]: Hello Down Under
May 30 00:39:00 server.example.com systemd[1039]: hello.service: Succeeded.

Is it possible to show an "example" drop in, or link to the systemd.template.service.custom.conf.in somehow to guide admins to the "how" to achivee this?

We can point them to man systemd.unit where it is described.

1 new commit added

Change xsan service into a drop-in file

4 years ago

mhonek commented 4 years ago

I've added a change as Viktor suggested. @firstyear Please have a look. This drop-in file will be installed in case the server is built with a sanitizer and automatically picked up by systemd.

firstyear commented 4 years ago

It's not possible as you can't make certain sections "drop in" IIRC. I experimented with this and chose the split file method to avoid this. I could be mistaken here or it's changed since, but previous attempts were not valid unit files :(

I'm sorry to disappoint, but this is exactly what drop-in files are for: to override certain parameters in certain sections. This functionality is in systemd for quite a while.

This is not my point. I know how drop in's work. My point was that drop in files may not function for some configuration sections in the unitfile. IIRC if you try to drop in over exec or certain other parameters you have to make a new unit file because you can't replace that statement. So if this works, whatever you are "replacing" is not falling under these rules. Additionally my comment was to ask you to check that it actually was working, because there are some conditions where the drop in is not used because of the requirement that you have to use or replace the section.

So if this is working for you, and tested, then great. But bear in mind there are some things that can not be drop-in overriden ...

We can point them to man systemd.unit where it is described.

I think I mean that our unitfile should say something like:

# To configure this, create a drop in file in:
# /etc/systemd/system/dirsrv.whatever.target.d/example.com
# It's contents should be from ...
# For more see <man page>

mhonek commented 4 years ago

... IIRC if you try to drop in over exec or certain other parameters you have to make a new unit file because you can't replace that statement. ...

IIUC what your concern is, you can do e.g. ExecStart= (that is, without a value) in a drop-in file first which resets the value and subsequent assignmnets would build the value of it again.

... So if this is working for you, and tested, then great. But bear in mind there are some things that can not be drop-in overriden ...

At least me and Mark have successfully tested this to find it working as expected.

I think I mean that our unitfile should say something like:

The main .service file contains pointer to the drop-in file where supposed handling is described -- do you find this sufficient? Suggestions to particular changes are welcome. :)

firstyear commented 4 years ago

Yep, that's fine, if it's all tested then my concerns are covered.

rebased onto 4981ebf6a44cb0c3dd1a091786ee0a58eeeaccfe

4 years ago

rebased onto 10bffac

4 years ago

Pull-Request has been merged by mhonek

4 years ago