Markdown source

#How to use systemd for applications start (revised on May 2019)

<abstract>
On the latest Debian distributions is installed by default systemd instead of sysvinit as initialization system.
This article illustrates how to use it by some examples of usual cases. 
</abstract>

## Create a service

To launch a script at startup a proper service file have to be created and added to the systemd.

Let's create a service file for our blik.py program. 
The simplest way to create a new service is to use the following command

	sudo systemctl --force edit blink.service

that will create a file called __blink.service__ in __/etc/systemd/system__ directory.
In any case the exact location of the file will be shown when you edit again the file.

Then save inside it this content:	

	[Unit]
	Description=Blinking led
	After=network.target
	
	[Service]
	Type=idle
	ExecStart=/usr/bin/python /home/pi/blink.py
	Restart=always
	User=pi
	
	[Install]
	WantedBy=multi-user.target

Next systemd has to be configured, enabling the service so at the next reboot it will be started automatically.
First off, though, the systemd itself has to be reloaded in order to detect the new service definition:

	sudo systemctl daemon-reload

Next, enable the service

	sudo systemctl enable blink

Check if it is enabled:

	sudo systemctl list-unit-files | grep enabled

Of course, it can be started manually just now:

	sudo systemctl start blink

The PID (process id) can be now checked:

	ps -ef | grep blink
	pi         817     1  0 09:19 ?        00:00:00 /usr/bin/python blink.py

it is 817 (the number will be different in your case).
Please note that it has been started from systemd process, in fact the PPID (parent process id) is <b>1</b>.

### Test what happens if the program crashes and/or terminates.

In order to simulate a crash, we kill it manually:

	sudo kill 817

Now we can check it was restarted looking again  to the PID (process id):

	ps -ef | grep blink
	pi        1037     1  0 09:19 ?        00:00:00 /usr/bin/python blink.py

it is now 1037 (PPID always <b>1</b>), so, as it is different, that means has been restarted automatically after the kill.

### How to edit again the service file

	sudo systemctl edit blink.service --full

## Use the systemd logging system

To check what is doing your service launch this command:

	journalctl -f blink

It is possible, of course, to add our own messages to this log. In Python, for example,
install the __python-systemd__ package:

	sudo apt-get update
	sudo apt-get install python-systemd
	
add these lines to your code:

	import logging
	from systemd.journal import JournalHandler
	
	log = logging.getLogger('blink')
	log.addHandler(JournalHandler())
	log.setLevel(logging.INFO)
	
	log.info("Blinking led started")

Open another terminal and launc journalctl with follow option (-f):

	journalctl -u -f blink
	...
	May 13 11:49:24 roadrunner systemd[1]: Started Blinking led.
	May 13 11:49:25 roadrunner /home/pi/blink.py[18081]: Blinking led started

## Create a timer for your service

<abstract>
It is possible to use the systemd timers to launch a service instead of using cron
</abstract>

### Example 1: Run a service every hour

Create a systemd service file 

	sudo systemctl --force edit blink.service	

with this contents:

	[Unit]
	Description=Blinking led
	
	[Service]
	WorkingDirectory=/home/pi
	ExecStart=python blink.py
	User=pi

Create a systemd timer file for this service 

    sudo systemctl --force edit blink.timer

with this contents::

	[Unit]
	Description=Runs blink.py every hour
	
	[Timer]
	# Time to wait after booting before we run first time
	OnBootSec=10min
	# Time between running each consecutive time
	OnUnitActiveSec=1h
	Unit=blink.service
	
	[Install]
	WantedBy=multi-user.target

Reload the service definitions:

	sudo systemctl daemon-reload

Then is your are using the same file of previous example disable the blink.service and enable just the blink.timer
to start it after the bootstrap:

	sudo systemctl disable blink.service
	sudo systemctl enable blink.timer
	
Let's try it now:	

	sudo systemctl start blink.timer

### Example 2: Run a service daily

	[Unit]
	Description=Runs blink.py daily at midnight
	
	[Timer]
	OnCalendar=*-*-* 00:00:00
	Persistent=True	
	Unit=blink.service
	
	[Install]
	WantedBy=multi-user.target

Changing the __OnCalendar__ parameter is possible to start the blink.service 
in a lot of different way:

	Minimal form                   Normalized form
	Sat,Thu,Mon-Wed,Sat-Sun    ==> Mon-Thu,Sat,Sun *-*-* 00:00:00
	Mon,Sun 12-*-* 2,1:23      ==> Mon,Sun 2012-*-* 01,02:23:00
	Wed *-1                    ==> Wed *-*-01 00:00:00
	Wed-Wed,Wed *-1            ==> Wed *-*-01 00:00:00
	Wed, 17:48                 ==> Wed *-*-* 17:48:00
	Wed-Sat,Tue 12-10-15 1:2:3 ==> Tue-Sat 2012-10-15 01:02:03
	*-*-7 0:0:0                ==> *-*-07 00:00:00
	10-15                      ==> *-10-15 00:00:00
	monday *-12-* 17:00        ==> Mon *-12-* 17:00:00
	Mon,Fri *-*-3,1,2 *:30:45  ==> Mon,Fri *-*-01,02,03 *:30:45
	12,14,13,12:20,10,30       ==> *-*-* 12,13,14:10,20,30:00
	mon,fri *-1/2-1,3 *:30:45  ==> Mon,Fri *-01/2-01,03 *:30:45
	03-05 08:05:40             ==> *-03-05 08:05:40
	08:05:40                   ==> *-*-* 08:05:40
	05:40                      ==> *-*-* 05:40:00
	Sat,Sun 12-05 08:05:40     ==> Sat,Sun *-12-05 08:05:40
	Sat,Sun 08:05:40           ==> Sat,Sun *-*-* 08:05:40
	2003-03-05 05:40           ==> 2003-03-05 05:40:00
	2003-03-05                 ==> 2003-03-05 00:00:00
	03-05                      ==> *-03-05 00:00:00
	hourly                     ==> *-*-* *:00:00
	daily                      ==> *-*-* 00:00:00
	monthly                    ==> *-*-01 00:00:00
	weekly                     ==> Mon *-*-* 00:00:00
	*:20/15                    ==> *-*-* *:20/15:00

* [Original article](https://www.certdepot.net/rhel7-use-systemd-timers/)



### Check the list of timers

	systemctl list-timers --all
	...
	NEXT                          LEFT     LAST                          PASSED       UNIT                         ACTIVATES
	Mon 2018-05-14 00:00:00 CEST  10h left Sun 2018-05-13 13:14:44 CEST  2min 17s ago blink.timer                  blink.service
	Mon 2018-05-14 10:38:28 CEST  21h left Sun 2018-05-13 10:38:28 CEST  2h 38min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
	n/a                           n/a      n/a                           n/a          apt-daily-upgrade.timer      apt-daily-upgrade.service
	n/a                           n/a      n/a                           n/a          apt-daily.timer              apt-daily.service

### Links

* [systemd.timer configuration](https://www.freedesktop.org/software/systemd/man/systemd.timer.html)
* [How to Use Systemd Timers](https://jason.the-graham.com/2013/03/06/how-to-use-systemd-timers/)


## How to start a script right before shutdown

Excerpted from 

* [Run a service right before starting any of reboot/shutdown/halt/kexec services](https://unix.stackexchange.com/questions/39226/how-to-run-a-script-with-systemd-right-before-shutdown)


To run a service (shutdown.py in the example) right before starting any of reboot/shutdown/halt/kexec services 
(i.e. in the last moment before root filesystem becomes remounted read-only) use this service config:

	sudo systemctl edit my_shutdown.py --full

	[Unit]
	Description=Save system clock on shutdown
	DefaultDependencies=no
	After=final.target

	[Service]
	Type=oneshot
	ExecStart=/home/pi/shutdown.py

	[Install]
	WantedBy=final.target

Enable it with:

	systemctl enable my_shutdown.service

Reload the new service definition:

	sudo systemctl daemon-reload
 
Then shutdown:

	sudo systemctl halt

To run a script right before actual reboot/shutdown/halt/kexec (when you cannot write to the root filesystem, 
because it was remounted read-only) add this script executable to the /usr/lib/systemd/system-shutdown directory.

Immediately before executing the actual system halt/poweroff/reboot/kexec systemd-shutdown 
will run all executables in /usr/lib/systemd/system-shutdown/ 
and pass one arguments to them: either "halt", "poweroff", "reboot" or "kexec", 
depending on the chosen action. 
All executables in this directory are executed in parallel, 
and execution of the action is not continued before all executables finished.



## How to setup root autologin on roadrunner and CM3 serial console

During intensive board test it is convenient to temporarily disable 
the standard login procedure on the physical console, usually found on all systems
on the ttyS0.

In any case, the physical serial associated at the console can be found 
reading a kernel file:

	# Raspbian on CM3
	cat /sys/devices/virtual/tty/console/active
	tty1 ttyS0 ttyAMA0

	# Debian 8.0 on RoadRunner
	cat /sys/devices/virtual/tty/console/active
	ttyS0

In Unix the consoles are managed using the program getty (agetty in modern distributions).
getty opens the serial line file and, when it detects a connection, it prompts for a username 
and runs the __login__ program to authenticate the user.

In the old systems, where the first process is the infamous __init__ , the getty processes were 
started from the /etc/inittab with the respawn option active so that when it should terminate, 
it will be restarted.

In order to bypass the password request the __a__ __<username>__ otpion has to be added to the command lineof getty.


### inittab

For inittab it is enough to edit it and force the reading with

	init q

next the getty have to be killed, init will restart it automatically.


### systemd

For recent systems using systemd, follow the procedure below.

Edit the service file using the following command

	sudo systemctl edit serial-getty@.service

that will create an override file

The original line starting with __ExecStart__ is as follows

	ExecStart=-/sbin/agetty --keep-baud 115200,38400,9600 %I $TERM

so, in the override file that has to be changed with

	ExecStart=-/sbin/agetty <b>-a root</b> --keep-baud 115200,38400,9600 %I $TERM

after that the systemd have to be forced to re-read the files

	systemctl daemon-reload
	
lastly the service has to be restarted

	systemctl restart serial-getty@ttyS0.service

For CM3 __sudo__ has to be preponed to all commands and the user should be changed from __root__ to __pi__.

### Links

* [systemd for Administrators, Part XVI](http://0pointer.de/blog/projects/serial-console.html)


## SystemV the old way

A few pieces of old way are still present and working as for example __/etc/rc.local__ where 
all the applications that implement the system functionality are started.  

A typical __rc.local__ contains:

	cat /etc/rc.local
	
	#!/bin/sh -e
	#
	# rc.local
	#
	# This script is executed at the end of each multiuser runlevel.
	# Make sure that the script will "exit 0" on success or any other
	# value on error.
	#
	# In order to enable or disable this script just change the execution
	# bits.
	#
	# By default this script does nothing.
	
	cd /home/acmesystems
	/usr/bin/nohup /usr/sbin/python blink.py &
	exit 0

In this example a Python program is started from application's home directory, and left in background (using the trailing ampersand). 
That works passably well till our python application never crashes: in such unfortunate case,  our system only chance
has, in order to regain the lost functionality, is to be manually restarted by user (probably cycling the power...).

Usually, the quick and dirty solution suggested is to add our application to the __/etc/inittab__ file, specifyng a __respawn__ 
in the line:

	id:2345:respawn:/bin/sh /path/to/application/startup

in such case startup must be a proper script able to start out application (and probably it could be convenient to reuse 
this script as is even in /etc/rc.local).


##Can systemd use __as is__ a classic System file startup script ?

How it can be seen using mosquitto server, systemd is able to keep in account all System V entire set of scripts.
Infact, if one take a look to mosquitto Debian package, the systemd service files cannot be found.
There is instead teh usual file for the rc (System V):

<b>/etc/init.d/mosquitto</b>

this is the file that is referred using a soft link, in the several directories under /etc/rc.d.
Really, even on Debian Stretch the rc subsystem seems to be there, at least lookning to tits configuration files.

That means systemd is able in some way to interpret the old files: but, as will be shown below, the story is more complex.

So, looking to a live working system, there is really a systemd service file active and running for mosquitto

	pi@raspberrypi:~ $ systemctl list-unit-files --type=service | grep mosqu
	mosquitto.service                            generated

except it has a different status than usual, it is _generated_ instead of __running__ .

therefore the System V files are present but not natively used, rather they are used by systemd 
in order to generate its .service files automatically and on the fly.

<pre>
cat  /run/systemd/generator.late/mosquitto.service 
# Automatically generated by systemd-sysv-generator

[Unit]
Documentation=man:systemd-sysv-generator(8)
SourcePath=/etc/init.d/mosquitto
Description=LSB: mosquitto MQTT v3.1 message broker
Before=multi-user.target
Before=multi-user.target
Before=multi-user.target
Before=graphical.target
After=remote-fs.target

[Service]
Type=forking
Restart=no
TimeoutSec=5min
IgnoreSIGPIPE=no
KillMode=process
GuessMainPID=no
RemainAfterExit=yes
SuccessExitStatus=5 6
ExecStart=<b>/etc/init.d/mosquitto start</b>
ExecStop=/etc/init.d/mosquitto stop
ExecReload=/etc/init.d/mosquitto reload
</pre>


<pre>
pi@raspberrypi:~ $ systemctl status mosquitto
● mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
   Loaded: loaded (/etc/init.d/mosquitto; generated; vendor preset: enabled)
   Active: active (running) since Wed 2018-04-18 21:01:34 CEST; 1 day 1h ago
     Docs: man:systemd-sysv-generator(8)
   CGroup: /system.slice/mosquitto.service
           └─408 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

Apr 18 21:01:33 raspberrypi systemd[1]: Starting LSB: mosquitto MQTT v3.1 message broker...
Apr 18 21:01:34 raspberrypi mosquitto[328]: Starting network daemon:: mosquitto.
Apr 18 21:01:34 raspberrypi systemd[1]: Started LSB: mosquitto MQTT v3.1 message broker.
</pre>

Of course, it is not advisable to make any change to that .service as it would be overwritten on the next run. 

## Links

* [Is there a way to see the execution tree of systemd?](https://serverfault.com/questions/617398/is-there-a-way-to-see-the-execution-tree-of-systemd)
* [systemd.unit — Unit configuration](https://www.freedesktop.org/software/systemd/man/systemd.unit.html)
* [Understanding Systemd Units and Unit Files](https://www.digitalocean.com/community/tutorials/understanding-systemd-units-and-unit-files)
* [Systems tutorial on Linux Magazine](http://www.linux-magazine.com/Issues/2017/200/Tutorials-Systemd)
* [How To Use Systemctl to Manage Systemd Services and Units](https://www.digitalocean.com/community/tutorials/how-to-use-systemctl-to-manage-systemd-services-and-units)
* [System shutdown logic](https://www.freedesktop.org/software/systemd/man/systemd-halt.service.html)
* [Run a service right before starting any of reboot/shutdown/halt/kexec services](https://unix.stackexchange.com/questions/39226/how-to-run-a-script-with-systemd-right-before-shutdown)
* [An Introduction to systemd, Erik Johnson](http://uniforumchicago.org/slides/systemd/An_Introduction_to_systemd.pdf)
* [Demystifying systemd - OHIO LINUX FEST 2015, Scott Seighman, Solutions ArchitectRed Hat](https://doc.lagout.org/operating%20system%20/linux/DemystifyingSystemd-OHLinuxFest15.pdf)
* [Systemd: Service File Examples](https://www.shellhacks.com/systemd-service-file-example/)
* [Introduction to systemctl](https://www.linode.com/docs/quick-answers/linux-essentials/introduction-to-systemctl/)
* [Controlling a Multi-Service Application with systemd](http://alesnosek.com/blog/2016/12/04/controlling-a-multi-service-application-with-systemd/)
* [Grouping systemd services](https://unix.stackexchange.com/questions/228277/grouping-systemd-services)
* [Complex restart of group of units within systemd](https://stackoverflow.com/questions/44152774/complex-restart-of-group-of-units-within-systemd)
* [How to group / compose systemd services?](https://serverfault.com/questions/714142/how-to-group-compose-systemd-services)
* [How to create a virtual systemd service to stop/start several instances together?](https://unix.stackexchange.com/questions/283714/how-to-create-a-virtual-systemd-service-to-stop-start-several-instances-together)
* [Understanding and Using Systemd](https://www.linux.com/learn/understanding-and-using-systemd)
* [Learning systemd](https://ma.ttias.be/learning-systemd/)

## Author
@include='bio_andrea_montefusco'
@include='bio_sergio_tanzilli'