Systemd tips and tricks

[securitybreach]

Since systemd has replaced sysvinit on most distros and because of the amount of uncertainty out there, I figured it would be nice to start a systemd thread that explains the basics and gives some pointers on how to use systemd. The following is from the Archlinux wiki but everything is the same on any distro that uses systemd. Please try to stay on topic and I would appreciate it if you kept the anti-systemd comments away as this is meant to be an informative and helpful guide to systemd.

 

The edits are mine and of course $ denotes normal user and # denotes root user in the following commands.

 

From the project web page:

 

systemd is a system and service manager for Linux, compatible with SysV and LSB init scripts. systemd provides aggressive parallelization capabilities, uses socket and D-Bus activation for starting services, offers on-demand starting of daemons, keeps track of processes using Linux control groups, supports snapshotting and restoring of the system state, maintains mount and automount points and implements an elaborate transactional dependency-based service control logic.

 

Basic systemctl usage

 

The main command used to introspect and control systemd is systemctl. Some of its uses are examining the system state and managing the system and services. See man 1 systemctl for more details.

 

Analyzing the system state

 

List running units:

$ systemctl or

$ systemctl list-units

 

List failed units:

$ systemctl --failed

 

The available unit files can be seen in /usr/lib/systemd/system/ and /etc/systemd/system/ (the latter takes precedence). You can see a list of the installed unit files with:

$ systemctl list-unit-files

 

Using units

 

Units can be, for example, services (.service), mount points (.mount), devices (.device) or sockets (.socket).

 

When using systemctl, you generally have to specify the complete name of the unit file, including its suffix, for example sshd.socket. There are however a few short forms when specifying the unit in the following systemctl commands:

 

If you do not specify the suffix, systemctl will assume .service. For example, netcfg and netcfg.service are equivalent.

 

Mount points will automatically be translated into the appropriate .mount unit. For example, specifying /home is equivalent to home.mount.

 

Similar to mount points, devices are automatically translated into the appropriate .device unit, therefore specifying /dev/sda2 is equivalent to dev-sda2.device.

 

See man systemd.unit for details.

 

Activate a unit immediately:

# systemctl start unit

 

Deactivate a unit immediately:

# systemctl stop unit

 

Restart a unit:

# systemctl restart unit

 

Ask a unit to reload its configuration:

# systemctl reload unit

 

Show the status of a unit, including whether it is running or not:

$ systemctl status unit

 

Check whether a unit is already enabled or not:

$ systemctl is-enabled unit

 

Enable a unit to be started on bootup:

# systemctl enable unit

 

Disable a unit to not start during bootup:

# systemctl disable unit

 

Show the manual page associated with a unit (this has to be supported by the unit file):

$ systemctl help unit

 

Reload systemd, scanning for new or changed units:

# systemctl daemon-reload

 

.................

 

 

Journal

 

systemd has its own logging system called the journal; therefore, running a syslog daemon is no longer required. To read the log, use:

# journalctl

 

In Arch Linux, the directory /var/log/journal/ is a part of the systemd package, and the journal (when Storage= is set to auto in /etc/systemd/journald.conf) will write to /var/log/journal/. If you or some program delete that directory, systemd will not recreate it automatically; however, it will be recreated during the next update of the systemd package. Until then, logs will be written to /run/systemd/journal, and logs will be lost on reboot.

 

Tip: If /var/log/journal/ resides in a btrfs file system, you should consider disabling Copy-on-Write for the directory. See the main article for details: Btrfs#Copy-On-Write (CoW).

 

Filtering output

journalctl allows you to filter the output by specific fields. Be aware that if there are many messages to display or filtering of large time span has to be done, the output of this command can be delayed for quite some time.

 

Tip: While the journal is stored in a binary format, the content of stored messages is not modified. This means it is viewable with strings, for example for recovery in an environment which does not have systemd installed. Example command:

$ strings /mnt/arch/var/log/journal/af4967d77fba44c6b093d0e9862f6ddd/system.journal | grep message

 

Examples:

 

Show all messages from this boot:

# journalctl -b

 

However, often one is interested in messages not from the current, but from the previous boot (e.g. if an unrecoverable system crash happened). This is possible through optional offset parameter of the -b flag: journalctl -b -0 shows messages from the current boot, journalctl -b -1 from the previous boot, journalctl -b -2 from the second previous and so on. See man 1 journalctl for full description, the semantics is much more powerful.

 

Follow new messages:

# journalctl -f

 

Show all messages by a specific executable:

# journalctl /usr/lib/systemd/systemd

 

Show all messages by a specific process:

# journalctl _PID=1

 

Show all messages by a specific unit:

# journalctl -u netcfg

 

Show kernel ring buffer:

# journalctl _TRANSPORT=kernel...........

 

 

https://wiki.archlin...dex.php/systemd

 

It is also very easy to write your own systemd service/daemon by stringing along commands in order. For instance, here is my custom static IP networking service using the systemd networking commands:

# /etc/systemd/system/network.service
[unit]
Description=Wired Static IP Connectivity
Wants=network.target
Before=network.target

[service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=/etc/conf.d/network
ExecStart=/sbin/ip link set dev enp3s0 up
ExecStart=/sbin/ip addr add 192.168.1.2/24 broadcast 192.168.1.255 dev enp3s0
ExecStart=/sbin/ip route add default via 192.168.1.1

ExecStop=/sbin/ip addr flush dev enp3s0
ExecStop=/sbin/ip link set dev enp3s0 down

[install]
WantedBy=multi-user.target

 

Systemd can be as easy or as advanced as you want. I have been using it for over 2 years and I still do not understand all of the tools/features. If anyone has any additional tips or tricks, feel free to post.

 

Thanks

[saturnian]

Good info, thanks! But, I'll admit, I hope I never need to delve too deeply into any of this! Haven't need to so far.

[securitybreach]

Well all I posted above was how to start, stop, restart and autoload daemons(services). I also posted on how to check if there is a problem with a service, nothing more really.

[Guest LilBambi]

Nice to know that these things are not often needed.

[securitybreach]

Another System versus Sysinit infographic:

 

[Guest LilBambi]

Thanks, good info!

[securitybreach]

Not a problem

[abarbarian]

What’s New in systemd, 2015 Edition

 

 

 

A packed room at Fosdem for Lennart Poettering's talk on systemd, the 2015 edition.

systemd is now a core component of most major distributions. In this talk I want to give an overview over everything new in the systemd project over the last year, and what to expect over the next year.

fosdem talk description

There were no slides. None whatsoever. No presentation. He just talked.

 

[abarbarian]

Systemd cheat sheet  Excellent resource

 

systemd-analyze - how to speed up boot time

 

Graphical interfaces for systemd

Quote

Systemd System Manager [4], designed for GTK work interfaces, comes with a very simple interface, which therefore also offers far fewer configuration and information options than Kcmsystemd. After installing the systemd-ui package, the systemadm entry is waiting for the user in the menu structure.

 

The last link is why I posted. There are several different systemd gui programs with different capabilities. The one I tried out was systemadm which comes as part of the systemd-ui program. This is the simplest offering and you can not do a great deal with it but it does give a lot of information on units and services in a neat way. Also it is in the Arch Community repo, the KDE offering is in the AUR and was last updated in 2019, the Cockpit offering is also in the Arch Community repo and is probably the most fully featured but is probably overkill for most ordinary users.

 

 

 

Although systemadm only has basic functionality there is one very useful feature. It has the ability to take a snapshot of your systemd set up which you can reload after fiddling around and borking a working set up.