SystemD socket activation lesson learned

It’s Friday and I thought I would share something I learned today. Like many things, the documentation is out there but it took a little finesse to put it all together. So here is hoping it helps someone in the future who hits this searching for an answer to a problem.

We want the netavark-dhcp-proxy, written in Rust, application to be able to run in two modes: manual and activated by a systemd socket. Manual means just simply as a command line application. Systemd socket activated basically amounts to systemd “owning” the socket and someone tries to use the socket, it will launch the service. Socket activation is a great approach to being able to have rarely ran services become available on-demand. And if you build in an inactivity timeout for your service, it can then shut itself after meeting whatever conditions you want.

One thing to note about a socket activated service with systemd is that when the service is started, instead of plainly listening to a specific socket file, it will listen on file descriptor (FD) 3.

“… as part of the socket-based activation logic. It returns the number of received file descriptors. If no file descriptors have been received, zero is returned. The first file descriptor may be found at file descriptor number 3…”

sd_listen_fds(3)

To determine if your service was activated by systemd, you check for the presence of the environment variable key LISTEN_FDS. Here is the actual example in Rust from the proxy code where we look for the environment variable and if found, we create a UnixListener from FD 3.

    let uds: UnixListener = match env::var("LISTEN_FDS") {
        Ok(effds) => {
            if effds != "1" {
                error!("Received more than one FD from systemd");
                return Ok(());
            }
            let systemd_socket = unsafe { stdUnixListener::from_raw_fd(3) };
            UnixListener::from_std(systemd_socket)?
        }
        // Use the standard socket approach
        Err(..) => UnixListener::bind(&uds_path)?,
    };

    let uds_stream = UnixListenerStream::new(uds);

While testing however, I noticed that the inactivity timeout to quit the application was not firing. I acquired the PID for the application and strace reveals it is waiting on an accept.

$ sudo strace -p 479503
strace: Process 479503 attached
accept4(3,

I was perplexed. Obviously it was something with how the proxy was being handed the socket but I could not pinpoint the problem. Then my teammate @pholzing pointed out that UnixListener::from_std expects a non-blocking FD which is not the default for from_raw. We just needed to alter the socket we opened from the raw file descriptor to be non-blocking.

     let systemd_socket = unsafe { stdUnixListener::from_raw_fd(3) };
     systemd_socket.set_nonblocking(true)?;

And bingo! Problem solved. The code in its entirety can be seen in my systemd socket activation pull request. I hope this helps someone in the future should you be faced with a similar challenge.

In:

Leave a Reply

Subscribe

Sign up with your email address to receive updates by email from this website.


Categories


Search