One of the frequent complaints I hear from users moving to Podman is about performance. I’ve always found this confusing, as the Podman team has spent significant time and effort on improving our performance, and we are quite confident that our speed is broadly comparable to Docker. Indeed, an academic paper found that Podman was faster for HPC applications than Docker. After some investigation, the cause seems to be that people are not just migrating from Docker to Podman but from root Docker to rootless Podman. This is somewhat of an apples to oranges comparison for reasons I’ll explain next.

Podman’s rootless functionality is a significant advancement for security and one we’re very proud of. The ability to avoid any interaction with the root user is a massive improvement in the security model of containers. However, there are a number of inherent limitations to rootless containers. Without root privileges, certain key elements of containers – namely, networking and storage – must be done in user space instead of kernel space. For storage, we must use a FUSE filesystem (fuse-overlayfs); for networking, a tool that manually reassembles and forwards packets to the container (slirp4netns). Together, these can cause a major performance drop for specific operations in a container: anything that accesses the network, or the disk, will be affected. Anecdotally, I see a 20% decrease in performance for `podman build` with certain types of build (large pulls from the network or decompressing a large archive) when running without root versus running with it. Thus, while the Podman team is very confident in the performance of Podman versus Docker, this is only when comparing root Podman and root Docker or rootless Podman and rootless Docker. To compare rootless Podman to root Docker is not a fair comparison.  That compares the lack of kernel support for networking and storage for rootless containers rather than comparing the tools themselves.

To an extent, this performance decrease found running in rootless is a necessary tradeoff for security; however, that does not mean there is no way around it. On newer kernels (5.11 or above), the requirement for fuse-overlayfs has been removed: the kernel now allows Podman to use the overlayfs filesystem natively as a non-root user, removing the penalty of a FUSE filesystem. You can read more about this major advance in a previous blog. For networking, we are adding support for pasta, an alternative networking tool that promises improved performance. It is also possible to entirely disable network separation to attain native networking performance by using the --net=host option, but this somewhat reduces the security benefits of containerization. Together, these changes should majorly reduce the performance penalty of rootless containers.

Currently, a hybrid alternative does exist in the form of `podman run –userns=auto`, which allows root Podman to create a container with all the performance advantages of root (kernel-native overlayfs and bridge networking) and the runtime security advantages of rootless (the container has no access to the host’s root user).

Going forward, the Podman team will focus on performance for the foreseeable future as we look to improve Podman for edge systems and automotive applications. This is especially true for the rootless container space.  We have already made a number of major improvements in container startup time, with more on the way. With these improvements, we plan to close the gap between root and rootless further. However, even with these improvements, rootless will remain a deliberate tradeoff users will make for improved security versus somewhat reduced performance.