Building Pluggable Authentication Modules (PAM)

Tailoring Linux for HPC Users


| What is PAM?

If you use any well-known Linux distributions like Ubuntu or CentOS, you use Linux PAM. It doesn't matter if you are a normal user or a system administrator. It aids with authentication of users on Linux-based systems. Wikipedia sums up Linux PAM as:

Linux Pluggable Authentication Modules is a suite of libraries that allow a Linux system administrator to configure methods to authenticate users. It provides a flexible and centralized way to switch authentication methods for secured applications by using configuration files instead of changing application code.

Linux PAM is used when users log into a system with ssh, through the login screen, and when sudo-ing. It is also extensible which enables software engineers and system administrators the ability to perform custom actions as a response to certain user actions.

| Why?

We care about Linux PAM because it can help set up users upon logging into the machine. We help manage an HPC for one of our clients and recently extra local and shared storage was added to the cluster. This will enable them to run their research faster and accrue more data. But, as system engineers, we need to ensure that users cannot delete other users' files on these new storage locations.

| Our Options

A very briefly thought about, but legitimate option was to manually create these locations for the users. The downside is that the system authenticates with the organization's Active Directory which means that we wouldn't know when a new user was given permission to the cluster. In addition, we would have additional work for onboarding new users which isn't beneficial for the client nor ourselves.

Another option is to use bash scripts and have them run during shell start. This is a standard practice and oftentimes system engineers will enhance the user experience by modifying system-wide login scripts. But there are many shells and we want to be able to support a wide range of user preferences. Shell scripting, as long as special shell features aren't used, can typically be transferred between different login scripts easily. Our original reason for this was to create user home directories on local and shared storage which requires root privileges at first. This means we would have to use a suid binary which adds additional overhead to ensure inputs to this binary cannot be abused by users or malicious actors. This overhead doesn't add efficiency for our client nor ourselves.

When mentioning user directories, others may think of the PAM module pam_mkhomedir which creates user directories upon initial login. This module is extremely useful and prevalent on Linux distributions, but cannot be configured to support multiple home directories.

After concluding that we needed an automated and user-preference agnostic method to add multiple home directories, following the standard set by Linux distribution developers, we decided creating a PAM module would best complete our client's goals.

| How?

We will provide a high level technical overview of the module we developed. It started by using man pages, ( man pam 3 ), we can find a description of all available interfaces that encompass the user login process. We can look through and for this see that session management is what we want:

The pam_open_session(3) function sets up a user session for a previously successful authenticated user. The session should later be terminated with a call to pam_close_session(3).

The process is to create a shared object that exports these functions, place the shared object in the appropriate directory, and watch PAM load it upon user login. This is how Linux PAM lives up to the name of Pluggable Authentication Modules. For our use case, we decided to use Golang as our language for our PAM integration. This might sound odd if you are familiar with Golang, but we believe it would address any scope creep.

Golang has three major benefits for our use case. First, as more and more systems become OIDC compatible, it is much easier to check JWTs with Golang than it is with C or C++. Secondly, it is easier to send web requests using native libraries in case we want to connect to additional monitoring infrastructure or provide our client with a personalized login process. Lastly, Golang is compiled which can be transferred to other machines without installing additional dependencies.

| pam-usher

So now we know what interfaces we have to implement and what language we are going to use. After developing and testing our system, we now have a module called pam-usher. This module can be found on GitHub. It is configurable with a yaml file and can create multiple home directories and any users that log in.

We will continue to develop pam-usher as we continue supporting our clients to complete their missions. As needs arise, we will enhance this module to provide more personalized support.