Identity management is a difficult problem in the cloud, especially when it comes to sharing user accounts — an all too familiar (and problematic) practice today. Sharing accounts is very common on EC2, in particular, because EC2 instances come with a standard set of user accounts that a team can begin using immediately. Although it’s possible to create more user accounts, doing so is a resource-intensive task that is not a top priority for most operations personnel — and as a result, teams often end up sharing the default accounts.
So when it comes to spinning up new EC2 instances, doing ops-related work, or even gaining access to the broader AWS environment, any number of users may be logging in under a single shared account with standard access rights. While this practice is common, it can lead to major issues down the road when something goes wrong and you need to find out precisely which user was responsible.
Shared Accounts = Identity Management Mayhem
To uphold security best practices and compliance regulations, many companies create user access policies. These are designed to ensure that the right users have the right access across the organization’s cloud environment. Organizations can attempt to enforce this by systematically monitoring for specific user logins and actions.
However, with shared user accounts becoming an increasingly common practice in the cloud, upholding user access policies has become next to impossible.
And the consequences can be severe. If, for example, a command was issued by a shared user account and led to a serious breach, it would be impossible to know who the culprit was. That’s because once a user logs into a shared account, all the operating system can verify is which account the user is logged into — not who the individual user is.
Until now there has been no easy way to deal with the problem, except by creating more individual user accounts or strictly enforcing user access policies.
Faced with this,Threat Stack conducted an investigation into whether there is, in fact, a way to identify a specific user behind a shared account. The short answer is “yes, there is” although the original process we discovered was manual and somewhat cumbersome. So we continued our efforts and came up with a way to use Threat Stack’s unique capabilities in order to automate the process.
Investigating How SSHD Identifies Individual Users
Let’s say there are three users: Amit, Ram, and Venkat. Each user has their own unique id_rsa.pub keys, and the keys are copied to each of the cloud servers using configuration management scripts. These users then use their own unique keys to log into the servers.
Does SSHD behave differently for each user who is logging in?
Since Threat Stack collects all system calls, we needed to answer this question: Does SSHD behave differently for each user who is logging in?
The first task was to find out what the SSHD path is for checking the keys.
SSHD obviously has to open the authorized_keys file to check whether there are matching keys for the user logging in. The typical configuration for the authorized_keys file is to store all the keys in one authorized_keys file in the EC2 user home directory. For us to make SSHD act differently for each user, we knew it would be useful if we could make each of the authorized_keys go into different files and make SSHD open different files for different users. After a bit of digging, we discovered that there is, in fact, a way to configure SSHD to look into multiple files. So for the three users, Amit, Ram, and Venkat, we created three different authorized_keys files in the /etc/ssh/authorized_keys directory and made the configuration change in /etc/ssh/sshd.conf to check the directory for keys:
Does SSHD behave in a deterministic way when a user logs in?
The second task was to confirm whether SSHD behaves in a deterministic way when a user logs in.
After a few tests, we found a pattern. When Venkat logs in, SSHD checks Amit’s file and Venkat’s file and it stops. Venkat Amit logs in, SSHD checks Amit’s file and stops.
Bingo! SSHD stopped opening files after it found a match of the key. The SSHD log below confirms this file behavior.
Okay, so it’s obviously possible to crack the user identification case manually. But it’s an extremely resource intensive job and simply too time-consuming to be of any value.
So how can we get around the time-consuming and tedious process of digging through all these logs when something does go wrong?
Enter Threat Stack’s unique File Integrity Monitoring (FIM) capability. Threat Stack’s FIM not only alerts on the modification of files, but also alerts on file opens. Let’s see how it can be used to automate the user identification solution described above.
Automating EC2 User Identification With Threat Stack’s FIM
As a security company, Threat Stack is deeply concerned about any and all security issues, which is why we mandate that everyone on our team using EC2 have a unique login. But we also understand that, in reality, many organizations don’t (and likely won’t) do this.
So, knowing just how critical it is to be able to quickly pinpoint individual user activity on EC2, we wanted to help our customers manage this problem even if they use shared logins — which is why we built this functionality directly into Threat Stack.
Because of the unique way Threat Stack handles FIM, it can automatically discover who executed a command, even on a shared account. Our FIM doesn’t rely on file hashes to detect file opens; instead, it tracks host-based commands and file open activity, two approaches that are unique to Threat Stack. With this information, it is possible to accurately identify which individuals did what on a shared account.
How it Works
1. Add a File Tracking Rule
In Threat Stack’s file integrity rules, users can add a rule to /etc/ssh/authorized_keys to track file opens. Instead of having all the keys in one authorized file, this approach requires multiple authorized files for each user in a directory (a best practice we encourage anyways).
2. Examine Alerts and Time Stamps
Once this rule is set up in Threat Stack, the only thing that users have to do is take a look at the automated alerts that Threat Stack already sends in order to see which key is being used, for which user, and at what time.
We are able to determine this by collecting information directly from the host. Essentially, when someone logs into a shared user account, behind the scenes, their OS opens up specific files depending on who the user is. And, even if there is no direct indicator in the OS (like logs), we can still detect the individual user by tracking user commands and file opens. This way, Threat Stack can determine the exact user who executed an unauthorized, malicious, or harmful command.
The following screenshot shows that the most recent file alert determines the actual user who is logged in. It’s really as simple as that.
Although the practice of sharing logins is not ideal, it is common with most companies that use EC2. While companies can often get away with this unscathed, it can clearly cause severe problems when something does go awry.
However, by using Threat Stack’s unique approach to FIM, even companies that share logins can apply a continuous monitoring approach to identity management in the cloud. So even with shared accounts, organizations can rest assured that they will still be able to determine quickly and easily who did what in the cloud.
To find out more about Threat Stack’s cloud security and compliance capabilities, set up a call for a quick Threat Stack demo.