Deconstructing Shellshock To Prepare For the Next One

Yesterday, our Co-Founder and Chief Scientist, Jen Andre, and CEO, Doug Cahill, hosted a live webinar, “Preparing for the Next Shellshock.” Shellshock is the most notable and destructive vulnerability to date, and alongside POODLE and Heartbleed, 2014 has now been deemed the year with the most reported vulnerabilities in history.

Shellshock widely affected Linux distributions, was easy to exploit, and allowed an attacker to gain unauthorized access to a computer system. This left hundreds of thousands of organizations left in the dark, unsure if they were compromised and if so, the extent of the damage.

So how did Shellshock actually happen and how did we succumb to such a vulnerability? At Threat Stack, we felt it was our duty to fully explain how Shellshock came about, and even more important, how to develop the right security monitoring and automation system protections to prevent the next one from devastating the cloud.

We had a great turnout for the webinar yesterday. Here are the top highlights, slides and the live recording:


Highlights from the Webinar

Trust but verify: Even though there may be many eyes on the security of our systems, it doesn’t mean that everyone is at all times looking for the next security vulnerability, such as Shellshock. We can never assume we’re completely safe, so therefore must always be verifying through continuous monitoring.

Why Shellshock was hard to find: This bug has been around for a very long time but was only uncovered this year. When it was uncovered, Security, Ops and Sysadmin experts alike had to be able to answer the question: “Did the particular usage in the application we deployed of the bash shell mean we’re severely vulnerable, or perhaps safe?” and this was not easy to answer by any means, leaving most companies without answers.

Prevention is a myth: You can prepare your systems up to a certain point, but you need to build in monitoring checkpoints to ensure you’re actually protected. On top of that, you can only test for what you know about, but with new classes of bugs, you can’t build a new tool to test for what you don’t know. This is why automation is key, allowing you to test for potential issues right away.

What Shellshock means for the Cloud and Internet of Things: Over 70% of infrastructure-as-a-service today is on Linux, and most users assume their cloud provider handles the security for them. However, in most cases, it’s really the user’s responsibility to make sure software is patched and being monitored for compromises. Even companies that have been using Linux embedded systems for years still have problems getting updates to users and keeping that software up to date. The point here is to always be monitoring and patching!

Implement continuous delivery for security: Continuous delivery is possible through the use of tools and DevOps practices that favor automation, which is very powerful from a security standpoint. Automation tools own “all the things” meaning they can also enable security. They can tell you everything from the time between initial compromise to when a hacker figures out where your data is and how to compromise it. Use this to your advantage.

Move faster! In the cloud, the time from initial compromise to stolen data is much, much quicker. So when designing your continuous deployment infrastructure, keep security top-of-mind as these CD tools can be very powerful security checkpoints. An important statement that Marc Goodman of the FBI made in relation to this was, “We’re about to transition from an Internet the size of a golf ball to an Internet the size of the sun.” We need to be prepared to deal with all possible cloud security issues at all times, and automation is the part that makes this happen.

Preparing for the next Shellshock: We need to be looking at behaviors, not signatures, as indicators of compromise on your systems in order to monitor them effectively. For instance, with Threat Stack, instead of using antiquated signatures that can’t detect new unknown malicious activity, we use an anomaly-based detection that gives much deeper visibility into what happened, where, when, and how. And since Threat Stack is built in the cloud, for the cloud, we fully support auto-scaling and so have added EC2 integration, allowing you to instantly detect which servers are protected, and monitoring new servers as you spin up them up over time.

Watching it in Action

Jen then showed Threat Stack in action by detecting a vulnerability like Shellshock live using the Threat Stack service. Once Jen ran the malicious script, we could instantly see the log activity, individual session details, process network activity, and how to follow users as they pseudo into different accounts to fully understand the scope of a potentially suspicious activity. To complete the investigation, she analyzed the process tree, metadata and the TTY Timeline to understand the entirety of the issue to determine if it was malicious. And sure enough — she discovered the vulnerability within a minute, all in one single interface.

Thank you for those who joined us yesterday. See Threat Stack in action yourself with a 14-day free trial.