I’m in the cloud, where did my network visibility go?

I’ve been on the infrastructure team at Threat Stack for just over one month now. It’s exciting to work here, not just because this is a sharp team with a great attitude, but because we’re exploring the bleeding edge of security practices in the cloud.

During my previous security-related work in the retail industry, I became accustomed to a high degree of visibility into the network traffic amongst my nodes. We leveraged Amazon Web Services (AWS) for central resource management, monitoring, and logging, but most of our application lived in physical point-of-sale endpoints and in-store networks. Managing this Linux-based physical network took a lot of time and attention, and one benefit of that work was that it simplified reviewing or alerting on network traffic at varying levels of detail.

Here at Threat Stack, however, we’re 100% cloud-based and have no need to manage our network infrastructure in that way. AWS takes care of all our heavy lifting and, because our routers, firewalls, and switches are virtual, I can focus my attention on things other than digging into network activity.

“So,” the traditional admin might ask, “If I’m not seeing the same level of detail as I’m used to, how am I to know exactly what is happening on my network?” Certainly not by setting firewall logs to verbose.

One half of the answer is that we need to trust our cloud provider. With AWS, I know I don’t have to worry about cabling, attenuation, VLAN configuration, or my site’s throughput, as all those things are abstracted away from me. Why, then, should I expect or even want packet-level reporting from those effectively invisible devices?

But that’s not a very satisfying response to a cautious admin who wants to be as sure as possible that nothing shady is going on under their purview. I am comfortable profiling my traffic and if any of my nodes did something funny, I would know about it.

And that’s the other half of the answer: To move forward, we need to listen closely to what we still can hear – our nodes. When you’re running on cloud infrastructure, you need to invert your thinking and put your compute nodes at the center of your world — it’s where most of the interesting data is coming from and what you, as an infrastructure engineer, will spend most of your time worrying about.

It’s easy to look at an AWS Linux instance as just a platform to run your application, and modern trends (containerization, anyone?) encourage this. But Linux has always been general-purpose. The tools are there to report on any level of system activity, we just have to use them.

Whether your solution is the Threat Stack service or an in-house solution involving auditd, it’s clear that if we don’t want to be taken by surprise, we need to listen to what we’ve got.

Never stop learning.