(Note: The full version of this post is on the SysAdvent blog. Below is a summary.)
It’s that time of year — again. All we really want is to be throwing back eggnog, “ooh-ing” and “aah-ing” at our fake fireplaces, and catching a funny holiday movie. But alas, we can’t because a certain entertainment company was just held hostage by a security breach. And not just any security breach, but one corporate America has never seen before. Bah humbug.
As I explained on the SysAvent blog, it’s an interesting time to be in security, especially since Sony breach has just topped the worst-of-the-worst scenarios that us tinfoil-hat, paranoid security people have been ranting about all along: one bad breach can be business shattering
But first, let’s step back and look at the topic of this blog post: The 12 days of SecDevOps. Besides the fact that it’s a ridiculous title (I’m 90% sure my Ops Director chose it specifically as a troll for me — thanks, Pete), it underlines an important concept. Whether ‘Security’ is in your job title or not, Operations is increasingly becoming the front-line for implementing security defenses.
Given that reality, the fact that security breaches are NOT subsiding, and that most of us don’t have insanely large security budgets, I thought I’d give 12 practical, high-impact tips that small companies should be doing to clean up their security posture.
Day 1: Fear, Loathing, Risk Assessment and Hipsters
Risk assessment simply means weighing the probability of bad things happening against the cost to mitigate the risk of that bad thing happening. Using that to make good security decisions as you make day-to-day architecture and ops choices involves:
risk = (threat) x (probability) x (business impact)*
*whoever told you there would be no math lied to you!
You may not be aware of it, but as an ops person, you are likely doing risk assessment already, except more likely around things like uptime and reliability.
Day 2: Shared Secrets: Figure Them Out Now
There are 3 things in life that are inevitable: death, taxes, and the fact that a sales guy left to his own devices will always put all of his passwords in a plain text file.
The lesson is this: Password management isn’t something that just the technical team decides and manages for itself. We should be advocating organization-wide education on managing credentials, because guess what? Access to Salesforce, Gmail, and all other SaaS services with sensitive business data are being used by people who are not engineers.
The solution? Start implementing password management from the outset, as it’s best to figure this out on Day 1 rather than 200 employees in.
Day 3: Shared Secrets for Infrastructure, Too
When it comes to infrastructure secrets, there are extra concerns because, in most cases, systems needs to be able to access these secrets in a non-interactive, automated way.
If all of your infrastructure passwords start unencrypted somewhere in a git repo, You Are Going To Have A Bad Time. Noah has a good article on various options for managing shared secrets in your infrastructure.
Day 4: Config Management On All Of The Things (So You Aren’t Sweating from Shell Shocks)
This should be obvious to everyone who drinks from the DevOps Koolaid, but CM has done beautiful things for patch management. Making the process easy for devs to get access to the infrastructure they need (while giving you the ability to manage systems) is key. Do this right away.
Day 5: Secure Your Development Environments (Because No One Else Will)
Left to their own devices, dev environments tend to veer towards chaotic. This isn’t just because developers are lazy (as a developer, I mean this in the nicest possible way) but because of the nature of the prototyping and testing process. Put some bounds around the chaos early on, and this will make it easy to mature the security controls as the product and organization mature.
Day 6: 2-Factor All Of The (Important) Things
Require 2-factor wherever and whenever you can. Easy as that — seriously.
Day 7: Encrypt Your Emails (And Other Communications)
It’s annoying to set up, but guess what? Hackers just *love* to post juicy stuff on pastebin. This is especially important to drill into your executive team because they usually have more sensitive emails, and as such, are particularly susceptible to phishing style-attacks. And with the recent Sony email leaks, you now have more leverage.
Day 8: Security Monitoring: Start Small, Plan Big
Instrumenting your infrastructure for security monitoring from Day 1 puts you in a good position to later on start sophisticated reporting and intrusion detection on that data.
Remember that risk assessment list you made? Identify what you are most afraid of (e.g. your PHP CMS that has hundreds of vulnerabilities reported per year) and tackle monitoring for those items first.
Day 9: Code/Design Reviews
Although there has been a lot of advancement in static and dynamic source code analysis tools (which you can integrate right into your CI process), a good-old fashion code review by a human being goes a long way.
Day 10: Test Your Users
Phish yourself regularly. It’s really easy to do and can be illuminating to the rest of your business. You can use open source tools, or pay to have someone do this for you.
Day 11: Make an Incident Response Plan Now
See something odd in your logs? Figure out a plan for escalating possible critical security issues and have an out-of-band channel for communicating details in case your normal network goes the way of Sony and is totally compromised or just not working.
Day 12: Don’t be the Security ‘A**hole’
You can be the security champion without being a blocker. In fact, that’s the only way to be effective. If a user is coming to you and saying ‘this is really really annoying, I don’t want to do it’ – listen to them. Too many security personnel disregard the usability issue of security controls for the sake of security theater, which unsurprisingly leads to abandonment, cynicism, and apathy when it comes to real security concerns.
Bringing it all Together
DevOps is really a philosophy more so than a job title or a set of tools. It’s the concept of using modern tools and processes to facilitate collaboration between the engineers who deliver the code and those who must maintain it.
The best security cultures are not prescriptive, they are collaborative. That’s why it’s no longer acceptable to throw ‘security over the wall’ and expect your users and ops people to just do what you say.
Bonus: Zane Lackey has a great talk on building a modern security engineering organization that expounds many of these ideas, and more.