VPNNotify: A VPN Notification bot for Slack

In an earlier post, we talked about how we implemented centralized authentication at Threat Stack. This project initially allowed us to create clearer access control for our servers. A side benefit of this work has allowed us to write tooling around common authentication processes.

One thing we’ve wanted to do is create an alert when folks are using a VPN to connect to one of our environments. In the event of a stolen laptop and stolen credentials, a user could be alerted to someone logging in with their credentials. With OpenVPN, performing actions on a client connect is possible using a client-connect script, so in the tradition of writing small Go applications to improve visibility, we did just that.

For the last few months our Slack bot VPN Notifier has been letting our engineers know when they connect into a Threat Stack environment. We’ve now done the work to open source the tool so that others can use and improve on it. We specifically mention improve, because our tool has limitations: The current version does extremely basic environment checking, and extremely basic alert suppression. Our hope is that we can collaborate with others who want to take this tool the extra mile. Read more “VPNNotify: A VPN Notification bot for Slack”

Authkeys: Making Key-Based LDAP Authentication Faster

Authkeys, Threat Stack’s new open source tool, performs LDAP lookups of SSH keys without the need for using scripts or other interpreted code.

You may recall from an earlier post that we’ve set up centralized authentication here at Threat Stack. Our motivation for doing so centered on the desire to achieve clearer access control for the servers that power our platform. By doing this, we no longer need to use Chef to deploy the majority of users to servers. Rather, we can use an internal application to add, lock, and update users and their associated metadata.

Read more “Authkeys: Making Key-Based LDAP Authentication Faster”

Balancing Security and Your On-Call Rotation Using Deputize

Threat Stack, like many other Software-as-a-Service providers, has an on-call rotation. During any week, two members of our engineering organization are tasked with responding to alerts across the platform they build and maintain. These two engineers are also responsible for a myriad of other services as well that provide support to the infrastructure: services that provide metrics and monitoring, log capture and collection, authentication, etc.

This presents a security issue with regard to access control: should all staff have access to all servers all the time? In early start-up life this is unavoidable. But as an organization matures and grows, it becomes a bigger risk. Administrator and similarly scoped credential theft is a goldmine for attackers, so we wanted to improve our story around internal access control.

Unwrapping who needs access to what is always an evolving task, but we put in the work to figure out who goes where and why, and then created groups to control that access. Since we already use groups as a way to control who can log into specific machines, and we use PagerDuty to assign on-call rotations, it seemed like we could create a tool that would query PagerDuty and update our on-call group. So we did! And as a gift to you, we’ve open sourced it.

Read more “Balancing Security and Your On-Call Rotation Using Deputize”

Reliable UNIX Log Collection in the Cloud

One way organizations can improve their security and operational ability is to collect logs in a central location. Centralized logging allows engineers across the entire organization to have a “common view” of the system under load, and can provide vital shared context when things go wrong.

Over the last few months, we at Threat Stack have been reworking how we handle all aspects of our logging system. This project encompasses everything, from the content of our log data to the infrastructure that collects it. In this post you’ll learn about how our internal applications send log data, where they send it to, and the trade offs we considered in making our collection system reliable. Read more “Reliable UNIX Log Collection in the Cloud”

Securing User Credentials With the YubiKey 4

I’m a big fan of the YubiKey 4.

The YubiKey is a security device that originally outputted a 44-character “one time password” that could be decoded and mathematically verified and used as a second factor for authentication. Over the last few years, improvements to the devices mean that they can also perform other important functions, such as storing:

  • Identity, Signature, and Encryption Certificates
  • U2F data for websites (GitHub and GMail, among others, support this)
  • GPG Keys

If you’re looking to set this up on your own, read on to learn how this extra functionality helps your security game, and how you can configure services to use it. Read more “Securing User Credentials With the YubiKey 4”

Five Lessons We Learned on Our Way to Centralized Authentication

In many startups, centralized authentication is a “future us” problem. Setting up centralized auth is useful for managing your network, but requires time, domain knowledge, and patience to get many of the technical solutions working. Compare this with the ease of user management via configuration management (CM) tools that your DevOps teams are already using — they work well enough (and, did we mention, are already in place?) — so it makes total sense that many organizations “punt” on this issue.

Read more “Five Lessons We Learned on Our Way to Centralized Authentication”

Protecting Sensitive Credentials by Sharing Secrets in the Cloud

In the life of many organizations, developers and operations people need credentials that they can use in case of emergency — when, for example, your external authentication services (either your multifactor service or your internal directory) experience an outage. The existence of these accounts presents a problem, however: one of the best ways for an adversary to ruin your organization is to compromise the login credentials of an account that is on every machine in your cloud.

Read more “Protecting Sensitive Credentials by Sharing Secrets in the Cloud”