Small Details, Big Impact: Improving Configuration Auditing

The Product Team at Threat Stack is always on the lookout for ways — big and small — that we can make the Threat Stack experience smoother and easier for our users. Recently we rolled out a small UI change that makes a big difference in helping you triage your AWS Configuration Auditing results.

Since we released AWS Configuration Auditing at the end of last year, we’ve had a great response to the feature from new and existing customers alike. But as the feedback rolled in, one theme caught our attention: At a glance, users were taking a while to discern where their focus was most needed — in other words, which violations to remediate first. We wanted to learn more.

Getting Into the Details

When you use Configuration Auditing, Threat Stack gets the configuration settings from within your AWS account and evaluates them against security best practices. As of earlier this year, the results you’d get back looked like this:

ConfigAudit-Original_Edited@2x.pngThe appearance of Configuration Auditing when it first launched.

For each rule, there are two key metrics that help you determine its priority for remediation:

  1. Resource Status tells you the percentage of your AWS resources (such as IAM users, EC2 security groups, or S3 bucket policies) that passed evaluation by that rule’s criteria.

  2. Severity tells you how critical the rule is overall, for the security of your environment.

And there’s no silver-bullet algorithm or single right way to prioritize a rule in any given situation. In some cases, a high-severity rule with a 95% pass rate will be most important; in other cases, a medium-severity rule with a 30% pass rate will be most important. A human decision is needed, and our aim is to give you as much information as you need to make that decision in the moment.

But when we gathered more feedback from our customers and internal users, it turned out there was one aspect of our design that wasn’t quite helping:

UX2.pngZoomed-in detail of the results list: Do I need to worry about these or not?

The visual interplay between the adjacent Resource Status and Severity columns was causing three specific problems:

  1. The close proximity of these two metrics was actually making it harder to focus on either one of them and mentally process it.

  2. Our bright Severity color-coding was capturing way too much attention — especially for rules whose Resource Status was “100% passed” (in other words, “everything is fine”).

  3. The relationship between these metrics was unclear: Is this rule high-severity because of  its Resource Status value? Or is it high-severity regardless of  that value?

How could we take the existing information on the page, and present it in a way that draws your attention to the items that really need it?

Our Solution

After a number of iterations toward solving the problems identified above, we released the following visual appearance update:

UX3.pngThe appearance of Configuration Auditing after being updated.

To directly address the central issue, we made three changes:

  1. We moved the Severity information away from the Resource Status column.  Now it’s clearer that the primary way we’re sorting the rules is by Resource Status, from “most violations” to “fewest violations.”

  2. We changed the appearance of Severity. The icons were distracting and didn’t add enough meaning to justify their existence, so we removed them, streamlining the appearance of the Severity indicator down to one small word in all-caps. But even more crucially, we now gray out the severity indicator for any rules whose status is “100% passed,” which keeps the focus on those rules that truly need attention.

  3. We made the Severity indicator a part of the Rule column. Now it’s much clearer that Severity is a property of each rule itself, not a reflection of your results for that rule at any given moment.

While making these UI upgrades, we took the opportunity to make a few additional tweaks:

  • We removed any checkmark icon indicators that did not specifically relate to items that are “100% OK.” Now, if you see a checkmark on something, you know it means you don’t have to worry about it.

  • We standardized on a green checkmark-in-a-circle icon to represent the “100% OK” status, for both individual rules and service groups.

  • To replace the previous green checkmark in the overall “Resources Passed” metric at the top of the page, we added a small donut chart that gives you an immediate visual indicator for how close you are to 100%.

In the weeks since releasing this update, we’ve talked to customers specifically about their Configuration Auditing experience, and have heard positive comments across the board. Of course, we will continue listening, learning, and making changes to make Threat Stack even better.

Summing Up

What’s earth-shattering about this change? Nothing.

When it rolled out to production, many  people probably didn’t notice anything different. But if a small effort on our part can keep our customers on a smooth, obstacle-free path to success, then it has a big impact down the line.

To experience Threat Stack’s AWS Configuration Auditing, including its new Guided Rules Editor, we invite you to sign up for a Free Cloud Security Audit Trial now.