You don’t have to look far to see that data breaches are running rampant among healthcare organizations. In 2015, three of the highest-profile cases – Community Health Systems, Anthem and Premera – resulted in the breach of over 96 million personal health records.
At Threat Stack our engineering and operations teams have embraced the concept of the polyglot data platform, recognizing that no one solution can provide for all of our needs. Those needs include rapid scaling, ideally linearly, to support growing customer demand and the elastic workloads of our new economy customers. We also require different forms of analysis to support stream analysis for our IDS feature set, efficient lookup tables and prematerialized views for our ETDR feature set, and offline analysis for analysis and research.
A core component of our data platform for several years has been Cassandra, which we upgraded to Datastax Enterprise (DSE) through their start up program last year. Originally we were expecting to use it as our single source of truth for all of our time series data, but this turned out to be an anti pattern. Instead we have found it very useful for look up tables and pre-materialized views (more on this later).