Threat Stack Use Cases

Andy Jassy, CEO of Amazon Web Services, Introduces New and Enhanced AWS Services at re:Invent

Las Vegas — Wednesday, November 29, 2017

It was 8:00 a.m. when AWS CEO Andy Jassy took to the stage to offer up the latest AWS news and announcements. And offer up he did. To my recollection, the number of services announced today dwarfed anything unveiled at any previous AWS re:Invent show. (To see the ever-growing list of services debuted this year, head over to the AWS blog.)

The sheer number of new services blew away all expectations. Not only did Amazon announce new compute instances and enhancements to some of their existing services, but the big news was their flurry of announcements about new services that continue down the path of Serverless and Machine Learning.

Here are some of the highlights, along with my points of view from a DevOps perspective.

S3 Select

S3 Select, which was launched in preview today, allows you to use SQL-like queries to retrieve data from S3. S3 was one of the earliest services created by AWS, and was designed to be a way for customers to store large amounts of data. To make it easy for customers to store more data and get it into S3 easier, they announced new features like AWS Snowball and AWS Snowmobile, which let customers move immense amounts of data into the cloud for later retrieval. The downside to storing your data in S3 is that, in order to query the data sets, you would need to have services that could parse that data and likely insert it into a database on RDS or maybe even a search engine like ElasticSearch.

S3 Select now allows you to directly query the data you stored on S3 — retrieving only the specific data you are interested in. Currently S3 Select supports uncompressed CSV and JSON files stored on S3. The biggest use case for S3 Select is for Serverless applications built on AWS Lambda. These services can now retrieve specific data stored in S3 without the need to spend time downloading, parsing, and querying it.

EC2 Bare Metal Instances

In a rare moment where AWS is actually following the market instead of leading by a mile, they announced (in preview) Bare Metal instances with direct access to hardware. You get all the benefits of cloud computing, but with direct access to the underlying hardware. Other cloud providers have been using the concept of Bare Metal cloud as a way to differentiate themselves from AWS, providing customers with the concept of elastic bare metal computing, but now AWS has decided to enter the fray. The biggest benefit of the new Bare Metal instances is that they are integrated into the existing AWS EC2 family of services such as Elastic Load Balancing, Auto Scaling Groups, Auto Recovery, and others. These would make great hosts for CoreOS which would allow you to run plenty of Docker containers for efficient and dense computing.

New EC2 Instance Types

AWS announced the next generation of the M5 Class instances as well as a new H1 instance type. The M5 Class instance is the 5th Generation AWS instance type which has a history that goes back to the original m1.small that was launched back in 2006. Following established tradition, the latest class of instances in this generation provides better performance at a more cost-effective price. They are based on Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz. Most impressively though, is the m5.24xlarge instance type (the largest size in that family), which is the second most powerful instance behind the immense X class instance types based on CPU count. They also support up to 25Gbps of network throughput which can allow customers to consolidate workloads without sacrificing network performance.

The H1 instances are new for AWS this year and provide faster computing for dense storage applications. The D2 class instances allowed customers access to a large amount of locally attached magnetic storage, but for high computing workloads, they didn’t provide enough CPU per TB of data storage. This was an early complaint I had when they launched last year: They seemed underpowered for the amount of storage they were given.

The H1 class instances fix that issue with more vCPU and Memory per TB of local magnetic storage. They are optimized for high throughput and can achieve 1.15GBps of writes when using a 2MB block size. Additionally, the disks are encrypted at rest, which helps customers run sensitive workloads without fear of data leaks.

AWS Fargate

Fargate is by far one of coolest services that AWS announced at re:Invent this year. What Fargate provides is a simple way to launch and run containers without having to think about any of the underlying infrastructure. In the past you would leverage services like ECS or even go down the path of running your own Kubernetes or CoreOS clusters on AWS with a large operational burden that they would provide. Now AWS allows you to treat the underlying container image as the fundamental computing primitive. Similar to EC2 where you make a request and receive a virtual instance, Fargate gives you a container to run your application on. With support for IAM and billed at a per-second granularity, Fargate will allow customers an incredibly easy and cost-effective way to run containers for various computing needs without having to manage all of the other underlying services to support it. And since Fargate is being treated like a first-class citizen on AWS, you can launch containers, adding them to VPCs, incorporating them into your load balancers, and assigning IAM roles to them to allow for secure and safe access to other AWS services.

Amazon Elastic Container Service for Kubernetes (EKS)

Continuing on the container talk, AWS also announced the Amazon Elastic Container Service for Kubernetes. Kubernetes has exploded in popularity over the last few years as companies have been wanting a way to schedule and run containers at scale within their environments. But the operational cost and complexity to running Kubernetes has kept many companies from going down that path. Amazon EKS, is a fully managed service that allows you to leverage the benefits of Kubernetes without having to deal with the operational complexity and challenges that are involved in running a large scale distributed container scheduler. EKS has full support for Kubernetes, which means applications running on any OTHER Kubernetes cluster can also run on the Amazon cluster. This allows companies to standardize their application no matter whether they are running on bare metal in their own data centers, or on another cloud provider. Similar to other AWS services, it is integrated with Elastic Load Balancing, IAM, VPC, and even AWS Private link which can allow customers to run hybrid cloud computing environments.

Final Words . . .

Well, that’s it for now, but stay tuned for updates. As I said earlier, the announcements came fast and furious today, and Amazon once again showed itself to be a powerhouse of cloud innovation.