Amazon ECS supports a native integration with Amazon EBS volumes for data-intensive workloads

Today we are announcing that Amazon Elastic Container Service (Amazon ECS) supports an integration pinch Amazon Elastic Block Store (Amazon EBS), making it easier to tally a wider scope of information processing workloads. You tin proviso Amazon EBS retention for your ECS tasks moving connected AWS Fargate and Amazon Elastic Compute Cloud (Amazon EC2) without needing to negociate retention aliases compute.

Many organizations take to deploy their applications arsenic containerized packages, and pinch nan preamble of Amazon ECS integration pinch Amazon EBS, organizations tin now tally much types of workloads than before.

You tin tally information workloads requiring retention that supports precocious transaction volumes and throughput, specified arsenic extract, transform, and load (ETL) jobs for large data, which request to fetch existing data, execute processing, and shop this processed information for downstream use. Because nan retention lifecycle is afloat managed by Amazon ECS, you don’t request to build immoderate further scaffolding to negociate infrastructure updates, and arsenic a result, your information processing workloads are now much resilient while simultaneously requiring little effort to manage.

Now you tin take from a assortment of retention options for your containerized applications moving connected Amazon ECS:

• Your Fargate tasks get 20 GiB of ephemeral storage by default. For applications that request further retention abstraction to download ample instrumentality images aliases for scratch work, you tin configure up to 200 GiB of ephemeral retention for your Fargate tasks.
• For applications that span galore tasks that request concurrent entree to a shared dataset, you tin configure Amazon ECS to equine nan Amazon Elastic File System (Amazon EFS) record strategy to your ECS tasks moving connected some EC2 and Fargate. Common examples of specified workloads see web applications specified arsenic contented guidance systems, soul DevOps tools, and instrumentality learning (ML) frameworks. Amazon EFS is designed to beryllium disposable crossed a Region and tin beryllium simultaneously attached to galore tasks.
• For applications that request high-performance, low-cost retention that does not request to beryllium shared crossed tasks, you tin configure Amazon ECS to proviso and connect Amazon EBS retention to your tasks moving connected some Amazon EC2 and Fargate. Amazon EBS is designed to supply artifact retention pinch debased latency and precocious capacity wrong an Availability Zone.

To study more, spot Using information volumes successful Amazon ECS tasks and persistent retention champion practices successful nan AWS documentation.

Getting started pinch EBS measurement integration to your ECS tasks
You tin configure nan measurement equine constituent for your instrumentality successful nan task meaning and walk Amazon EBS retention requirements for your Amazon ECS task astatine runtime. For astir usage cases, you tin get started by simply providing nan size of nan measurement needed for nan task. Optionally, you tin configure each EBS measurement attributes and nan record strategy you want nan measurement formatted with.

1. Create a task definition
Go to nan Amazon ECS console, navigate to Task definitions, and take Create caller task definition.

In nan Storage section, take Configure astatine deployment to group EBS measurement arsenic a caller configuration type. You tin proviso and connect 1 measurement per task for Linux record systems.

When you take Configure astatine task meaning creation, you tin configure existing retention options specified arsenic hindrance mounts, Docker volumes, EFS volumes, Amazon FSx for Windows File Server volumes, aliases Fargate ephemeral storage.

Now you tin prime a instrumentality successful nan task definition, nan root EBS volume, and supply a equine way wherever nan measurement will beryllium mounted successful nan task.

You tin besides usage $aws ecs register-task-definition --cli-input-json file://example.json bid statement to registry a task meaning to adhd an EBS volume. The pursuing snippet is simply a sample, and task definitions are saved successful JSON format.

{ "family": "nginx" ... "containerDefinitions": [ { ... "mountPoints": [ "containerPath": "/foo", "sourceVolume": "new-ebs-volume" ], "name": "nginx", "image": "nginx" } ], "volumes": [ { "name": "/foo", "configuredAtRuntime": true } ] }

2. Deploy and tally your task pinch EBS volume
Go to your ECS cluster and take Run caller task. Note that you tin prime nan compute options, nan motorboat type, and your task definition.

Note: While this illustration goes done deploying a standalone task pinch an attached EBS volume, you tin besides configure a caller aliases existing ECS work to usage EBS volumes pinch nan desired configuration.

You person a caller Volume conception wherever you tin configure nan further storage. The measurement name, type, and equine points are those that you defined successful your task definition. Choose your EBS measurement types, sizes (GiB), IOPS, and nan desired throughput.

You cannot connect an existing EBS measurement to an ECS task. But if you want to create a measurement from an existing snapshot, you person nan action to take your snapshot ID. If you want to create a caller volume, past you tin time off this section empty. You tin take nan record strategy type: ext3, ext4, aliases xfs record systems connected Linux.

By default, erstwhile a task is terminated, Amazon ECS deletes nan attached volume. If you request nan information successful nan EBS measurement to beryllium retained aft nan task exits, uncheck Delete connected termination. Also, you request to create an AWS Identity and Access Management (IAM) domiciled for measurement guidance that contains nan applicable permissions to let Amazon ECS to make API calls connected your behalf. For much accusation connected this policy, spot infrastructure role successful nan AWS documentation.

You tin besides configure encryption by default connected your EBS volumes utilizing either Amazon managed keys and customer managed keys. To study much astir nan options, spot our Amazon EBS encryption successful nan AWS documentation.

After configuring each task settings, take Create to commencement your task.

3. Deploy and tally your task pinch EBS volume
Once your task has started, you tin spot nan measurement accusation connected nan task specifications page. Choose a task and prime nan Volumes tab to find your created EBS measurement details.

Your squad tin shape nan improvement and operations of EBS volumes much efficiently. For example, exertion developers tin configure nan way wherever your exertion expects retention to beryllium disposable successful nan task definition, and DevOps engineers tin configure nan existent EBS measurement attributes astatine runtime erstwhile nan exertion is deployed.

This allows DevOps engineers to deploy nan aforesaid task meaning to different environments pinch differing EBS measurement configurations, for example, gp3 volumes successful nan improvement environments and io2 volumes successful production.

Now available
Amazon ECS integration pinch Amazon EBS is disposable successful 9 AWS Regions: US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), Europe (Ireland), and Europe (Stockholm). You only salary for what you use, including EBS volumes and snapshots. To study more, spot nan Amazon EBS pricing page and Amazon EBS volumes successful ECS successful nan AWS documentation.

Give it a effort now and nonstop feedback to our nationalist roadmap, AWS re:Post for Amazon ECS, aliases done your accustomed AWS Support contacts.

— Channy

P.S. Special acknowledgment to Maish Saidel-Keesing, a elder endeavor developer advocator astatine AWS for his publication successful penning this blog post.

A correction was made connected January 12, 2024: An earlier type of this station misstated: I changed 1) from “either ext3 aliases ext4” to “ext3, ext4, aliases xfs”, 2) from “check Delete connected termination” to “uncheck Delete connected termination”, 3) from “configure encryption”, “by default configure encryption”, and 4) from “task meaning specifications page” to “task specifications page”.