S3 vs EBS vs EFS: AWS Storage Compared

AWS has many storage services, and three of them cover most needs: Amazon S3, Amazon EBS and Amazon EFS. Newcomers often compare them by price per gigabyte, which misses the point. They are three different kinds of storage, object, block and file, and each kind answers a different question. Pick by the kind first; the price follows.

Three kinds of storage

Object storage: S3. Data is stored as objects in buckets, each identified by a key (S3 docs: what is Amazon S3). You do not mount S3 or format it. Programs put and get whole objects over an HTTPS API. There is no disk to size and no capacity to provision.

Block storage: EBS. An EBS volume is a block-level storage device that you attach to an EC2 instance and use as you would a physical hard drive (EBS docs: EBS volumes). The operating system formats it with a file system and reads and writes blocks. It is the disk of a virtual server.

File storage: EFS. EFS is a managed network file system. It is mounted over the NFS protocol and presents directories and files that many machines can use at the same time (EFS docs: what is Amazon EFS).

A rough analogy: EBS is the hard drive inside one computer, EFS is the shared drive on the office network, and S3 is a warehouse where items are stored and fetched by label.

Side by side

S3: the default for data

S3 Standard stores each object across a minimum of three Availability Zones and is designed for 99.999999999 percent durability (S3 docs: data protection). Storage classes and lifecycle rules let cold data move to cheaper tiers. Nearly every AWS analytics service, Athena, Glue, EMR, Redshift, reads from it directly, which is why a data lake on AWS means S3.

Its limits come from being object storage:

• No in-place edits. To change one byte you write the whole object again. Appending to a log file line by line is the wrong pattern.
• Not a file system. Folders are a naming convention inside keys. Tools exist to mount a bucket, and AWS's own Mountpoint client is honest about the gap: it can read and create files but cannot modify existing ones, and does not support file locking (S3 docs: Mountpoint).
• Per-request latency. Each operation is an API call. That is fine for large files and poor for a database's thousands of small random reads and writes.

Amazon S3 explained covers classes and good practice; S3 lifecycle policies covers ageing data out.

EBS: the disk for one server

EBS is for workloads that need a real disk with low latency and frequent small updates. The user guide names the system drive of an instance and storage for a database application. Volume types split into SSD-backed, for transactional workloads, and HDD-backed, for throughput-intensive ones.

What to know:

• Tied to one Availability Zone. The volume and its instance must be in the same zone.
• Usually one instance at a time. Multi-Attach exists only for certain volume and instance types.
• Persistent. A volume persists independently of the instance's running life, so stopping the server does not lose the data.
• Provisioned. You pay for the size you create, used or not. An oversized, half-empty volume is a common waste, and so is a volume left behind after its instance is gone.
• Snapshots. Point-in-time backups persist independently of the volume and can restore a new one.

If you run a database yourself on EC2, or a self-managed Kafka broker, its data sits on EBS.

EFS: a shared folder that scales itself

EFS fits when several machines need the same files through ordinary file operations. It is mounted through mount targets in your VPC and can be accessed concurrently by many clients on instances in multiple Availability Zones (EFS docs: how it works). It is also available to containers on ECS and EKS and to Lambda functions.

What to know:

• Elastic. No size to choose; it grows and shrinks as files are added and removed, and you pay for what is stored.
• Linux-oriented. It speaks NFS; the guide recommends current Linux NFS clients.
• Network latency. Every operation crosses the network, so it is slower per operation than a local EBS disk.
• Typically priced higher per gigabyte than S3. Compare the two pricing pages for your Region. EFS earns its place when applications need a shared file system, not as bulk storage.

Typical uses: a shared directory for a fleet of web or application servers, home directories, configuration or model files read by many containers, the DAGs folder of a self-managed Airflow cluster.

Choosing

Two questions settle most cases. Does the software need a file system or a disk, or can it call an API? And does more than one machine need the data at once? API access and many readers point to S3. A disk for one machine points to EBS. A file system for many points to EFS.

For data engineers

Most data engineering on AWS is S3 work. Pipelines land files in S3, engines read them from S3, warehouses load from S3. EBS matters when you size the disks of an EMR cluster or a self-managed service, where shuffle and temporary data live. EFS appears occasionally for shared code and configuration. Knowing why data belongs in S3, and why a database does not, is the part worth being able to explain; the AWS roadmap for data engineers places these services in a learning order.

Common mistakes

• Treating S3 as a disk: many tiny appends, or renaming "folders" of millions of objects.
• Keeping analytical data on EBS, where only one server can reach it and capacity is paid for in advance.
• Using EFS as cheap bulk storage.
• Forgetting unattached EBS volumes and old snapshots, which continue to be billed.
• Assuming an EBS volume survives a zone failure. It lives in one Availability Zone; snapshots are the protection.

Amazon S3 and data lake storage on AWS are covered in the Data Engineering course.

Sources

• What is Amazon S3? (Amazon S3 User Guide) — AWS Documentation
• Amazon EBS volumes (Amazon EBS User Guide) — AWS Documentation
• What is Amazon Elastic File System? (Amazon EFS User Guide) — AWS Documentation
• How Amazon EFS works (Amazon EFS User Guide) — AWS Documentation
• Data protection in Amazon S3 (Amazon S3 User Guide) — AWS Documentation
• Mount an Amazon S3 bucket as a local file system (Amazon S3 User Guide) — AWS Documentation