Start here in AWS
The most useful guides in this topic, not just the most recent.
AWS Guides & Tutorials
AWS for data engineers — S3, EC2, Lambda, Glue, Redshift, Athena, EMR and the certification roadmap that gets you hired.
AWS articles
Amazon MSK: Managed Kafka on AWS
Amazon MSK runs open-source Apache Kafka for you on AWS. This guide explains what the service manages and what remains your job, Provisioned clusters with Standard or Express brokers versus Serverless, MSK Connect and Replicator, authentication choices, tiered storage, and how to choose between MSK and Kinesis.
Redshift Performance Tuning: Distribution and Sort Keys
Redshift performance tuning starts with two table properties: how rows are distributed across slices and how they are sorted on disk. This guide explains the four distribution styles and the sort key options, simulates block skipping and slice skew on 2.96 million real rows, and shows when AUTO is the right answer.
Redshift Spectrum: Querying S3 from Redshift
Redshift Spectrum lets Amazon Redshift query files in S3 through external tables and join them with tables stored in the warehouse. This guide covers external schemas and the shared catalog, which operations are pushed down, partitions and file formats, cost, and a hot and cold data pattern.
AWS Glue Jobs: PySpark on Serverless Spark
A practical guide to AWS Glue Spark jobs: what a job is, how the script is structured, when to use a DynamicFrame and when a DataFrame, how job bookmarks avoid reprocessing, and how workers and DPUs set the bill. The transform is written as plain PySpark and run locally on 2.96 million rows, then wrapped for Glue.
AWS Interview Questions for Data Engineers, Part 2: The boto3 Coding Round
Many AWS data engineering interviews include a short coding round with boto3. The tasks look simple, but each hides a trap: pagination, retries, duplicates, request limits and key ordering. Here are five such tasks, run against a local AWS emulator, with the trap and the answer that shows experience.
AWS Glue Studio: Visual ETL for Beginners
AWS Glue Studio lets you build a Spark ETL job by drawing it: source, transform and target nodes on a canvas, with Glue generating the script. This beginner's guide explains the editor, the built-in transforms, data previews and what they cost, the generated script, and when a visual job should become code.