Start here in Data Engineering
The most useful guides in this topic, not just the most recent.
Data Engineering Guides & Tutorials
Build the data backbone — ETL/ELT, warehouses, lakes, Spark, Kafka, Airflow, Databricks and the medallion architecture.
Data Engineering articles
Partitioning and Bucketing in Spark Explained
“Partition” means three different things in Spark: in-memory partitions, directory partitions on disk, and buckets. What each one controls, which setting or method changes it, and how to choose partition and bucket columns without creating millions of small files.
Data Engineering Interview Questions and Answers, Part 2: The Debugging Round
Some data engineering interviews skip theory and hand you a broken query: the numbers are wrong, find out why. This guide reproduces five classic pipeline bugs on real public data, shows the symptom each produces, how to diagnose it quickly and the fix an interviewer expects.