IT Career Guides & Free Resources
Expert roadmaps, tutorials, salary guides and interview prep across Data Science, Generative AI, Python, Cloud & Analytics — written by GloryTecks trainers in Hyderabad.
All articles
Metadata-Driven Pipelines in Azure Data Factory
A metadata-driven pipeline reads a list of what to copy from a control table and applies one generic Copy activity to every row. This guide explains the building blocks in Azure Data Factory, shows the pattern working on real tables, and walks through the pipelines and control tables the Copy Data tool generates.
Lookup and Get Metadata Activities in ADF
Lookup and Get Metadata are how an Azure Data Factory pipeline finds things out before it acts. This guide explains what each returns, the shape of their outputs, the documented limits of 5000 rows and 4 MB, and the patterns they enable: a loop driven by a control table, a watermark, a file check and a folder listing.
Building a Metadata Framework for ADF Pipelines
A control table and a ForEach copy many tables. A framework is what makes that safe to operate: configuration you can validate, a log of every load, stages that run in order, and reruns that resume instead of repeating. This guide designs each part for Azure Data Factory, with the restart logic shown working.