Flight Data EtLT Pipeline
|Data Engineer
Summary
As a Data Engineer, Walid designed and implemented an end-to-end ETLT pipeline, efficiently processing over 7 million records to support critical flight data analysis and ensure data availability.
Highlights
Developed and deployed a scalable end-to-end ETLT pipeline, successfully processing over 7 million records to ensure high data availability and accuracy for critical flight operations.
Orchestrated complex data ingestion and transformation tasks using Apache Airflow and dbt, significantly streamlining data workflows and enhancing data quality for downstream analytics.
Implemented robust data validation checks within the pipeline, improving data reliability and reducing error rates by an estimated 15%.