Data Engineer (3-6 Years)
Title: Data Engineer - Azure & Databricks
Responsibilities
· Design, develop, and maintain ADF pipelines with parameterization and error handling
· Write and optimize SQL queries for analytical and reporting workloads
· Develop Databricks notebooks using PySpark for ingestion and transformation
· Implement incremental data loads and reprocessing strategies
· Design and manage data structures in ADLS across multiple layers
· Develop production-ready Python code for data engineering workflows
· Implement data quality validations and reconciliation logic
· Investigate and resolve pipeline failures and data issues
· Collaborate with architects, analysts, and business stakeholders
Required Skills
· Strong proficiency in SQL
· Solid hands-on experience with Azure Data Factory
· Strong working knowledge of Databricks and PySpark
· Experience designing and managing data in ADLS
· Proficiency in Python for data engineering use cases
· Understanding of data modeling and analytical concepts