Azure Data Engineer

Tailored Job Description – Azure Data Engineer

Data Modelling & Architecture

• Strong understanding of semantic models used in enterprise reporting and analytics.
• Expertise in dimensional modelling, including star and snowflake schema design.
• Practical experience implementing Slowly Changing Dimensions (SCD) Type 1 and Type 2.
• Ability to translate business requirements into scalable data models.

Advanced Spark SQL

• Proficiency in Spark SQL, including complex joins, aggregations, and transformations.
• Strong command over window functions, ranking functions, and analytical queries.
• Ability to troubleshoot performance issues and optimize Spark SQL queries.

PySpark Development

• Working knowledge of PySpark for data ingestion, transformation, and workflow orchestration.
• Ability to write and understand basic pyspark functions
• Basic understanding of Spark architecture, including executors, partitions, and caching.

Data Engineering Skills

• Experience building ETL/ELT pipelines using Azure Databricks and load to Synapse.
• Familiarity with Delta Lake, data versioning, and ACID transactions.

Independence & Collaboration

• Ability to work independently with minimal oversight.
• Strong communication skills to collaborate with cross-functional teams and business stakeholders.
• Proactive in identifying issues and proposing data-driven solutions.