Senior Data Engineer

We're partnering with a highly respected organization that is making significant investments in its enterprise data and AI capabilities. This is an opportunity for a senior-level engineer to help shape the future of a modern Azure-based data platform while working with cutting-edge technologies including Databricks, Unity Catalog, AI-ready data architectures, governance tooling, and cloud-native engineering practices.

This role is ideal for someone who enjoys building platforms from the ground up, solving complex enterprise data challenges, and driving best practices across data engineering, platform engineering, security, and governance.

What You'll Be Doing

As a Senior Data Platform Engineer, you'll be responsible for the design, operation, optimization, and evolution of a modern enterprise data platform supporting analytics, business intelligence, AI, and machine learning initiatives.

Key responsibilities include:

• Designing and managing scalable Azure and Databricks-based data platforms
• Building and optimizing lakehouse architectures leveraging Azure Data Lake, Delta Lake, Spark, and PySpark
• Creating robust data ingestion, integration, ETL/ELT, and orchestration pipelines
• Managing platform reliability, scalability, observability, and cost optimization
• Implementing governance, lineage, metadata management, and security controls
• Developing AI-ready data foundations to support advanced analytics and machine learning initiatives
• Automating infrastructure deployments and platform management through Infrastructure-as-Code and CI/CD practices
• Providing technical leadership, mentoring engineers, and influencing platform strategy
• Partnering closely with architecture, security, governance, analytics, and business teams

Required Experience

✅ 10+ years of experience in Data Engineering, Data Platform Engineering, Cloud Engineering, or related disciplines

✅ Deep experience with:

• Microsoft Azure
• Databricks
• Spark / PySpark
• Delta Lake
• Python
• SQL
• Azure Data Factory
• Azure Data Lake Storage

✅ Experience building and supporting enterprise-scale cloud data platforms

✅ Strong understanding of:

• AI-focused data engineering
• Data readiness for Machine Learning and Generative AI initiatives
• Data governance, metadata, lineage, and cataloging
• Identity management, RBAC, ABAC, and enterprise security models
• Performance tuning, scalability, and cost optimization

✅ Experience leading projects and providing technical direction to engineering teams

Ideal Candidate Background

We're looking for engineers who have worked in mature, modern technology environments rather than traditional legal organizations.

Strong backgrounds may include:

• Software product companies
• SaaS organizations
• Enterprise technology leaders
• Companies operating large-scale cloud platforms and modern engineering practices

What Will Make You Stand Out?

The strongest candidates will demonstrate:

• Experience designing and launching enterprise data platforms from the ground up
• Expertise in Databricks architecture, optimization, and governance
• Strong Azure cloud engineering knowledge
• Proven success supporting AI and advanced analytics initiatives through scalable data architecture
• A platform engineering mindset focused on reliability, automation, observability, and security