Data Engineering Manager

Data Engineering Manager

The Applied Intelligence Data Engineering team is seeking a Data Engineering Manager to lead the design, delivery, and operational excellence of large-scale batch and real-time data platforms. This role will oversee high-performance data engineering teams that power analytics, APIs, AI workflows, and mission-critical data services across the organization. You will guide the design and execution of distributed data systems. This includes ETL and ELT pipelines. It also includes streaming platforms, warehouse ecosystems, and cloud-native infrastructure. As a manager, you will shape technical direction, develop engineering talent, and ensure production-grade reliability, scalability, and operational rigor across data platforms. This role requires expertise in modern data architectures and distributed systems. You should also have knowledge of cloud-native engineering. It is essential to have proven experience in leading high-performing engineering teams.

Key Responsibilities:

• Lead & Scale Data Engineering Teams
• Manage and develop a team of data engineers responsible for batch and streaming data systems.
• Drive technical execution across ingestion, transformation, modeling, and serving layers.
• Establish engineering standards, code quality practices, and architectural review processes.
• Mentor engineers in distributed systems design, performance optimization, and production reliability.
• Architect Scalable Data Platforms
• Ensure scalable data modeling standards and efficient query performance across analytical workloads.
• Guide decisions about architecture. This includes orchestration and schema evolution. It also covers partitioning strategies and storage optimization.
• Partner with streaming engineers to align batch and real-time data patterns into cohesive platform designs.
• Cross-Functional Collaboration
• Partner closely with Data Product Management to align roadmap priorities, SLAs, and platform KPIs.
• Collaborate with software engineers to integrate streaming systems, APIs, and microservices into the broader data ecosystem.
• Clearly communicate architectural tradeoffs to stakeholders. Share delivery risks and operational constraints as well.
• Technical Leadership
• Lead architectural reviews and drive long-term platform strategy.
• Foster a culture of ownership, documentation, and continuous improvement.

Required Skills & Experience:

• Modern Data Architecture
• Strong expertise in data lakes, warehouses, and lakehouse architectures.
• Deep understanding of ETL/ELT frameworks, orchestration platforms, and distributed data processing systems.
• Solid understanding of Kafka-based architectures and event-driven data patterns.
• Team Development & Coaching
• Proven experience hiring, onboarding, and retaining high-performing engineers across varying seniority levels.
• Commitment to building an inclusive, collaborative, and accountable team culture.
• Execution & Delivery Leadership
• Strong sprint planning, capacity modeling, and roadmap sequencing skills.
• Ability to manage competing priorities across reliability, feature delivery, and technical debt reduction.
• Experience establishing engineering KPIs such as reliability, latency, throughput, and deployment frequency.

Qualifications:

• Proven experience leading engineering teams delivering large-scale production data platforms.
• Strong foundation in distributed systems engineering and cloud-native architecture.
• Great communication skills.
• Ability to collaborate with different teams.
• Self-motivated.
• Focus on quality.
• A commitment to engineering excellence and operational discipline is essential.