Lead Data Engineer

Role Description

We're looking for a Lead Data Engineer to own the reliability and operational excellence of our data platform. You'll work hands-on across Snowflake, Airflow, AWS, Python, and SQL, building and operating systems that support analytics, machine learning, and business decisions. This is a technical leadership role, but it is also deeply hands-on. You'll write code, debug pipelines, tune performance, and operate production systems. You’ll also set standards, lead incidents, drive cost optimization, and remove technical blockers across teams. The expectation is simple: you do the work, and you set up the bar for how the work gets done.

Responsibilities

• Build, maintain, and operate data pipelines and curated data products across Snowflake, Airflow (MWAA), AWS, Python, and SQL.
• Optimize Snowflake architecture at scale.
• Design warehouse strategies, implement clustering and materialized views, enforce resource monitors, control costs, and tune performance across large datasets.
• Implement observability and data quality controls.
• Build monitoring for freshness, volume, schema, distribution, and lineage. Define data quality SLOs and ensure teams can see issues before users do.
• Support and operate production data systems.
• Troubleshoot failures, respond to alerts, debug pipeline issues, and improve system behavior based on real incidents.
• Own SLAs, uptime, and service commitments for data platforms and critical services.
• Monitor performance, track reliability metrics, and make sure availability and freshness targets are met.
• Lead major incident responses and postmortems.
• Coordinate resolution during critical outages, drive root cause analysis, and implement corrective actions that prevent recurrence.
• Architect data platforms across dev, test, and production.
• Design for disaster recovery, upgrades without downtime, automation that recovers from failure, and availability that supports business continuity.
• Define and enforce data platform standards.
• Establish orchestration patterns, DAG anti-patterns, deployment practices, observability standards, data quality patterns, and operational runbooks used across the organization.
• Lead cost optimization programs.
• Identify savings across Snowflake (compute/storage) and AWS resources, forecast capacity needs, and balance cost against reliability and performance goals.
• Lead AI and LLM adoption and governance.
• Build and validate AI-assisted pipelines, define approved use cases and guardrails, and ensure AI-generated output meets reliability and quality expectations.
• Make architectural decisions with an organization-wide impact.
• Evaluate build versus buy, set migration strategies for legacy systems, define data contracts and API standards, and balance innovation with operational stability.
• Lead technical planning and remove blockers.
• Coordinate with Cloud, Security, Analytics, Data Science, and Product teams to resolve dependencies and keep delivery moving.
• Prevent knowledge silos.
• Write documentation, maintain runbooks, record architectural decisions, and create shared practices that let engineers operate independently and onboard quickly.
• Design and enforce data governance controls for regulated environments.
• Implement PII handling, access controls, audit logging, data retention policies, and compliance validation for SOC 2, GDPR, HIPAA, or similar frameworks.

Qualifications

• Deep hands-on experience with Snowflake, Airflow (MWAA), AWS, Python, and SQL in production environments.
• Strong ability to design, build, debug, and operate data pipelines at scale.
• Experience owning production systems with uptime and reliability expectations.
• Proven ability to lead incident response and drive postmortems that result in real system improvements.
• Solid data architecture and system design skills, including disaster recovery, high availability, and upgrades without downtime.
• Experience implementing observability and data quality controls across complex data systems.
• Demonstrated success defining and enforcing engineering standards adopted by other teams.
• Strong cost management experience, including running optimization efforts and forecasting capacity.
• Experience leading technical initiatives across teams without direct authority.
• Experience implementing governance and compliance controls in regulated or high-risk environments.
• Ability to mentor through standards, examples, and shared practices rather than formal people management.
• 8+ years of data engineering experience, with several years owning critical platforms or services.

Preferred Qualifications

• Experience leading AI and LLM adoption in data platforms, including validation of AI-generated pipelines and guardrail design.
• Advanced Airflow experience, including reusable DAG patterns, scheduler tuning, and failure prevention.
• Experience with dbt at scale, including testing standards, macros, and deployment practices.
• Experience with OpenLineage, schemachange, or similar tools for lineage and schema management.
• Experience leading platform migrations or modernization efforts with minimal disruption.
• Experience streaming or near-real-time data systems such as Kafka, Kinesis, or Flink.
• Experience running resilience testing such as failure injection or disaster recovery drills.
• Cloud certifications such as AWS Certified Solutions Architect, AWS Data Analytics Specialty, or Snowflake SnowPro.
• Bachelor's or master's degree in computer science, Data Engineering, or equivalent practical experience.

Benefits

• Flexible Paid Time Off Program
• Medical, Dental, Vision, Life Insurance, Disability, and other voluntary coverages
• 401k program with company match, starting on the first of the month following 30 days of employment

Base Salary

$125,000 - $150,000 USD