Lead Data Engineer - Data & Analytics

Salary: $105,000 - 145,000 per year Requirements:

• We need a bachelors degree in Computer Science, Engineering, or a related discipline with 6 years of experience, or an associates degree in a related field with 8 years of experience.
• We expect strong knowledge of professional software engineering practices across the full SDLC, including coding standards, code reviews, source control, build processes, testing, and operations.
• We require deep expertise in data engineering and data pipeline development, with at least 5 years of experience in data engineering, data science, or analytical modeling.
• We look for experience leading Data Engineering teams in a Continuous Integration/Continuous Delivery environment.
• We require experience building resilient data products, test automation suites, unit test coverage, and data quality, monitoring, and observability capabilities.
• We need at least 5 years of experience working with relational and NoSQL databases.
• We expect experience with cloud platforms such as Google Cloud Platform, Azure, or AWS.
• We require experience with CI tools such as Jenkins, GitHub Actions, or Azure Pipelines.
• We look for hands-on experience with advanced data processing technologies such as Apache Spark, Hive, Airflow, Kafka, and Google Cloud Platform Dataflow.
• We value knowledge of big data, statistics, and data aspects of machine learning.
• We require familiarity with Google BigQuery, FHIR APIs, and Vertex AI.
• We look for knowledge of workflow orchestration tools such as Apache Airflow and Google Composer.
• We expect experience using infrastructure-as-code and container technologies such as Kubernetes and Docker in cloud environments.
• Preferred candidates will have advanced Python and SQL skills with production-grade delivery experience.
• Preferred candidates will have advanced experience designing scalable distributed systems using Spark, Flink, Ray, or similar frameworks.
• Preferred candidates will understand cloud-agnostic architecture and modern data platform design.
• Preferred candidates will have experience with open data architecture technologies such as Apache Iceberg, Delta Lake, and Apache Hudi.
• Preferred candidates will have strong expertise in analytical data formats such as Parquet, Avro, and ORC.
• Preferred candidates will have designed data platforms that support analytics, AI/ML, and operational workloads at enterprise scale.
• Preferred candidates will have strong experience with CI/CD, automated testing, Infrastructure as Code, observability, and engineering best practices.
• Preferred candidates will have experience designing systems for scalability, reliability, security, resilience, and long-term maintainability.

• We lead the design, prototyping, and development of data pipeline architecture.
• We drive internal process improvements by automating manual work, improving data delivery, and redesigning infrastructure for better scalability.
• We perform root cause analysis on internal and external processes and data to uncover improvement opportunities and answer business questions.
• We analyze unstructured datasets and apply strong analytical thinking to complex problems.
• We understand the architecture deeply, contribute as a senior individual contributor, and participate in technical discussions as a team player.
• We serve on the Technical Review Board alongside managers and principal engineers.
• We act as a technical liaison between management, software engineers, and principal engineers.
• We collaborate with software engineers to analyze, develop, and test functional requirements.
• We actively design, develop, review, and optimize production-grade data pipelines, data products, and platform capabilities.
• We maintain meaningful contributions to production codebases while setting engineering standards and helping deliver scalable, resilient solutions.
• We mentor and coach engineers.
• We investigate design approaches, prototype new technologies, and evaluate technical feasibility with the team.
• We work in an Agile/SAFe/Scrum environment to deliver high-quality software.
• We establish architectural principles, choose design patterns, and guide team members in applying them effectively.
• We facilitate communication across front-end, back-end, data, and platform engineering teams.
• We serve in a formal engineering lead capacity in our area of expertise.
• We stay current with industry trends and emerging developments.

• AI
• Airflow
• AWS
• Azure
• Big Data
• BigQuery
• CI/CD
• Cloud
• Composer
• Docker
• Flink
• GitHub
• Hive
• Support
• Jenkins
• Kafka
• Kubernetes
• Machine Learning
• NoSQL
• Phoenix
• Python
• SQL
• Security
• Spark
• Git
• JIRA
• Network
• SharePoint

More:

We are Mayo Clinic, a top-ranked care provider recognized by U.S. News & World Report for excellence across more specialties than any other provider. We are committed to putting patients first while also investing in our employees through competitive compensation, comprehensive benefits, and ongoing education and advancement opportunities. Our benefits include medical, dental, vision, pre-tax savings options, and a competitive retirement package. We have employees across major campuses in Phoenix/Scottsdale, Arizona, Jacksonville, Florida, Rochester, Minnesota, throughout Midwestern Mayo Clinic Health System communities, and at international locations. We are an equal opportunity employer and welcome qualified applicants from all backgrounds.

last updated 33 week of 2026