Lead Data Engineer

Role Description

We are seeking a Lead Data Engineer with strong experience in AI/ML data pipelines, feature engineering, and scalable data architectures. This role will lead the design, development, and optimization of data platforms that power advanced analytics and machine learning applications across the organization.

• Lead the architecture and development of scalable, secure data pipelines supporting AI/ML workloads.
• Own end to end data engineering processes: ingestion, transformation, storage, quality, and monitoring.
• Collaborate with data scientists and ML engineers on model features, training pipelines, and deployment.
• Drive best practices in data modeling, orchestration, versioning, and performance optimization.
• Mentor and guide junior engineers; contribute to technical roadmaps and solution patterns.
• Ensure data governance, lineage, and compliance standards are met across platforms.
• Support real time and batch processing frameworks in production environments.

Qualifications

• Bachelor's degree or higher in computer science (completed and verified prior to start) from an accredited institution.
• Seven (7) or more years of data engineering experience, including leading technical initiatives.

• Strong expertise in Python, SQL, and distributed data systems (e.g., Spark, Databricks, Synapse).
• Experience building AI/ML ready data pipelines, including feature stores and model training data flows.
• Hands on experience with cloud platforms (Azure preferred — Data Lake, Data Factory, Databricks, Cosmos DB).
• Strong understanding of ML concepts, lifecycle, and MLOps practices.
• Proven experience with workflow orchestration (Airflow, Data Factory, Synapse Pipelines, etc.).
• Strong communication and ability to translate business needs into technical solutions.
• Experience with streaming platforms (Kafka, Event Hub).
• Familiarity with vector databases, embeddings, or LLM oriented data pipelines.
• Background in DevOps, CI/CD, or infrastructure as code.
• Experience designing data solutions for AI applications or intelligent platforms.

Requirements

• Work location: Remote in the US.
• Travel: May include up to 5% primarily domestic travel.
• Relocation: Not authorized.
• Must be legally authorized to work in country of employment without sponsorship for employment visa status (e.g., H1B status).

Benefits

• 3M offers many programs to help you live your best life – both physically and financially.
• Competitive pay and benefits, regularly benchmarked with other companies that are comparable in size and scope.
• The expected compensation range for this position is $188,251 - $230,084, which includes base pay plus variable incentive pay, if eligible.
• This position may be eligible for a range of benefits (e.g., Medical, Dental & Vision, Health Savings Accounts, Health Care & Dependent Care Flexible Spending Accounts, Disability Benefits, Life Insurance, Voluntary Benefits, Paid Absences and Retirement Benefits, etc.).