Senior AI Data Engineer

Senior AI Data Engineer (Cloud)

Location: Chicago, IL

Schedule: Full-time, 40 hours/week

Type: Contract

Overview

We are looking for a Senior AI Data Engineer to support a large-scale Data Engineering team focused on building scalable data and AI capabilities. This role will work closely with Data Product Management and Data Science teams to design, build, and operationalize data ingestion, transformation, and Generative AI pipelines using Snowflake, AWS, and Python.

The ideal candidate brings a strong traditional Data Engineering foundation along with hands-on experience developing and deploying production Generative AI solutions. This is a hands-on engineering role requiring deep Snowflake expertise, strong AWS and Python skills, and experience taking AI/LLM workflows into production.

Responsibilities

• Design and build scalable data ingestion pipelines from multiple source systems into Snowflake using AWS services and Python.
• Develop and optimize SQL-based transformation pipelines for cleansing, normalization, deduplication, and processing of free-form text.
• Build Generative AI workflows for classification, extraction, and structured data generation using Snowflake Cortex AI or comparable production LLM platforms.
• Develop prompt versioning, response parsing, validation, and fallback logic for production AI workflows.
• Build and deploy Snowflake Native Apps, including application packaging, permissions, versioning, upgrades, and release workflows.
• Integrate Snowpark-based logic and CI/CD processes to support repeatable application delivery.
• Define and track data and model quality KPIs, including precision, recall, coverage, drift, and error rates.
• Implement ML/LLM observability and monitoring using Arize or comparable platforms.
• Build structured extraction pipelines capable of processing free-text data at scale.
• Support production deployments, operational readiness, automation, documentation, runbooks, and handoff to internal teams.
• Partner with Data Science, Product, and technical stakeholders throughout design and delivery.

Required Qualifications

• 8+ years of overall technology experience, including 5+ years of Data Engineering experience.
• 2+ years of hands-on experience developing and deploying Generative AI applications.
• Strong hands-on experience with Snowflake , including advanced SQL, Tasks/Streams, stored procedures, and performance tuning.
• Experience with Snowflake Cortex AI or a comparable production LLM platform.
• Experience developing Snowflake Native Apps using the Snowflake Native App Framework.
• Strong Python development experience, including production-grade coding, testing, and packaging.
• Strong AWS experience, including S3, Lambda, Glue, Step Functions, DynamoDB, EC2, CloudFormation, and Kinesis.
• Experience with Snowpark using Python or Scala.
• Experience with CI/CD tools such as GitHub Actions and/or Jenkins.
• Infrastructure-as-Code experience with Terraform and/or CloudFormation.
• Experience building structured extraction pipelines from free-text inputs, including parsing, validation, deduplication, and quality controls.
• Experience with ML/LLM observability platforms such as Arize.
• Strong understanding of AI/LLM evaluation, error analysis, quality measurement, and drift monitoring.
• Ability to work independently and deliver against defined milestones.
• Strong communication skills with both technical and non-technical stakeholders.

Preferred Qualifications

• Experience within hospitality, customer profile, personalization, or preference-management environments.
• Experience building human-in-the-loop workflows and taxonomy-driven classification systems.
• Understanding of data privacy and compliance requirements involving customer data.
• Experience with Snowflake governance and security, including RBAC, masking policies, row-access policies, and tagging.
• Experience with automated regression testing for prompts and LLM outputs.
• Experience with Snowpark Container Services or containerized AI/ML services operating close to the data.
• AWS and/or Snowflake certifications are a plus.

Education

Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field, or equivalent practical experience.