Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Data Platform Engineer

Eliassen Group

Senior Data Platform Engineer

Our client is seeking a Senior Data Platform Engineer to own and improve engineering practices supporting enterprise data pipelines. The role focuses on Astronomer and Apache Airflow, automated deployment pipelines, environment management, testing, observability, and operational reliability. The engineer will partner with data engineers, data architects, infrastructure, security, and application teams to ensure pipelines are developed, tested, deployed, monitored, and supported consistently across development, QA, UAT, and production. The ideal candidate brings strong hands-on experience with Astronomer or production Apache Airflow, CI/CD automation, Python, Azure Data Factory, and Microsoft SQL Server.

This is a contract to hire opportunity. Applicants must be willing and able to work on a w2 basis and convert to FTE following contract duration. For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $70.00 to $75.00/hr. w2

Responsibilities:
  • Own the technical implementation and operational support of Astronomer and Apache Airflow.
  • Develop, maintain, test, and troubleshoot Airflow DAGs.
  • Establish standards for DAG structure, naming, dependencies, retries, scheduling, alerting, and error handling.
  • Build and maintain CI/CD pipelines for Airflow DAGs, plugins, Python packages, configuration, and supporting components.
  • Automate deployments across development, QA, UAT, and production environments.
  • Implement automated validation, unit testing, integration testing, and deployment checks for data pipelines.
  • Establish source-control, branching, pull-request, code-review, and release-management practices.
  • Manage Airflow connections, variables, secrets, pools, queues, executors, and environment configuration.
  • Integrate Airflow with Azure Data Factory, Microsoft SQL Server, APIs, file systems, cloud storage, and other platforms.
  • Develop and support Azure Data Factory pipelines, datasets, linked services, triggers, parameters, and integration runtimes.
  • Orchestrate ADF pipelines from Airflow where appropriate.
  • Develop and optimize SQL Server extraction, transformation, and loading processes.
  • Troubleshoot SQL queries, stored procedures, connectivity, performance, and pipeline-related failures.
  • Build reusable operators, hooks, sensors, libraries, and pipeline templates.
  • Implement pipeline monitoring, logging, alerting, dashboards, and operational metrics.
  • Improve pipeline reliability, recoverability, scalability, and performance.
  • Define retry, restart, backfill, catch-up, and failure-recovery procedures.
  • Implement secure secret management and eliminate credentials from code and configuration.
  • Support network connectivity, identity, access control, certificates, and private endpoints with infrastructure and security teams.
  • Maintain technical documentation, deployment procedures, support runbooks, and architectural diagrams.
  • Mentor data engineers on Airflow, Python, testing, CI/CD, and production-support practices.
  • Participate in evaluation and potential adoption of Databricks and modern data-platform technologies.
Experience Requirements:
  • Five or more years in data engineering, platform engineering, DevOps, software engineering, or data-platform operations.
  • Strong production experience with Apache Airflow and hands-on Astronomer or comparable managed Airflow.
  • Strong Python and DAG development skills with CI/CD design and support.
  • Experience deploying pipelines across multiple controlled environments and using Git-based workflows.
  • Strong Azure Data Factory and Microsoft SQL Server experience with advanced SQL troubleshooting.
  • Integration experience with databases, REST APIs, files, cloud storage, and enterprise applications.
  • Automated testing for data pipelines and orchestration code with logging, monitoring, alerting, retry, and failure-recovery patterns.
  • Diagnosis of complex production failures across application, orchestration, database, network, and infrastructure layers.
  • Understanding of secrets management, identity, access control, and secure configuration.
  • Airflow and Astronomer: DAG design, task groups, operators, hooks, sensors, callbacks, dynamic mapping, scheduling and timetables, connections and variables, retries and SLAs, backfills and recovery, plugins and shared packages, DAG testing, Astronomer deployments and CLI, containerized development, upgrades and dependency management, logging and metrics, performance troubleshooting, integration with ADF and SQL Server.
  • CI/CD practices: Python linting and formatting, DAG validation, unit and integration testing, SQL validation, dependency scanning, container image building, security scanning, environment configuration, artifact versioning, automated approvals, promotion across environments, rollback and recovery, audit history, release notes.
  • SQL Server: complex SQL, stored procedures, views, functions, performance tuning, deadlocks, incremental loads, reconciliation queries, authentication and managed identity, collaboration with DBAs, resource efficiency for Airflow and ADF workloads.
  • Azure Data Factory: pipeline development, datasets, linked services, triggers, parameters, integration runtimes, reusable patterns, self-hosted runtimes, troubleshooting, orchestration coordination, platform selection, monitoring and alerting, promotion across environments.
  • Preferred: Azure DevOps Pipelines, GitHub Actions, GitLab CI, Jenkins, Docker, Kubernetes, Airflow on Kubernetes, Terraform, Azure Key Vault, managed identities and service principals, ADLS Gen2, Databricks, Spark, PySpark, Delta Lake, Kafka, dbt, VaultSpeed or Data Vault 2.0, OpenTelemetry, Azure Monitor, Grafana, Prometheus, and experience with manufacturing, ERP, finance, supply-chain, or operational data.
  • Six-month success measures: assess current state, document artifacts and risks, set standards, implement reliable CI/CD and testing, reduce manual deployments, improve observability, resolve recurring failures, document recovery procedures, improve secret management, build templates and libraries, train engineers, and propose future-state architecture.
Education Requirements:
  • Bachelor's degree in computer science, data engineering, software engineering, information systems, or a related discipline is preferred. Equivalent professional experience will be considered.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Platform Engineer in United States vacancy
  • THE GLOBAL LEADER IN DATA & ANALYTICS RECRUITMENTHarnham Search and Selection Company Number: 05723485Harnham Search and Selection is a registered company in England and Wales. Reg no. 05723485Harnham Europe Limited Company Number: 09956940Harnham GmbH HRB: 196954Harnham... 
    Senior
    Work at office

    Harnham

    Los Angeles, CA
    5 days ago
  • $60 - $65 per hour

    DescriptionKforce has a client in Kansas City, MO that is looking for a Senior Data Platform Engineer with strong experience building and evolving modern data- and AI-enabled platforms. This is a hands-on, senior individual contributor role focused on designing, developing... 
    Senior
    Work experience placement

    KForce

    Kansas City, MO
    5 days ago
  • $150k - $215k

     ...empowers members to perform at a higher level through a deeper understanding of their bodies and daily lives.We are seeking a Senior Data Platform Engineer to build and evolve the foundational infrastructure that powers our data platform, lakehouse, and real-time data systems... 
    Senior
    Full time
    Work at office
    Relocation

    WHOOP

    Boston, MA
    3 days ago
  •  ...Our strategy includes building teams across our global network to pioneer and shape the future of PCI.Summary:The Senior Data Engineer (Databricks Platform) is a senior individual contributor responsible for the hands-on design, development, optimization, and operational... 
    Senior
    Full time

    PCI Pharma Services

    Rockford, IL
    4 days ago
  • $100k - $145k

     ...resilience required in rapidly expanding industries such as data centers, healthcare, pharmaceuticals, advanced...  ...Harm policyRole Summary Johnson Controls Inc. are seeking a Senior Data Platform Engineer to design and operate a secure, scalable data platform that... 
    Senior
    Full time

    Johnson Controls

    Milwaukee, WI
    2 days ago
  •  ...NimbleRx in Redwood City is seeking a Senior Data Engineer to own and evolve the data platform crucial for driving business growth and operational efficiency. As part of a fast-growing healthtech startup, you will build and optimize data pipelines, partner with teams... 
    Senior

    NimbleRx

    Redwood City, CA
    1 day ago
  • $145k - $190k

     ...that enables organizations to share and analyze sensitive data at speed without compromising security or control. VIA’s decentralized...  ...An impressive mission requires an equally impressive Senior Data Platform Engineer. As a Senior Data Platform Engineer at VIA, you'll... 
    Senior
    Full time
    Summer work
    Remote work
    Work from home
    Flexible hours

    VIA

    Somerville, MA
    2 days ago
  • $114k - $143k

    Senior Data Platform Engineer/SDE Hybrid role (3 days/week in office) at our Burlington, Renton, Spokane, Vancouver, Portland, Medford, Salt Lake City, Boise, Lewiston, or Fargo offices.Candidates must reside within commutable distance of that location or be willing to... 
    Senior
    Full time
    Contract work
    Work at office
    Immediate start
    Work from home
    Relocation
    Flexible hours
    3 days per week

    Cambia Health Solutions

    Lewiston, ID
    3 days ago
  • $165k - $242k

     ...create the most powerful end-to-end platform to develop, deploy, and iterate AI faster...  ...has operated a growing footprint of data centers covering every region of the...  ...all. About the role: As a Senior Data Platform Engineer, you will: Partner with other... 
    Senior
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    Weights & Biases

    Remote
    2 days ago
  •  ...Virta Health is seeking a senior backend/data platform engineer to design and own our data contracts, processing semantics, and end-to-end observability across partner integrations. You will lead a data-centric squad focused on reliability, scalability, and AI-assisted... 
    Senior
    Remote work

    Virta Health

    Denver, CO
    3 days ago
  •  ...A dynamic tech firm in Concord, NH is seeking a Big Data Infrastructure DevOps / Support Administrator. This role emphasizes automation and support for a data analytics platform. Requirements include 5+ years in development or systems administration, experience with Hadoop... 
    Senior

    Right Talent Right Now

    Concord, NH
    1 day ago
  •  ...A technology solutions firm in Dallas is seeking a Big Data Infrastructure DevOps/Support Administrator. This role involves supporting and developing the Enterprise Infrastructure Big Data Service using Java, Python, and Hadoop. Ideal candidates will have a strong background... 
    Senior

    Right Talent Right Now

    Dallas, TX
    1 day ago
  • $166k - $220k

     ...operating system that turns thousands of data streams into a realtime, 3D command...  ...) threats. Working across product, engineering, sales, logistics, operations, and mission...  ....ABOUT THE JOBWe are seeking a Senior Mission Data Platform Engineer to design, build, and operate... 
    Senior
    Full time
    Work experience placement
    Immediate start
    Worldwide

    Anduril Industries

    Irvine, CA
    5 days ago
  • $110k - $140k

    A renewable energy tech company in North Carolina is seeking a Machine Learning Engineer to build data pipelines and deliver innovative solutions for solar panel optimization. The ideal candidate has a strong programming background, experience with data services, and familiarity... 
    Senior
    Full time

    Sunergi Inc

    Durham, NC
    4 days ago
  •  ...A leading technology solutions provider is seeking a Big Data Infrastructure DevOps/Support Administrator. This role is critical for designing and supporting the innovative data analytics platform used by data scientists. Candidates should have a strong background in software... 
    Senior

    Right Talent Right Now

    Denton, TX
    1 day ago
  • A leading data solutions company is seeking a candidate to join their Managed Services team. This remote position focuses on maintaining...  .... Responsibilities include leading cloud-native data platform implementations, optimizing performance, and ensuring robust security... 
    Senior
    Remote work

    Jobot

    United States
    3 days ago
  •  ...without the need for current or future sponsorship.Senior Engineer will be a key member of the Azure Data as a Service (DaaS) team responsible for designing,...  ...implementing, supporting, and optimizing Azure-based data platforms and services. The role will initially focus on... 
    Senior
    Internship
    Monday to Friday

    Navy Federal Credit Union

    Vienna, VA
    3 days ago
  • A premier cloud services provider is seeking a candidate for a remote position focused on leading cloud-native data platform implementations. The ideal candidate has a strong background in AWS, Snowflake, and Python, with experience in managing enterprise data infrastructures... 
    Senior
    Casual work
    Remote work

    Jobot

    New York, NY
    1 day ago
  •  ...Senior Data Platform Engineer Onsite Phoenix, AZ Job Description We are hiring a Senior Data Platform Engineer for an onsite role in Phoenix, AZ. This role requires strong hands-on experience building and operating data pipelines on Google Cloud Platform.... 
    Senior

    EITACIES

    Austin, TX
    2 days ago
  • $112.8k - $153.7k

     ...network of smart and supportive people, willing to listen to your ideas. This is a senior engineering role. You will architect, operate, and continuously improve cloud data and analytics platforms (e.g., Microsoft Fabric, Snowflake, Databricks) and own BI delivery... 
    Senior
    Full time
    Contract work
    Local area
    Flexible hours

    Armanino

    Los Angeles, CA
    7 days ago
  •  ...A leading data solutions firm is seeking a Big Data Infrastructure DevOps / Support Administrator to manage the development and support...  .... This role involves building a general purpose data analytics platform, developing features, and providing ongoing support. Candidates... 
    Senior

    Right Talent Right Now

    Nashua, NH
    2 days ago
  •  ...As a Senior Data Platform Engineer, the full-time role focuses on ensuring the reliability, automation, and operability of the database and datastore fleet, collaborating closely with engineering teams to build self-service, automated foundations for data management in... 
    Senior
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    4 days ago
  •  ...Renterra, a Series A–backed SaaS company in Chicago, is seeking a senior engineer to own data systems that enable AI-powered product experiences. The role focuses on data architecture, modeling, and production systems, with hybrid in-office collaboration and flexibility... 
    Senior
    Work at office
    Remote work

    Renterra

    Chicago, IL
    2 days ago
  • $112.71k - $183.14k

     ...layer that connects machines, data, and people to make jobsites...  ...world data daily. These data and platform capabilities are enabling the...  ...to create a foundation where engineers like you can build and deploy...  ...Caterpillar. Job SummaryAs a Senior Data Engineer, you will... 
    Senior
    Full time
    Part time
    Work at office
    Relocation
    Visa sponsorship
    Relocation package
    Flexible hours

    Caterpillar

    Chicago, IL
    4 days ago
  • $181.2k - $317.1k

     ...Company Description It all started when engineer Fred Luddy wrote code that...  ...business reinvention. Our ServiceNow AI platform brings together any AI, any data, and any workflow— helping 85% of...  ...We are looking for a seasoned IC5 Senior Staff Engineer with deep expertise... 
    Senior
    Permanent employment
    Full time
    Work experience placement
    Work at office
    Immediate start
    Remote work
    Flexible hours
    2 days per week

    ServiceNow

    San Diego, CA
    2 days ago
  • $58 per hour

     ...Position Title: Senior Data Platform Engineer Location: Remote, VA Clearance Requirements: None (Active clearance preferred) Pay Rate: $58.00/hour Position Description: Join a mission-critical project supporting federal background investigation... 
    Senior
    For contractors
    Remote work

    Seneca

    United States
    3 days ago
  • $201.3k - $352.3k

     ...Company Description It all started when engineer Fred Luddy wrote code that...  ...business reinvention. Our ServiceNow AI platform brings together any AI, any data, and any workflow— helping 85% of...  ...impactful experiences. About the Role — Senior Staff Data Platform Software... 
    Senior
    Full time
    Work experience placement
    Work at office
    Immediate start
    Remote work
    Flexible hours
    2 days per week

    ServiceNow

    Santa Clara, CA
    2 days ago
  • $205k - $215k

     ...~5 years of experience as a software engineer ~ Bachelor's degree in a technical discipline...  ...responsible for creating and maintaining data workflows and automation pipelines using...  ...collaborate closely with data engineers, platform teams, and stakeholders to ensure... 
    Senior
    16 hours
    Immediate start
    Remote work

    Shield Consulting Solutions

    United States
    2 days ago
  • $130k - $170k

     ...States of America)Please review the following job description:Senior Engineer/Platform Leader accountable for designing, building, and operating...  ...prompt management, retrieval and knowledge-grounding patterns, data access patterns, CI/CD automation, evaluation/testing, and... 
    Senior
    Full time
    Part time
    Shift work
    Day shift

    Truist

    Richmond, VA
    1 day ago
  •  ...Senior Data Platform Engineer This job is onsite located in Minnetonka, MN. Please do not apply unless you are within 45 miles from our office. Thank you! A job at RAZR is like no other. We are on a mission to Be Relevant - make a meaningful difference in the lives... 
    Senior
    Work at office
    Day shift

    RAZR Marketing

    Minnetonka, MN
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Data Platform Engineer. Be the first to apply!