Senior Data Pipeline Engineer/Developer
Gene by Gene
Senior Data Pipeline Engineer/Developer We are seeking a Senior Data Pipeline Engineer/Developer to design, build, and own the data pipelines that move genomic and operational data from lab instruments and LIMS through to analytics, products, and clinical reporting. In this senior individual-contributor role you will set technical direction for our data platform, build production pipelines that are reliable and reproducible at scale, and provide architectural and code-level guidance to existing engineering teams. You will own data quality, lineage, and observability end to end, and operate in a regulated environment where reproducibility and auditability are non-negotiable. This is a Python-first, full-stack engineering role. Accountabilities and Responsibilities Pipeline Engineering & Integration Builds and operates production ETL and ELT for high-volume genomic data as well as operational and business data. Integrates data across LIMS, lab instruments, internal applications, and third-party sources through robust, well-tested interfaces. Develops across the stack in Python (data services, internal APIs, and supporting application code) and provides architectural and code-level guidance to engineering teams on data-layer integration. Architecture & Technical Leadership Sets technical direction for batch and streaming data pipelines by evaluating frameworks, orchestration, storage, and processing patterns, making recommendations, and leading adoption. Models and tunes data stores (Microsoft SQL Server and PostgreSQL, plus a cloud warehouse or lake) for performance and scale. Defines and enforces engineering standards for testing, CI/CD, infrastructure as code, code review, and architecture decision records. Data Quality & Pipeline Observability Owns data quality, lineage, and observability, including freshness, completeness, schema-drift detection, cost-per-job, and SLAs. Builds pipelines for reproducibility and audit-readiness, incorporating versioned data and code, lineage, decision logging, access controls, and evidence collection. Partners with security and compliance on data privacy, PII and PHI handling, and regulatory requirements across the data lifecycle. Regulatory Compliance and Security Governance Apply deep understanding of CAP/CLIA, HIPAA, GDPR, and GxP regulations specifically to data pipeline architecture. Oversee protected health information (PHI) handling, data lineage, retention, and comprehensive audit logging. Produce and maintain the critical operational evidence required to carry the data platform successfully through compliance audits. Adhere to strict data-governance controls necessary for securely handling sensitive genomic data across global regions. Enforce least-privilege access and ensure zero data egress to personal or unapproved infrastructure. Utilize exclusively de-identified or synthetic data within development environments. Maintain strict compliance with international data-residency and localization requirements. Position Requirements Skills and Knowledge Strong SQL proficiency on Microsoft SQL Server and PostgreSQL, including schema design, query tuning, and performance troubleshooting. Strong Python proficiency across the stack (data pipelines, backend services, and APIs). This is a Python-first role. Production experience with a workflow orchestrator (Argo Workflows, Step Functions, Prefect, Dagster, Airflow, or similar). Production experience with containers (Docker) and Kubernetes, which the lead orchestrator (Argo Workflows) runs on. Strong testing discipline, including unit, integration, and data-quality or contract tests. Excellent written and verbal communication, including the ability to explain data tradeoffs to technical and compliance stakeholders. Genomics or NGS data formats and handling (FASTQ, BAM/CRAM, VCF). Bioinformatics workflow engines (Nextflow, WDL/Cromwell, or Snakemake). AWS data stack proficiency (S3, Glue, EMR, Batch, Lambda, Redshift) and/or AWS HealthOmics. Distributed processing and streaming frameworks (Spark, Kafka, Kinesis). Data warehouse and transformation tooling (Snowflake, dbt). Infrastructure as code practices (Terraform, CDK, CloudFormation, or Pulumi). Familiarity with .NET (C#) and/or C++ as supporting languages for integrating with existing services. Knowledge of LIMS integration and on-premises plus hybrid data architectures. Experience 8+ years of professional software or data engineering experience. 4+ years building and operating production data pipelines at scale. 3+ years of production cloud experience (AWS preferred). Demonstrable experience working in regulated environments. Compliance is a hard requirement for this role. Experience supporting HIPAA or GxP audits. Education Bachelor's degree in Computer Science, Bioinformatics, or a related field, or equivalent professional experience. An advanced degree in a quantitative field is preferred. Why Join Us At Gene by Gene, you'll join a mission-driven team advancing the science of genetics and discovery. You'll have the opportunity to shape meaningful campaigns, tell compelling brand stories, and collaborate with talented professionals who share your passion for creativity, curiosity, and impact. Gene by Gene
- Senior Data Engineer - Azure / DatabricksLocation: Houston, TX - HybridPosition Type: Full-TimeA well-established Houston-based company... ...will build and maintain scalable Lakehouse solutions, develop ELT/ETL pipelines, integrate data from multiple enterprise systems, and...SeniorPermanent employmentWork at office
- ...its clients in their end-to-end Data & AI transformation by combining... ...types (e.g., H-1B).We are seeking a Senior Data Engineer with strong Energy Trading... ...Engineering & Cloud Solutions:Design, develop, and optimize cloud-based data pipelines, ETL/ELT workflows, and data...SeniorH1bVisa sponsorshipWork visa3 days per week
$104k - $120k
...experience with agile engineering and problem-solving creativity... ...across technology, data sciences, consulting,... ...seeking a hands-on Senior Data Engineer with... ...scalable batch and streaming pipelines, implement Lakehouse... ...Databricks Workflows.Develop scalable ingestion and...Senior- ...seeking a hands-on, enterprise-minded Senior Manager, Data Platform & Engineering to lead the technical foundation... ..., templates, documentation, developer portals, and governed access mechanisms... ..., data reconciliation, deployment pipelines, environment promotion,...SeniorFor contractorsImmediate start
$124k - $280k
...Applied AI Health Services Data Science-Senior Manager you will lead client... ...focused on designing and developing AI-enabled data solutions that... ...draw on data and analytics engineering, artificial intelligence,... ...integration patterns, and data pipelines to support scalable AI...SeniorFull timeH1b- ...Technology Centers (ATCs) are the engine for reinvention in our clients... ...are:An experienced, hands-on Data & AI Engineer with deep hands-... ..., and optimize scalable data pipelines. You design and implement... ...Responsibilities:• Architect, develop, and maintain scalable, low-latency...SeniorWork experience placementLive inWork at officeLocal area3 days per week
- ...Technology Centers (ATCs) is the engine for reinvention in our clients... ...in class, hands -on Snowflake Data & Analytics Engineer with deep... ...Integration, Data Analytics, Pipeline Development, Agentic AI, RAG,... ...Search, and Cortex Analyst.• Develop production-grade Retrieval-Augmented...SeniorWork experience placementLive inWork at officeLocal area3 days per week
- ...encourages diverse perspectives, develops our people, and fosters a... ....Position SummaryThe Senior Platform Engineer will lead the design, operation... ...of the organization’s data and platform services across... ....Builds and supports CI/CD pipelines and Azure DevOps workflows...SeniorFull timeLocal area
- ...invest in technology, we are seeking an experienced Senior Director of Software Engineering & Data to lead our Enterprise Applications organization and... ...ability to maintain critical business systems while developing the capabilities, practices, and solutions that will...SeniorFull time
- ...Senior Pipeline Engineer - Transmission & Distribution Houston, Texas, United States The Senior... ...equipment and construction specifications and data sheets, Requests for Qualifications (... ...experience to complex projects. Find/develop non-standard design solutions....SeniorFor contractors
- ...Our client is seeking a Senior Pipeline Engineer to join their team within the Oil & Gas sector, focusing on transmission and distribution projects... ...technical reports for clients and industry publications Develop and maintain client relationships and identify new business...SeniorPermanent employment
- ...Senior Pipeline Engineer At EXP, we're driven to provide innovative solutions for the world's built and natural environments. As a team of engineers... ...direction to junior/intermediate engineers to complete data analysis, drawing and mapping assignments. Interacts with...SeniorWork at office
- ...Senior Level Pipeline EngineerDo you want to build an impactful career to change the world for the... ...opportunity for a Senior Level Pipeline Engineer with at least 8 years of experience to... ...gas pipeline systems and to further develop a new line of service for existing clients...SeniorContract workFor contractorsWork at officeNight shift
- ...Senior Liquids/Gas Pipeline EngineerDNV – Digital & Data Solutions Pipeline Hydraulics Delivery team is seeking a Senior Liquids/Gas Pipeline Engineer. The role will focus on the implementation and support... ...modeling and simulation projects.Develop, configure, validate, and...SeniorFull timeTemporary workWork experience placementWork at office
$120k - $146.56k
...consulting environmental and engineering firm and one of the top design... ...stand the test of time. From pipelines to power plants, roadways to... ...tremendously and is seeking a Senior Pipeline Engineer to join our... ...equipment specifications and data sheet creation....SeniorFull timeTemporary workPart timeWork at officeLocal areaFlexible hours$166k - $220k
...experience with agile engineering and problem-solving creativity... ...across technology, data sciences, consulting,... ...experienced, hands-on Senior Architect/Data Engineering... ...Delivery Architect and develop batch, streaming, and real-time data pipelines. Build and support data...Senior- ...026-09-14This OpportunityWSP USA Inc is currently seeking a Senior Pipeline Engineerto workout of our office in Chicago, Illinois. The following... ...Kansas City, MO. The candidate in this role will work with engineering team members across the world to deliver pipeline solutions...SeniorFor contractorsWork at officeLocal areaFlexible hours
$100k - $130k
...Job Description Job Description Senior Pipeline Engineer – Transmission & Distribution Location: Houston, TX Employment Type: Full-Time,... ...technical and regulatory compliance. Prepare specifications, data sheets, RFQs, bid summaries, and procurement recommendations...SeniorPermanent employmentFull time- ...the development of a robust data ecosystem that transforms drilling... .... You will work closely with engineers, data scientists, and... ...scalable batch and streaming data pipelines.Strong knowledge of data... ...machine learning, or AI, including developing or productionizing analytical...Senior
- ...communities we serve. Responsibilities The Senior Data Engineer designs, builds, optimizes, and supports scalable... ...focused environment. Qualifications • Design, develop, and optimize scalable ETL/ELT pipelines integrating data from CRM, OSS/BSS, GIS, network monitoring...Senior
- ...OpportunityAccelerate possibilities. The WSP Data Center CoE (Center of Excellence) is... ...? How to secure power and water? How to develop responsibly and become a valued part of... ...a search for a Data Center Mechanical Engineer, Senior Consultant to join our Houston, TX or Dallas...SeniorFor contractorsWork at officeLocal areaFlexible hours
- Overview We are seeking a hands-on Enterprise Data Architect to lead a data modernization... ...and build modern data architecture and pipelines, enabling consistent reporting,... ...Requirements 7+ years in data architecture or data engineering Experience with modern cloud data...Senior
- ...Description We are looking for an experienced Senior Data Engineer to support modernization of the Credit... ...Credit Risk technology platforms. Develop and maintain complex Oracle PL/SQL... ...Unity Catalog, and dbt. Develop data pipelines using Java, Python, Scala, and Apache...Senior
- Data Engineer The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal... ...include: Create and maintain optimal data pipeline architecture Assemble large, complex data sets that...Senior
- Senior Data Engineer - (100% Remote) Fully Remote: Work from anywhere with flexible hours that fit... ...and implement scalable, efficient data pipelines using Azure Data Lake , Databricks , Snowflake , and Synapse Analytics . Develop and optimize workflows with Apache Spark...SeniorRemote workFlexible hours
- Databricks Data Engineer About the Role: We are seeking a highly skilled Databricks Data Engineer to design, build, and optimize our next... ...capabilities of the Databricks platform—from declarative pipelines to advanced governance—to deliver high-quality data products...Senior
- Senior Data Engineer We are seeking a Senior Data Engineer to design, build, and operate cloud-scale data platforms and pipelines that support analytics, machine learning, and data products across... ...principles, and streaming pipelines. Develop software engineering solutions in...Senior
- ...collective progress. We are currently recruiting for a Senior Pipeline Design Engineer to join our office in Houston, TX within the Plant... ...technical bid evaluations. Development and Reporting: Develop personnel skills and flexibility through work assignments...SeniorFull timeContract workTemporary workWork at officeLocal areaFlexible hours
- ...include approximately 50,000 miles of pipelines; 260 million barrels of storage... ...seeking an experienced Python Software Engineer to join the Big Data and Advanced Analytics department. The... ...directly with subject matter experts to develop high quality, reliable, scalable,...Senior
$128.4k - $192.5k
...the CTO, the Advanced Forward Engineering (AFE) team is comprised of... ...products. We are seeking a Data Engineer to join AFE delivery... ...legacy data sources. Ensure data pipelines comply with governance,... ...the future with confidence and develop answers for the most pressing...SeniorSummer holidayWork at officeImmediate startFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Pipeline Engineer/Developer. Be the first to apply!
- junior data engineer remote Houston, TX
- principal data engineer Houston, TX
- data engineer analytics Houston, TX
- data science developer Houston, TX
- data engineer manager Houston, TX
- data center engineer Houston, TX
- big data cloud engineer Houston, TX
- entry level data engineer Houston, TX
- remote data engineer Houston, TX
- senior data center engineer Houston, TX




