Data Pipeline Technical Lead
R3 Management Services
Job Description
Job Description
Data Pipeline Technical Lead
Lead an engineering team delivering production data pipelines on AWS for IRS data modernization. Set the technical direction for Databricks and Spark, plan the transition from legacy and mainframe sources, and coordinate delivery with federal stakeholders.
Key Responsibilities
- Set architecture and engineering standards for ingestion, transformation, orchestration, data quality, and monitoring across Databricks, Delta Lake, and AWS data services.
- Guide engineers through design and code reviews, resolve technical dependencies, and address pipeline performance and reliability issues.
- Plan migration waves and define reconciliation, cutover, and rollback criteria with source-system owners and downstream data consumers.
- Manage the Agile backlog and release plan in Jira. Coordinate priorities, acceptance criteria, and release readiness with federal stakeholders.
- Partner with deployment and production support teams to document pipeline dependencies, validate operational readiness, and hand off releases.
Required Qualifications
- 12+ years in data engineering or data platform delivery, including 5+ years leading teams working with Spark or Databricks.
- Experience delivering production ETL/ELT pipelines on AWS using Databricks, Delta Lake, Amazon Redshift, Amazon RDS for PostgreSQL, and AWS Database Migration Service (DMS).
- Experience leading a migration or modernization from legacy or mainframe sources to a cloud data platform.
- Experience managing an Agile backlog and releases in Jira for a federal customer.
- Bachelor’s degree in computer science, engineering, or a related field, or equivalent experience.
- Ability to pass an IRS background investigation.
Preferred Qualifications
- Databricks Certified Data Engineer Professional certification. AWS Certified Solutions Architect or AWS Certified Data Engineer certification.
- IRS or Treasury experience, especially with EDP, CADE 2, or Individual Master File (IMF) modernization.
- Experience with Informatica and enterprise business intelligence tools such as Tableau, SAP BusinessObjects, or Power BI.
- ICF is seeking a Technical Product Manager for a federal data modernization program. The role focuses on maintaining the software delivery pipeline, backlog standards, and cross-team coordination in a remote setting with occasional client-site meetings in Washington, DC...PipelineDataRemote job
- ...Group is seeking a hands-on Project Manager/Technical Lead to drive the growth of our Digital Twin... ..., IoT devices and live operational data. Design the data integration, calibration... ...Experience with streaming and IoT data pipelines (e.g., Kafka, MQTT) and time-series data...PipelineDataFor contractors
- ...About Infinitive Infinitive is a data and AI consultancy that helps clients modernize... ...About the Role We are seeking a Technical Lead / Architect to define the technical vision... ...polyglot microservices, streaming pipelines, and storage boundaries. Durable Orchestration...PipelineDataContract workLocal area
- ...systems, and scale innovation through cloud, data, and intelligent engineering. We partner... ...seeking a highly skilled and hands-on Technical Lead to join our application modernization... ...Development CI/CD Implementation Jenkins Pipeline Development and Administration Oracle...PipelineData
- ...Overview Halvik is seeking a Salesforce Technical Lead to join the delivery team supporting the... ...Agentforce, headless architecture, and Data 360. Core Responsibilities... ...Code with Salesforce Extensions, and CI/CD pipelines (GitHub Actions, Copado or equivalent)....PipelineDataPermanent employmentLocal areaFlexible hours
- ...Nightwing provides technically advanced full-spectrum cyber, data operations, systems integration and intelligence... ...commercial markets. The Technical Lead is responsible for providing hands... ...of continuous integration (CI) pipelines and automated testing frameworks....PipelineDataContract workShift work
$134k - $265k
...Summary Salesforce Life Sciences Technical Lead Our Deloitte Sales & Service team empowers... ...service. We are a team of strategists, data scientists, operators, creatives,... ...and 1+ implementation using CI/CD release pipelines· Led 10+ client presentations/workshops...PipelineDataWork experience placementLocal area- ...Description Senior Software Engineer / Technical Lead Location: Herndon, VA (2303 Dulles Station... ...Engineering, Modeling & Simulation, data analytics, cybersecurity, and AI-enabled... ..., Kubernetes, containers, and DevSecOps pipelines. Familiarity with AI/ML...PipelineDataContract work
$130k
...Description Job Description Software Engineer/Technical Lead (Sr.) Herndon, VA Pay From: $130,000 per... ..., containerization, and modern DevSecOps pipelines. Familiarity with AI/ML-enabled mission applications, data engineering, and advanced analytics....PipelineDataFor subcontractor$119.32k - $202.85k
...the status quo.The TeamYou'll lead a delivery team within a cross... ...business analysts, product managers, data analysts, accessibility and... ..., you will provide day-to-day technical leadership while remaining... ...Splunk.Experience improving CI/CD pipelines, automated quality checks,...PipelineDataFull timeContract workWork experience placementWork at officeRemote work$89.6k - $194k
...is seeking a hands-on, forward-thinking Technical Lead and engineering culture champion to drive... ...client-server desktop tools and nightly data synchronization feeds into a modern,... ...deployments.• Database Tuning & Schema Migration Pipelines: Lead the root-cause diagnosis and...PipelineDataWork at officeLocal areaNight shift$131.3k - $237.35k
...seeking a Full Stack Software Developer Team Technical Lead, with strong systems, software, cloud,... ...in building and maintaining CI/CD pipelines using Jenkins, GitHub Actions, Git, Docker... ...CDExperience in AWS App DevelopmentHands on data reporting, through web/html or Power BI...PipelineDataFull time2 days per week$161k - $241k
...purpose.Position OverviewAs a Senior Engineering Lead, you will be a hands-on technical leader within the Platforms & Infrastructure... ...proven experience designing and operating CI/CD pipelines.Working knowledge of cloud data platform technologies — including NoSQL and document...PipelineDataLocal area- ...space systems. The Software Architect / Technical Product Lead will be the technical authority... ...emulation, simulation, digitalengineering, data, and software platforms. This role blends... ...with heterogeneous compute pipelines, GPU kernels, and large-scale simulation...PipelineDataPermanent employmentContract work
$120k - $150k
...experienced Dynamics 365 & Power Platform Technical Lead to own the technical direction of... ...DevSecOps practices using Azure DevOps, CI/CD pipelines, automated testing, and secure... ...and user acceptance testing (UAT), plus data validation. Compliance, Sustainment,...PipelineDataFull timeContract workTemporary workLocal area$51.4 - $66.71 per hour
...Overview: The Cloud Architect is the technical design authority for how the Microsoft Fabric data platform is provisioned,... ...branching and DevSecOps promotion pipelines, build proofs of concept, and... ...area, Cynet Systems Inc. is a leading technology staffing and workforce...PipelineDataTemporary workFor contractorsLocal area- ...applications, multi-agent workflows, and Retrieval-Augmented Generation (RAG) pipelines for enterprise use on Microsoft Azure. You will collaborate with architects, software engineers, data engineers, and business stakeholders to translate requirements into AI-...PipelineData
- ...implementing robust event-driven architectures that enhance observability and enable comprehensive tracking of events across diverse data pipelines. Your work will involve developing modular systems that support seamless integration with new data formats and domains,...PipelineData
- ...with significant experience in scalable data platforms, cloud-based services, and big... ...Elasticsearch. They must have strong, firsthand technical expertise in a variety of data ingestion/... ...software into an AWS-based data ingest pipeline. Write software that interacts with...PipelineData
- ...event-driven and distributed software systems supporting complex data pipelines. This role will focus on building modular architectures that... ...in Computer Science, Engineering, Finance, or a related technical field, or equivalent practical experience. Strong software...PipelineData
$131.3k - $237.35k
The Leidos Health & Civil Sector is seeking a Technical Lead to support the National Institutes of Health (NIH), National Cancer Institute... ...advanced Artificial Intelligence (AI), Machine Learning (ML), and data science capabilities that accelerate cancer research, clinical...DataFull time$156k - $211k
...TimeCountry: United StatesJob Summary The Lead Software Engineer provides technical leadership for the design,... ...third-party services, and SQL Server data stores, ensuring alignment with business... ...integration and delivery (CI/CD) pipelines.Leads proofs of concept to evaluate...PipelineData$124.77k - $171.56k
Do you thrive at the intersection of technical execution and solution design? Are you passionate... ...building Workday integrations, agents, data architectures and Extend applications?... ...! We are looking for a Workday Technical lead who can roll up their sleeves and build,...DataPermanent employmentFull timeWork experience placement$107.9k - $195.05k
...seeking a Software Engineering Lead to support the National... ...Engineering Lead will provide technical leadership for software engineering... ...architects, AI/ML engineers, data engineers, cybersecurity specialists... ...implementing DevSecOps, CI/CD pipelines, configuration management,...PipelineDataFull time- ...community. We partner with our customers to deliver high-impact technical solutions while fostering a culture built on trust, expertise... ...and enable comprehensive event tracking across diverse data pipelines. This role develops modular, scalable solutions that accommodate...PipelineDataFull time
$122.7k - $182.2k
...currently initiating a search for a Lead Professional, Civil... ...Herndon, Virginia office.Provides technical and project leadership to ensure... ...sustainability; Review audit data collected by ADA Quality Assurance... ...-term strategy, growth, and pipeline development of civil...PipelineDataFor subcontractorWork at officeLocal areaFlexible hours$123k - $161k
...IT architecture, large scale data stores, and cloud-based systems... ...the following responsibilities:Lead the design, development, and... ...and contribute to code reviews, technical design sessions, and... ...weekly sprint cycles and CI/CD pipelines.Ability to thrive in a fast-paced...PipelineDataFull timeWork at officeRemote work$160k - $200k
...designing, deploying, and supporting AI/ML solutions in the cloud, including data pipelines, model hosting, and integration with application services. Advising senior business and technical personnel to aid in the delivery of robust, scalable cloud architectures...PipelineDataFull timeRemote workShift work$107.9k - $195.05k
...creation and configuration management. Use data mapping, data mining and data... ...systems. Jenkins, Git, Artifactory, CI/CD pipelines AppDynamics Experience with data migrations... ...Framework) AWS experience (RDS) STIG hardening Scrum master or team lead...PipelineDataFull timeFlexible hours- ...Lead Software Engineer Location: Herndon VA Employment Type... ...Software Engineer to provide technical leadership across multiple... ...design integration and application data layers. ~ Strong experience... ...and maintaining CI/CD pipelines particularly Jenkins. ~ Experience...PipelineDataFull timeContract workRelocationRelocation packageFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Pipeline Technical Lead. Be the first to apply!





