Data Engineering (DE) Lead
Full-time
American IT Systems
Data Engineering (DE) Lead
Location: Rochester, Minnesota(Open for Remote).
| Role Family | Data Engineering |
| Experience | 8+ years |
| Platform Focus | GCP DataFlow |
| Role Description |
Own data-engineering delivery on GCP/DataFlow, including ingestion, transformation, standardisation, reconciliation, publication and operational engineering standards.
| Key Responsibilities |
Define reusable DataFlow/GCP ingestion and transformation patterns.
- Implement source ingestion from Epic, ELMS, Oracle, terminology and other approved sources.
- Build raw/source-preserving, standardised/cleansed and curated/mastered processing layers.
- Implement standardisation, validation, transformation and reference-data enrichment.
- Build reconciliation controls between source, MDM/RDM and downstream publication.
- Build publication pipelines to GCP data products, APIs, events and tables/files.
- Implement replay, restart, idempotency, audit and monitoring; optimise performance and cost.
- Establish CI/CD, code quality and production support procedures.
| Primary Deliverables |
- Data engineering architecture / design
- Ingestion and transformation pipelines
- Reconciliation framework
- Publication pipelines
- Operational monitoring and runbooks
- CI/CD and engineering standards
| Required Experience & Skills |
- 8+ years in data engineering, hands-on with GCP and DataFlow (or equivalent).
- Large-scale batch, incremental and event-driven pipeline experience.
- Experience feeding and reconciling MDM/RDM data.
- Strong on operationalisation: monitoring, replay, CI/CD.
| Good to Have |
- BigQuery, Pub/Sub and Dataflow depth.
- Healthcare data engineering.
- Integration with Informatica MDM loads.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Engineering (DE) Lead in Rochester, MN vacancy
- Develops and deploys data pipelines, integrations and transformations to support analytics and machine learning applications and solutions... .... A Bachelor's degree in a relevant field such as engineering, mathematics, computer science, information technology, health...SuggestedFlexible hours
- The Principal Data Engineer serves as a hands-on technical authority responsible for defining and implementing enterprise-scale data architecture and engineering strategies while actively contributing to solution design, development, optimization, and technical delivery...SuggestedRemote workFlexible hours
- ...Mayo Clinic is seeking an experienced Data/Software Engineer to lead design, development, and delivery of scalable data pipelines and platform capabilities. The role involves mentoring teams, driving production-grade code, and collaborating across engineering disciplines...SuggestedRemote work
Mayo Foundation for Medical Education and Research
Rochester, MN1 day ago $102.3k - $143.2k
...Savings:HSA and FSAs for eligible expenses. Retirement: Competitive retirement package to secure your future. Responsibilities Data Engineer gains access to data across the organization and provides ongoing analysis of the data by monitoring, profiling and analyzing...SuggestedFull timeMonday to FridayFlexible hoursWeekend work- Lead data design, prototype, and development of data pipeline architecture pipelines. Lead implementation of internal process improvements... .... Be a senior Individual contributor of the Data or Software Engineering teams. Be part of Technical Review Board along with Manager...SuggestedRemote workFlexible hours
- ...time series analytics and Generative AI Solutions tailored for finance applications, not clinical.AI/ML Engineers at Mayo Clinic play a pivotal role in the union of data, systems, and computer sciences. They work closely with a multidisciplinary team, including clinicians...Flexible hours
- AI/ML Engineers at Mayo Clinic play a pivotal role in the union of data, systems, and computer sciences. They work closely with a multidisciplinary team, including... ...sophisticated AI technologies and clinical applications.• Leading component design, development, integration, and...Remote workFlexible hours
- The Director - Artificial Intelligence (AI) leads the engineering and research team that builds and delivers Mayo Clinic DigPath's digital-biology... ...clinical, operational, technical, legal, compliance, and data leaders to advance AI initiatives from concept through...Visa sponsorshipFlexible hours
$84.63k - $112.84k
...powered world, connecting people, data, and applications through our... ...Door is responsible for leading end‑to‑end presales program execution... ...workstreams (delivery, engineering, finance, contract, product, field... ...in these states: AK CA CT DC DE IL MA MD NJ NY TX VA WA...Full timeContract workTemporary workRemote workLumen
Rochester, MN17 hours ago- ...approach to where we work. About the Role: As a Machine Learning Engineer, you will have the opportunity to collaborate closely with... ...problems executing on a high throughput system with dynamic data. About the Job: Design, develop, and deploy machine learning...Permanent employmentFull timeWork at officeRemote workWork from homeFlexible hours
Instacart
Rochester, MN4 days ago $107.66k - $161.7k
...working on our Poe product. About the Team and Role: Our small engineering team works on challenging problems every day. We have a culture... ...end ownership of Machine Learning systems -- from prototyping, data pipelines and training, to realtime LLM application at scale...Remote jobFull timeWork experience placementInternshipQuora
Rochester, MN4 days ago- The IT Manager for the Outpatient Ancillary Unit leads a team responsible for the support, configuration, and optimization of Epic modules... ...operational teams, advancing service excellence, enabling data-driven decisions, and maintaining compliance within a rapidly evolving...Visa sponsorshipFlexible hours
- ...and fulfillment. We use machine learning and Internet-scale data to elevate customer experience, improve efficiency, and reduce... ...and Discovery ML team at Instacart works alongside world-class engineers, data scientists, and product managers to shape the future of search...Remote jobPermanent employmentWork experience placementInternshipWork at officeWork from homeFlexible hours
Instacart
Rochester, MN2 days ago $110.61k
Distributed Systems Software Engineer, Python / Go Join to apply for the Distributed Systems... ...bring deep engineering insights and a data driven approach to test automation, reporting... ...these problems, as well as imagining and leading new initiatives within the team and...Full timeLocal areaRemote workWorldwide- ...timely and predictable delivery of Enterprise Data & Analytics initiatives on behalf of business owners and product, engineering, and data and analytics teams. This role... ...and support financial and capacity management• Lead sprint and Agile delivery operations, including...Remote workFlexible hours
$65 per hour
...Minneapolis Bloomington Duluth Brooklyn Park Plymouth Woodbury The 10 most popular job searches in Rochester, MN are: work from home data entry data entry clerk virtual assistant nursing informatics travel nursing computer security travel rn scientist customer care...Remote jobHourly payWeekly payExtra incomeFull timeTemporary workPart timeFor contractorsSeasonal workWork from homeFlexible hours$141.02k - $204.53k
...secure your future. Responsibilities The Role As a Senior Software Engineer for AI Platform Engineering , you will bridge the gap between... ...(GCP, AWS, Azure). Experience in databases, analytics, big data systems or business intelligence products. Experience building...Full timeRemote workFlexible hoursWeekday work$134k - $168k
...a global technology solutions provider leading the AI and Digital Revolution. WWT combines... ...! What is the Solutions Consulting & Engineering (SC&E) Team and why join? Solutions... ...organizational lines — engineering, design, data, AI, cloud — ensuring we present as one...Full timeContract workShift workWorld Wide Technology
Rochester, MN4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineering (DE) Lead. Be the first to apply!
Related searches
- data engineering intern summer Rochester, MN
- junior data engineer remote Rochester, MN
- data intern Rochester, MN
- data collection researcher Rochester, MN
- clinical data Rochester, MN
- data visualization Rochester, MN
- data loss prevention engineer Rochester, MN
- clinical data coordinator remote Rochester, MN
- data recovery Rochester, MN
- data internship Rochester, MN