Lead Data Engineer
Expert Technology Services
Job Summary (Lead Data Engineer)
- Lead the design, development, and deployment of large-scale data engineering solutions.
- Provide technical leadership and guidance to data engineering teams.
- Architect, build, and maintain enterprise-level ETL/ELT pipelines using Python and Google Cloud Platform (GCP) tools.
- Ensure high performance and scalability of data processing workflows with BigQuery, Dataflow, Dataproc/PySpark, Cloud Composer (Airflow), Pub/Sub, and Cloud Storage.
- Develop and optimize complex SQL queries and data models to support business analytics and data science.
- Collaborate with stakeholders to translate business requirements into robust data solutions.
- Implement CI/CD, DevOps practices, and automated deployment frameworks for data engineering projects.
- Drive initiatives independently from concept through to production, ensuring best practices in coding, testing, and deployment.
- Support data governance, data quality, and compliance requirements as needed.
- Stay current with emerging technologies, especially in cloud data engineering and AI/ML enablement.
- (Preferred) Leverage experience with Snowflake, Databricks, lakehouse architectures, and feature engineering pipelines.
- (Preferred) Contribute to Agile processes within large enterprise environments.
- Lead the design, development, and deployment of large-scale data engineering solutions.
- Provide technical leadership and guidance to data engineering teams.
- Architect, build, and maintain enterprise-level ETL/ELT pipelines using Python and Google Cloud Platform (GCP) tools.
- Ensure high performance and scalability of data processing workflows with BigQuery, Dataflow, Dataproc/PySpark, Cloud Composer (Airflow), Pub/Sub, and Cloud Storage.
- Develop and optimize complex SQL queries and data models to support business analytics and data science.
- Collaborate with stakeholders to translate business requirements into robust data solutions.
- Implement CI/CD, DevOps practices, and automated deployment frameworks for data engineering projects.
- Drive initiatives independently from concept through to production, ensuring best practices in coding, testing, and deployment.
- Support data governance, data quality, and compliance requirements as needed.
- Stay current with emerging technologies, especially in cloud data engineering and AI/ML enablement.
- (Preferred) Leverage experience with Snowflake, Databricks, lakehouse architectures, and feature engineering pipelines.
- (Preferred) Contribute to Agile processes within large enterprise environments.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in Madison, WI vacancy
$70 - $80 per hour
DescriptionKforce has a client that is seeking a Lead Data Engineer in Madison, WI.Key Responsibilities:Technical Leadership:* Lead the design, development, and implementation of enterprise-scale data platforms and pipelines* Provide technical leadership and mentorship...Suggested- Hello, Position: Lead Data Engineer Location: Madison, WI (Hybrid) Duration: 6+months C2H (Contract to Hire) Job Description Bachelor's Degree in Computer Science, Engineering, Information Systems, or related field. 8-10+ years of Data Engineering experience. 3...SuggestedContract work
$78k - $156k
...-changing technologies spans the spectrum of healthcare, with leading businesses and products in diagnostics, medical devices, nutritionals... ...160 countries.JOB DESCRIPTION:Position OverviewAs a Senior Data Engineer, this seasoned professional will demonstrate competence and...SuggestedFull timePart timeWork visaShift workNight shiftWeekend work$85.91k - $162.89k
OverviewBaker Tilly is a leading advisory, tax and assurance firm, providing clients with a genuine coast-to-coast and global advantage... ...system.Job Description:As a Senior Consultant - Associate Data Engineer you will design, build, and optimize modern data solutions for...SuggestedFull timeLocal areaWorldwide- ...Senior Information Security Engineer Anywhere Type: Contract Category: Security Industry: Financial Services Workplace... ...will contribute to build and feature work under direction from Lead and Principal Engineers. This role participates in a four-person...SuggestedHourly payContract workLocal areaRemote workShift workNight shiftRotating shift
Eliassen Group
Madison, WI1 day ago - ...Job Posting • 6+ years of experience in analytics engineering, data engineering, or related field. Good proficiency in SQL, Python, and R; extensive experience with Snowflake, Alteryx, Azure, Git and business intelligence tools. Good proficiency with client-side...
Samprasoft
Madison, WI3 days ago $90k - $110k
...a Madison-based organization that’s investing heavily in a modern data platform to deliver reliable, scalable, and well-governed data across analytics, applications, and integrations. This Data Engineer will play a key role in designing and building production-grade data...- Job Description VAS, a URUS Group company, is looking for a Data Engineer to help design, build, and improve the data solutions that support our global business. Working primarily in Databricks, you’ll develop and maintain pipelines and datasets that power reporting, analytics...
- Senior Data Engineer - Microsoft Fabric & Power BI What is IP Corporation? IP Corporation is a privately held, family-owned chemical manufacturing organization with more than 70 years of stability, innovation, and growth. Headquartered in Minnesota, IP Corporation supports...Work at office
$64.89k - $173.04k
...of something bigger - helping to simplify health care one person, one family and one community at a time. Position Summary The Data Engineer will be responsible for delivering high quality modern data solutions through collaboration with our engineering, analysts, data...Hourly payFull timeTemporary workLocal area$99k - $167k
American Family Insurance looking for an experienced and driven Senior Data Engineer to guide our AI Platform team. In this important role, you will oversee a team of engineers in advancing our service-oriented data and AI platform. The Data and AI Platform team influences...Full timeWork at officeLocal areaRelocation packageFlexible hours$171.3k - $342.7k
...changing technologies spans the spectrum of healthcare, with leading businesses and products in diagnostics, medical devices, nutritionals... ...-generation, early-detection diagnostics.As the Director, Data Engineering, you’ll serve as Cancer Diagnostics’ senior data executive -...WorldwideShift workNight shift- ...RippleMatch Inc. is seeking an Integration Solutions Engineer in Madison, WI, who will enhance and support data integrations between Epic and healthcare systems. The role involves analyzing and solving integration challenges, ensuring efficient data flow for high-quality...Relocation package
RippleMatch
Madison, WI2 days ago - ...Junior Data ScientistSYNERGISTICIT wants every job seeker to be aware that the job market is challenging and to stand out you need... ...Junior Data Scientist.Who Should Apply: Recent computer science/engineering/mathematics/statistics or science graduates looking to make their...H1b
SynergisticIT
Madison, WI4 days ago - ...Senior Data ScientistElephas is a growing biotechnology company located in Madison, WI... ...modeling, which includes working with data engineers to assess data quality of incoming assay... ...rigorous and appropriate analytic approaches.Lead complex data presentation to technical...Local areaFlexible hours
Elephas
Madison, WI3 days ago - ...Investment Data ScientistSophisticated work. In a great city. Making a difference.The State of Wisconsin... ..., you know your work matters.Data Services & Engineering Teams at SWIB supports, implements & develops industry-leading systems and platforms to support SWIB's diverse...Work at officeRemote workRelocation
State of Wisconsin Investment Board
Madison, WI2 days ago - ...Senior Data Scientist The Weather Company provides the best weather insight in the world, and is leading the charge in the growing area of weather decision support for business. We are offering you a unique opportunity to apply and/or develop your mathematical modeling...
The Weather Channel
Madison, WI2 days ago $102.3k - $209.5k
...Description Tracks and monitors ongoing Data Center critical infrastructure... ...suitability for data center builds. Acts as the engineering representative on a wide range of moderately... ...knowledge of engineering systems to lead standard diagnostics and repairs, escalating...Contract workTemporary workFor contractorsFlexible hoursShift workOracle
Madison, WI5 days ago$18 - $50 per hour
...career with us at Siemens Digital Industries Software! We are a leading global software company dedicated to the world of computer... ...: Siemens Digital Industries Software is seeking an AI & Engineering Data Intern to support emerging Artificial Intelligence and...Remote jobHourly payFull timeInternshipLocal areaSiemens
Madison, WI3 days ago$121k - $215k
...Blackwell we believe that diverse, equitable and inclusive teams lead to better outcomes. Husch Blackwell is committed to retaining,... ...be valued and empowered, then we invite you to apply to our Data Engineering Manager position. This position may be filled remotely or in a...Full timeTemporary workRemote workFlexible hours$126.2k - $264.1k
...Business Operations & Organizational Effectiveness Lead business operations programs for Engineering & Infrastructure organizations in partnership with... ..., staff meetings, and leadership updates. Provide data-driven insights and recommendations to support strategic...Temporary workWork at officeFlexible hoursOracle
Madison, WI4 days ago- ...We're a profitable, growth-stage company building industry-leading martech and data products for the PropTech space. Our focus is on our category... ...M units. The Role We are looking for a Senior Software Engineer to drive the evolution of our shared data platform - the...Temporary workRemote workWork from homeWork visaFlexible hoursShift work
ApartmentIQ
Madison, WI13 days ago - ...we are looking for entry-level software programmers, Java Full stack developers, Python/Java developers, data analysts/data scientists, machine learning engineers for full-time positions with clients.Who Should Apply: Recent computer science/engineering/mathematics/statistics...Full timeH1bImmediate startRemote work
SynergisticIT
Madison, WI4 days ago - ...Data Engineer - OffshoreDuration: 5+ months (Remote Role)Bachelors in Computer Science, Information Systems or related educational background. Have good knowledge of ETL and Data warehousing concepts using ICEDQ, Informatica, Oracle, Teradata, Control-M and SQL language...Remote work
Compu-Vision Consulting
Sun Prairie, WI4 days ago - ...We are looking for entry-level software programmers, Java Full stack developers, Python/Java developers, Data analysts/ Data Scientists, Machine Learning engineers. Who Should Apply Recent Computer science/Engineering /Mathematics/Statistics or Science Graduates looking...H1b
SynergisticIT
Madison, WI5 days ago $94.4k - $142.9k
...enhancement and effective operations of complex data movement between various systems using ETL methodologies. Responsible for leading company database development efforts and... ...or application consumers.Applies data engineering best practices for security, performance,...Temporary workFor contractorsWork experience placementSummer workCasual workInternshipSeasonal workWork at officeRemote workWork visaFlexible hours$86.4k
...: Highmark Health Job Description : JOB SUMMARY This job promotes the understanding, integration, and use of healthcare data and information technology in support of scholarly research and addressing healthcare business challenges. Leveraging experience in...For contractorsWork at officeLocal areaHighmark Health
Madison, WI4 days ago$5,000 per month
...drive to be the best. Our work improves lives. Find out how a career as an IT Manager of Data Platform & Engineering can change yours.The IT Manager of Data Platform & Engineering leads the development and evolution of Michels’ enterprise data platform. As a founding...Temporary workFlexible hours- Qualifications Extensive knowledge of data principles, patterns, processes, and practices... ...data science solutions or platforms Data engineering certification (e.g IBM Certified Data... ...leadership to shape sound architectural decisions Lead the analysis of the technology...
- A Data Engineer is a data professional who uses their expertise in data engineering and programming to build systems that collect, manage, and convert raw data into usable information for business analysts. Requirements and skills Previous 7+ years of experience as a data...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
Related searches