Lead Data Engineer
Eliassen Group
Job Description
Job Description
Description:
Hybrid At least 2 days per week in office in Chicago, IL
Our client seeks a Lead Data Engineer to design, build, and optimize large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. The role requires leadership of a major workstream, strong Scala, Spark, SQL, and Python skills, and cloud data platform expertise. The engineer will partner across Product, Data Science, Security, Privacy, and Platform Engineering to deliver secure, scalable, and compliant data solutions.
For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.
Rate: $75.00 to $85.00/hr. w2
Responsibilities:- Lead design, implementation, and optimization of large-scale data processing using Scala, Spark, SQL, and modern data platforms for attribution.
- Design and operate trusted data pipelines handling advertiser, customer, and measurement datasets across AWS, GCP, Azure, and approved ecosystems.
- Collaborate with Product, Data Science, Security, Privacy, and Platform Engineering to deliver privacy-preserving attribution, measurement, forecasting, and analytics.
- Own scalable batch and streaming workflows using orchestration frameworks and cloud-native services.
- Implement data classification, access controls, and privacy-preserving techniques aligned with security and compliance requirements.
- Drive clean-room and trusted data-sharing environments, exposing only aggregated or privacy-protected outputs.
- Build observability, monitoring, alerting, and operational tooling for reliability, performance, and compliance.
- Troubleshoot complex platform, performance, and pipeline issues across distributed systems.
- Influence technical design, architecture, and best practices in partnership with senior engineering leadership.
- Mentor engineers, lead design and code reviews, and provide technical leadership.
- Ensure all sensitive processing occurs within controlled, auditable boundaries with no unintended egress of PII or proprietary signals.
- Define data handling standards and document trust boundaries, data contracts, lineage, and permitted movement between zones.
- Apply privacy-preserving computation, including aggregation-before-export, pseudonymization, tokenization, differential privacy concepts, and privacy-aware reporting.
- Implement encryption, key management, and secure handling using cloud-native security and governance services.
- Support audits, compliance, governance reviews, and secure data-sharing initiatives.
- 8+ years in Data Engineering with strong Scala and extensive Apache Spark on AWS and/or GCP.
- Strong Python for pipelines, tooling, automation, and infrastructure modules.
- Advanced SQL across relational, cloud warehouses, and lakehouse platforms handling TB-scale datasets.
- Design, build, and maintenance of batch and streaming pipelines.
- Data warehousing, dimensional modeling, data quality, partitioning, and performance optimization.
- Distributed data processing and lakehouse architectures such as Databricks, Delta Lake, or Apache Spark.
- Operating distributed data platforms at scale with orchestration tools like Airflow, Databricks Workflows, or AWS Step Functions.
- Git-based workflows and automated testing frameworks.
- Cloud-native development on AWS and/or GCP with CI/CD, code reviews, observability, and production support.
- Proven technical leadership, mentorship, and delivery within tight timelines.
- Trusted environment execution: clean rooms or secure data-sharing platforms, handling PII and regulated data, fine-grained access controls, governance policies, and policy enforcement.
- Familiarity with tokenization, pseudonymization, aggregation-before-export, and differential privacy concepts.
- Experience with measurement, attribution, audience analytics, or privacy-preserving reporting solutions.
- Data lineage and governance tooling such as Unity Catalog, AWS Glue Data Catalog, Apache Atlas, or OpenLineage.
- Understanding of trust boundaries, secure data-sharing patterns, and zero-trust data architecture.
- Experience documenting data contracts, flows, lineage, and permitted data movement between zones and domains.
- Encryption, key management, and secure handling of sensitive data with cloud-native security services.
- Design of observability and alerting to detect anomalous movement, policy violations, and potential leakage.
- Experience in environments where only aggregated, anonymized, tokenized, or privacy-protected outputs may leave trusted boundaries.
- Strong written and verbal English communication and experience in Agile/SCRUM.
- Good to have: Databricks, AWS Clean Rooms or PETs, advertising and retail media platforms, collaboration with Security/Privacy/Risk/Compliance, ELK/Grafana/OpenTelemetry, Docker/Kubernetes, and secure design reviews.
Recruitment Transparency Notice
Eliassen Group values transparency in our recruitment practices. Please be advised that Eliassen Group utilizes artificial intelligence (AI) tools as part of its initial application screening and hiring process. You may receive email and SMS notifications from the Eliassen Virtual Recruiting Team ( View email address on ziprecruiter.com , View phone number on ziprecruiter.com) inviting you to complete a brief voice screening as part of your application process. These tools assist our hiring teams in different ways, including but not limited to, assistance in reviewing application materials to help identify candidates whose qualifications most closely match the requirements of the position. All AI-assisted evaluations and responses are reviewed by human recruiters before any hiring decisions are made. The use of AI in our process is intended to support fairness, efficiency, and consistency, and Eliassen Group takes measures to prevent bias or discrimination in connection with its hiring practices. By proceeding, you acknowledge, agree, and consent to Eliassen Group’s use of these tools, including AI tools, as part of the application and hiring process.
Skills, experience, and other compensable factors will be considered when determining pay rate. The pay range provided in this posting reflects a W2 hourly rate; other employment options may be available that may result in pay outside of the provided range. W2 employees of Eliassen Group who are regularly scheduled to work 30 or more hours per week are eligible for the following benefits: medical (choice of 3 plans), dental, vision, pre-tax accounts, other voluntary benefits including life and disability insurance, 401(k) with match, and sick time if required by law in the worked-in state/locality. If anyone reaches out to you about an open position connected with Eliassen Group, please ensure that you are working directly with us by confirming the following:
· When you work with Eliassen Group, all email communication will come from an Eliassen.com address, never Gmail, Yahoo, etc.
· Eliassen Group will never ask you for personal information (home address, bank account, or check routing number) until you have worked with someone clearly associated with Eliassen Group.
If you have any indication of fraudulent activity, please contact View email address on ziprecruiter.com.
About Eliassen Group:
Eliassen Group is a strategic consulting firm that helps organizations reach further and achieve more through our technology, business advisory, and life sciences solutions. For nearly 40 years, we have combined exceptional people, deep domain expertise, and intelligent capabilities to expand our clients’ capacity and accelerate meaningful outcomes. We are driven by a purpose to positively impact the lives of our employees, clients, consultants, and the communities we serve.
Eliassen is committed to building a diverse and inclusive team from a variety of backgrounds, perspectives, and skills. We are an Equal Opportunity and Affirmative Action Employer and all employment decisions are based on merit, performance, and business needs. Eliassen does not discriminate on the basis of race, color, gender identity or expression, sexual preference or orientation, sex (including pregnancy, childbirth, and related medical conditions), marital status, creed, religion, physical or mental disability, genetic information, military or veteran status, age, ancestry, national origin, citizenship status, prohibited criminal record inquiries of applicants and employees, or any other category protected by federal, state, or local laws.
Don’t miss out on our referral program! If we hire a candidate that you refer us to then you can be eligible for a $1,000 referral check!
- ...Lead Data EngineerWe are seeking a Lead Data Engineer to drive the design, architecture, and delivery of scalable data platforms, AI-powered data products, and enterprise-grade data services. This role goes beyond hands-on engineering to include technical leadership, architectural...Suggested
$128.47k - $208.77k
...Career Area: Technology, Digital and Data Job Description: Your Work Shapes the... ...latest technologies to build industry leading digital solutions for our customers and dealers... ...& insights. As a Senior Lead Data Engineer on the Data Engineering team, you will be...SuggestedFull timePart timeWorldwideVisa sponsorshipRelocation packageFlexible hours$179.4k - $204.7k
...Lead Data EngineerDo you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast... ...problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies...SuggestedFull timePart timeInternshipLocal area- ...Lead Data EngineerThis role is responsible for providing technical leadership and execution expertise to design and deliver end-to-end data engineering solutions to support new product development and advanced analytics capabilities and drive innovation and decision-making...Suggested
$98k - $182k
...business. We are a small, high‑impact team of engineers operating at massive scale—trillions of... ...), your work directly connects our data ecosystem to the biggest platforms in the... ...performance. Team Mentorship & Quality: Lead code reviews, enforce standards and mentor...Suggested$106.5k - $182.6k
...Modern Data Environment DeveloperProvides continuous build-up and operationalization of an enterprise-class modern data environment... ...integrationSupport existing data pipelines on-prem, cloud and business engineering extractProvide technical support to meet application service...Full timeTemporary workPart timeWork from home3 days per week- ...Must have skill: Snowflake SQL, Data warehouse, Python, AWS-Lambda, ETL/ELT Job Description: Required Skills: 6-8 years+ experience... ...metrics in any specific business Responsibilities: • Lead data warehouse development activities for a team, leading more...
- ...Lead Data EngineerLocation: 100% Remote (Preferred: Chicago, IL / Chicagoland Area) Client Location: Chicago, IL Duration: 6+ Months... ...1Position Overview: We are seeking a highly skilled Lead Data Engineer to join a growing Data & Analytics team responsible for building...Contract workRemote work
- ...Lead Data EngineerProven strong experience on building re-usable framework, libraries, and data platform capabilities Azure and Databricks experience Strong experience of python, building re-usable python based frameworks Experience with data lake environment must Work...
- ...Role Overview We are seeking a hands-on Senior / Lead Data Engineer to provide technical leadership for the platforms, pipelines, and data products that power analytics, applications, and investment decision-making across the firm. You will architect scalable, cloud...
$98k - $182k
...The Epsilon Activation Delivery Platform team seeks a senior engineer to build and maintain high-scale data pipelines and integrations with major advertising and CTV platforms. The role reports to the Engineering Director and collaborates with data engineers, product...Full timeFlexible hours$111.8k - $160k
...Lead Data EngineerBuild the Future of Data at GATXRemote (Chicago-area preferred)Founded in 1898 and headquartered in Chicago, IL, GATX... ...intelligence ecosystem. We are looking for a Lead Data Engineer who's ready to do more than just build pipelines on Databricks...Temporary workWork at officeRemote work$179.4k - $204.7k
...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers...Full timePart timeInternshipH1bLocal area- ...Position Summary: The Enterprise Data Architect is a hands-on technical leader... ...artificial intelligence initiatives. The role leads modernization through Microsoft Fabric,... ...consistency, timeliness, and uniqueness. Engineering Standards and Automation Implement...
- ...Lead Data EngineerRoel: Lead Data Engineer Location: Chicago IL, Locals only. Duration: Longterm Contract Interview Type: Video and Final interview is in-person interview.Snowflake/Snowpipe, SQL, Python/PySparkData Modeling, Data Engineering with Modeling Techniques -...Contract workLocal area
- ...Lead Data EngineerThe Epsilon Activation Delivery Platform team builds the core framework and connectors powering our audience activation business. We are a small, high-impact team of engineers operating at massive scale—trillions of user events flow through our pipelines...Local areaFlexible hours
- ...Lead Data Engineer Locations: Richmond - VA / McLean - VA / Plano - TX / Chicago - IL / NYC - NY / Wilmington - DE (Preferred Location; Hybrid Role; Needs to work 3 days from Office in a week) Job Type: Long Term Contract Responsibilities: Lead the design...Long term contractWork at office
$125k - $207k
84.51° Overview: 84.51° is a retail data science, insights and media company. We help The Kroger Co., consumer packaged goods... ..._________________________________________________________ Lead Data Engineer We are a full-stack data science company and a wholly...Work at office- ...Geneva Trading in Chicago, IL is seeking a Data Engineering Manager to own market data platforms and analytical data systems. You will manage a small team, write production code, review designs, debug systems, and stay close to technical details. The role emphasizes reliable...
- ...Tokio Marine Highland (TMH) is modernizing its enterprise data platform to better support reporting, analytics, operational decision... ...across the organization. We are seeking a hands-on Lead Data Engineer to lead the design, development, and evolution of our Snowflake...Work at officeRemote work2 days per week3 days per week
$128.47k - $208.77k
...Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World... ...the latest technologies to build industry leading digital solutions for our customers and dealers... ...leads & insights. As a Senior Lead Data Engineer on the Data Engineering team, you will be...WorldwideRelocation package- ...agile methodologies, encouraging a culture of innovation and high performance in a leading financial institution. As an Agility Lead in the Corporate Sector's Reference Data Engineering team, you will play a significant role as a champion for the adoption and...Temporary work
$179.4k - $204.7k
...Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment?...Full timePart timeInternshipH1bLocal area- Work Flexibility: RemoteAs a Senior Lead, Data Engineering, you will serve as a technical leader who helps shape the future of enterprise data solutions. In this role, you will drive complex data initiatives, influence technical strategy, and partner with teams across...Full timeFor contractorsRemote work
$117.2k - $175.8k
Sr Data Engineer - GE07BEWe’re determined to make a difference and are proud to be an insurance company that goes well beyond coverages and... ...Department’s Global Specialty team is seeking a skilled Lead Data Engineer with a proven track record. The ideal candidate...Full timeTemporary workWork at office3 days per week$180k - $200k
...harborcapital.com/careers, search for this position and click on "Apply" to start your application!Role DescriptionWe are seeking a Lead Data Engineer to drive the design, architecture, and delivery of scalable data platforms, AI-powered data products, and enterprise-grade...Full timeWork at office- ...Machine Learning / Data Science Engineer CapTech is an award-winning consulting firm that collaborates with clients to achieve what's possible... ..., attending various business development events, and leading teams of junior data scientists and engineers. Qualifications...Work at officeRemote workVisa sponsorshipWork visaFlexible hours
- ...the Premier Global Partnership. We are hiring a Senior AI Engineer to join a newly formed AI Innovation Function, part of the Investment... ..., and close deals. You will work across the full stack—data pipelines, ML models, LLM-powered applications, and Salesforce integrations...Full timeImmediate start
- ...No visa restrictions Job Title: Data Engineer Location : Chicago, IL Primary Skills: Business Intelligence, Azure, C#, Data Warehouse, Python, SQL, Java Location: 100% Remote Overview: Companyy (IT) vision and strategy...Temporary workRemote workFlexible hours
$100k
...Cloud World/Oracle Java One (Las Vegas) -2023/2022 and at Gartner Data Analytics Summit (Florida)-2023.All positions are open for all... ...developers, Data analysts/Data Scientists, Machine Learning engineers for full time positions with clients. Who should apply? Recent...Full timeH1b
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!



