Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer

Eliassen Group

Job Description

Job Description

Description:

Hybrid At least 2 days per week in office in Chicago, IL

 

Our client seeks a Lead Data Engineer to design, build, and optimize large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. The role requires leadership of a major workstream, strong Scala, Spark, SQL, and Python skills, and cloud data platform expertise. The engineer will partner across Product, Data Science, Security, Privacy, and Platform Engineering to deliver secure, scalable, and compliant data solutions.

 

For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

 

Rate: $75.00 to $85.00/hr. w2

Responsibilities:
  • Lead design, implementation, and optimization of large-scale data processing using Scala, Spark, SQL, and modern data platforms for attribution.
  • Design and operate trusted data pipelines handling advertiser, customer, and measurement datasets across AWS, GCP, Azure, and approved ecosystems.
  • Collaborate with Product, Data Science, Security, Privacy, and Platform Engineering to deliver privacy-preserving attribution, measurement, forecasting, and analytics.
  • Own scalable batch and streaming workflows using orchestration frameworks and cloud-native services.
  • Implement data classification, access controls, and privacy-preserving techniques aligned with security and compliance requirements.
  • Drive clean-room and trusted data-sharing environments, exposing only aggregated or privacy-protected outputs.
  • Build observability, monitoring, alerting, and operational tooling for reliability, performance, and compliance.
  • Troubleshoot complex platform, performance, and pipeline issues across distributed systems.
  • Influence technical design, architecture, and best practices in partnership with senior engineering leadership.
  • Mentor engineers, lead design and code reviews, and provide technical leadership.
  • Ensure all sensitive processing occurs within controlled, auditable boundaries with no unintended egress of PII or proprietary signals.
  • Define data handling standards and document trust boundaries, data contracts, lineage, and permitted movement between zones.
  • Apply privacy-preserving computation, including aggregation-before-export, pseudonymization, tokenization, differential privacy concepts, and privacy-aware reporting.
  • Implement encryption, key management, and secure handling using cloud-native security and governance services.
  • Support audits, compliance, governance reviews, and secure data-sharing initiatives.
Experience Requirements:
  • 8+ years in Data Engineering with strong Scala and extensive Apache Spark on AWS and/or GCP.
  • Strong Python for pipelines, tooling, automation, and infrastructure modules.
  • Advanced SQL across relational, cloud warehouses, and lakehouse platforms handling TB-scale datasets.
  • Design, build, and maintenance of batch and streaming pipelines.
  • Data warehousing, dimensional modeling, data quality, partitioning, and performance optimization.
  • Distributed data processing and lakehouse architectures such as Databricks, Delta Lake, or Apache Spark.
  • Operating distributed data platforms at scale with orchestration tools like Airflow, Databricks Workflows, or AWS Step Functions.
  • Git-based workflows and automated testing frameworks.
  • Cloud-native development on AWS and/or GCP with CI/CD, code reviews, observability, and production support.
  • Proven technical leadership, mentorship, and delivery within tight timelines.
  • Trusted environment execution: clean rooms or secure data-sharing platforms, handling PII and regulated data, fine-grained access controls, governance policies, and policy enforcement.
  • Familiarity with tokenization, pseudonymization, aggregation-before-export, and differential privacy concepts.
  • Experience with measurement, attribution, audience analytics, or privacy-preserving reporting solutions.
  • Data lineage and governance tooling such as Unity Catalog, AWS Glue Data Catalog, Apache Atlas, or OpenLineage.
  • Understanding of trust boundaries, secure data-sharing patterns, and zero-trust data architecture.
  • Experience documenting data contracts, flows, lineage, and permitted data movement between zones and domains.
  • Encryption, key management, and secure handling of sensitive data with cloud-native security services.
  • Design of observability and alerting to detect anomalous movement, policy violations, and potential leakage.
  • Experience in environments where only aggregated, anonymized, tokenized, or privacy-protected outputs may leave trusted boundaries.
  • Strong written and verbal English communication and experience in Agile/SCRUM.
  • Good to have: Databricks, AWS Clean Rooms or PETs, advertising and retail media platforms, collaboration with Security/Privacy/Risk/Compliance, ELK/Grafana/OpenTelemetry, Docker/Kubernetes, and secure design reviews.

Recruitment Transparency Notice

 

Eliassen Group values transparency in our recruitment practices. Please be advised that Eliassen Group utilizes artificial intelligence (AI) tools as part of its initial application screening  and hiring process. You may receive email and SMS notifications from the Eliassen Virtual Recruiting Team ( View email address on ziprecruiter.com , View phone number on ziprecruiter.com) inviting you to complete a brief voice screening as part of your application process. These tools assist our hiring teams in different ways, including but not limited to, assistance in reviewing application materials to help identify candidates whose qualifications most closely match the requirements of the position. All AI-assisted evaluations and responses are reviewed by human recruiters before any hiring decisions are made. The use of AI in our process is intended to support fairness, efficiency, and consistency, and Eliassen Group takes measures to prevent bias or discrimination in connection with its hiring practices. By proceeding, you acknowledge, agree, and consent to Eliassen Group’s use of these tools, including AI tools, as part of the application and hiring process.

 

Skills, experience, and other compensable factors will be considered when determining pay rate. The pay range provided in this posting reflects a W2 hourly rate; other employment options may be available that may result in pay outside of the provided range. W2 employees of Eliassen Group who are regularly scheduled to work 30 or more hours per week are eligible for the following benefits: medical (choice of 3 plans), dental, vision, pre-tax accounts, other voluntary benefits including life and disability insurance, 401(k) with match, and sick time if required by law in the worked-in state/locality. If anyone reaches out to you about an open position connected with Eliassen Group, please ensure that you are working directly with us by confirming the following:

· When you work with Eliassen Group, all email communication will come from an Eliassen.com address, never Gmail, Yahoo, etc.

· Eliassen Group will never ask you for personal information (home address, bank account, or check routing number) until you have worked with someone clearly associated with Eliassen Group.

If you have any indication of fraudulent activity, please contact  View email address on ziprecruiter.com.

 

About Eliassen Group:

 

Eliassen Group is a strategic consulting firm that helps organizations reach further and achieve more through our technology, business advisory, and life sciences solutions. For nearly 40 years, we have combined exceptional people, deep domain expertise, and intelligent capabilities to expand our clients’ capacity and accelerate meaningful outcomes. We are driven by a purpose to positively impact the lives of our employees, clients, consultants, and the communities we serve.

 

Eliassen is committed to building a diverse and inclusive team from a variety of backgrounds, perspectives, and skills. We are an Equal Opportunity and Affirmative Action Employer and all employment decisions are based on merit, performance, and business needs. Eliassen does not discriminate on the basis of race, color, gender identity or expression, sexual preference or orientation, sex (including pregnancy, childbirth, and related medical conditions), marital status, creed, religion, physical or mental disability, genetic information, military or veteran status, age, ancestry, national origin, citizenship status, prohibited criminal record inquiries of applicants and employees, or any other category protected by federal, state, or local laws.

 

Don’t miss out on our referral program! If we hire a candidate that you refer us to then you can be eligible for a $1,000 referral check!

 

Vacancy posted 23 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in Chicago, IL vacancy
  • Work Flexibility: RemoteAs a Senior Lead, Data Engineering, you will serve as a technical leader who helps shape the future of enterprise data solutions. In this role, you will drive complex data initiatives, influence technical strategy, and partner with teams across... 
    Suggested
    Full time
    For contractors
    Remote work

    Stryker

    Chicago, IL
    1 day ago
  • $90k - $200k

     ...the country.Job DescriptionCapTech Machine Learning Engineers are responsible for designing and implementing data-driven solutions for our clients, with a specific...  ...and our clients.Specific responsibilities for the Lead Machine Learning Engineer position include:... 
    Suggested
    Work at office
    Remote work
    Visa sponsorship
    Work visa
    Flexible hours

    CapTech

    Chicago, IL
    2 days ago
  • $106.5k - $182.6k

     ...build-up and operationalization of an enterprise-class modern data environment, which may include various components within the Azure...  ...existing data pipelines on-prem, cloud and business engineering extractProvide technical support to meet application service level... 
    Suggested
    Full time
    Temporary work
    Part time

    Alliant Credit Union

    Chicago, IL
    3 days ago
  • $180k - $200k

     ...harborcapital.com/careers, search for this position and click on "Apply" to start your application!Role DescriptionWe are seeking a Lead Data Engineer to drive the design, architecture, and delivery of scalable data platforms, AI-powered data products, and enterprise-grade... 
    Suggested
    Full time
    Work at office

    Harbor Funds

    Chicago, IL
    4 days ago
  • $111.8k

     ...Overview Lead Data Engineer | Build the Future of Data at GATX Remote (Chicago-area preferred) Founded in 1898 and headquartered in Chicago, IL, GATX Corporation (NYSE: GATX) is an industry leader with 125+ years of success - success that is powered by our people. We... 
    Suggested
    Temporary work
    Work at office
    Remote work

    GATX

    Chicago, IL
    2 days ago
  • $98k - $182k

     ...business. We are a small, high‑impact team of engineers operating at massive scale—trillions of...  ...), your work directly connects our data ecosystem to the biggest platforms in the...  ...performance. Team Mentorship & Quality: Lead code reviews, enforce standards and mentor... 

    Publicis Groupe Holdings B.V

    Chicago, IL
    3 days ago
  • $209k - $238.5k

    ## Senior Lead Data EngineerApplylocations: Chicago, ILtime type: Full timeposted on: Posted Todayjob requisition id: R246107Senior Lead...  ...problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies... 
    Full time
    Part time
    Internship
    Local area

    Capital One

    Chicago, IL
    3 days ago
  • $200k

     ...Optiver Chicago is seeking a seasoned Data Engineer to support and advance the data capabilities of our local research team and contribute...  ...C++, Rust") and exposure to MLOps or MLflow Proven ability to lead projects independently and deliver outcomes in fast‑paced environments... 
    Work at office
    Local area
    Immediate start

    Optiver

    Chicago, IL
    2 days ago
  •  ...Hybrid At least 2 days per week in office in Chicago, IL Our client seeks a Lead Data Engineer to design, build, and optimize large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. The role requires leadership of a major... 
    Work at office
    2 days per week

    Eliassen Group

    Chicago, IL
    4 days ago
  •  ...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers... 
    Internship
    H1b
    Local area

    Capital One National Association

    Chicago, IL
    4 days ago
  •  ...Rate: Not Specified Job type: contractor Location: Chicago, IL Must have skill: Snowflake SQL, Data warehouse, Python, AWS-Lambda, ETL/ELT Job Description: Responsibilities: Lead data warehouse development activities for a team, leading more than one project in parallel.... 
    For contractors

    TechDigital Group

    Chicago, IL
    1 day ago
  •  ...Geneva Trading in Chicago, IL is seeking a Data Engineering Manager to own market data platforms and analytical data systems. You will manage a small team, write production code, review designs, debug systems, and stay close to technical details. The role emphasizes reliable... 

    Geneva Trading

    Chicago, IL
    2 days ago
  •  ...Lenovo is seeking a technical lead to own the full agentic stack—from LLM orchestration...  ...agentic pipelines, remember and reason over data, and integrate tools across multi-step...  ...deployments in production. Collaboration with AI engineers, ML scientists, and product managers is... 

    Lenovo

    Chicago, IL
    2 days ago
  • $145k - $165k

     ...Avison Young - USA is seeking a Lead of Analytics Engineering based in Chicago, Illinois. In this role, you will lead strategies and execution for proprietary data products that aid commercial real estate decision-makers. The ideal candidate is experienced in commercial... 

    Avison Young

    Chicago, IL
    2 days ago
  •  ...Caterpillar Inc. in Chicago, IL seeks a Lead Data Engineer for the Physical AI Platform. You will design, build, and maintain scalable data pipelines and microservices, collaborating with software engineers and data architects in an agile environment. Key responsibilities... 

    Caterpillar Brazil

    Chicago, IL
    1 day ago
  •  ...UNAVAILABLE is seeking an experienced engineer to build and maintain data pipelines using Spark and SQL. This role focuses on high-volume data environments and offers the opportunity to work with state-of-the-art technologies like AWS and Databricks. Ideal candidates will... 

    Unavailable

    Chicago, IL
    2 days ago
  • $112.2k - $202.6k

     ...HCSC is looking for a Lead Analytics Engineer in Chicago, Illinois, to design and develop data integrations for Enterprise BI solutions. The ideal candidate will lead projects and provide technical guidance within the Enterprise Business Analytics & Business Intelligence... 

    Corporate Strategy Division

    Chicago, IL
    2 days ago
  •  ...U.S. Bank is seeking a Principal Integration Engineer to lead data workstreams for a major integration program in Capital Markets and Electronic Trading. You will design data backends, warehouses, and ETL processes for pre- and post-trade data across back-office systems... 

    Relha LLC

    Chicago, IL
    1 day ago
  • $118.7k - $218.6k

    Position Summary Lead AI and Data Science Engineer II Drive the design and delivery of advanced analytics, artificial intelligence (AI), and generative artificial intelligence (GenAI) solutions that inform Talent strategy and workforce decisions. In this role, you... 

    Deloitte

    Chicago, IL
    1 day ago
  • $159.1k - $265.1k

    Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make...  ...ventures, and fuel growth through innovation. Work You’ll Do As a Lead AI and Data Science Engineer II on the team, you will be responsible for... 
    Local area

    Deloitte

    Chicago, IL
    19 hours ago
  • $125k - $207k

    84.51° is a retail data science, insights and media company. We help The Kroger Co., consumer...  ...2 million households. As a member of our engineering team, you will use cutting‑edge...  ...scalable enterprise applications. Summary As a Lead Data Engineer, you will design and... 
    Live in
    Work at office
    Relocation
    Relocation package
    Monday to Friday

    84.51°

    Chicago, IL
    1 day ago
  • Lead Data Engineer / Technical Architect Location: Chicago Reporting to: CIO Open to SOW to Hire, SOW, or FTE About the Role We are looking for an experienced Lead Data Engineer / Technical Architect to design and deliver a full end-to-end data architecture strategy... 

    InterEx Group

    Chicago, IL
    4 days ago
  • Senior Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast‑paced, collaborative, inclusive, and iterative delivery environment? At Capital One you will be part of a group of makers, breakers... 
    Internship

    Capital One National Association

    Chicago, IL
    1 day ago
  • $128.47k - $208.77k

     ...intelligence layer that connects machines, data, and people to make jobsites safer, more...  ...data to create a foundation where engineers like you can build and deploy against.If...  ...autonomy at Caterpillar. Job SummaryAs a Lead Data Engineer, you will design, build, and... 
    Full time
    Part time
    Work at office
    Relocation
    Visa sponsorship
    Relocation package
    Flexible hours

    Caterpillar

    Chicago, IL
    1 day ago
  • $133.37k - $156.9k

     ...One.Job DescriptionWe are seeking a highly skilled Senior Cloud Data Engineer to join our Data Product Engineering team. This role is hands-...  ...products across Azure and AWS using Databricks and Snowflake.Lead and contribute to the modernization and migration of on‑prem... 
    Full time
    Work experience placement
    Local area
    3 days per week

    US Bank

    Chicago, IL
    1 day ago
  • 84.51° is a retail data science and media company driving personalized shopper experiences for Kroger and partners. As Lead Data Engineer, you will design and implement data ingestion, storage, transformation, and distribution solutions on Azure Cloud, leveraging Databricks... 

    84.51°

    Chicago, IL
    4 days ago
  • $138.21k - $172.76k

     ...why McDonald’s remains one of the world’s leading corporations after almost 70 years....  ...creating smarter, more responsive digital and data systems that transform operations and elevate...  .... McDonald's is hiring a Data Engineer III focused on Data Clean Room data ingestion... 
    Full time
    Local area
    Flexible hours

    McDonald's Corporation

    Chicago, IL
    2 days ago
  • $127.2k - $228.96k

    Anticipated End Date: 2026-08-21 Position Title: Senior Data Engineer Job Description: Senior Data Engineer Hybrid 1: This...  ...by law. The Senior Data Engineer serves as the technical lead and platform owner for the security data lake supporting... 
    Full time
    Work experience placement
    Work at office
    Local area
    Day shift
    2 days per week
    1 day per week

    Elevance Health

    Chicago, IL
    4 days ago
  • $89.21k - $144.96k

    Career Area: Technology, Digital and Data Job Description: Your Work Shapes the World at Caterpillar Inc. When you join Caterpillar...  ...a better world, so we can all enjoy living in it. As a Data Engineer on the Platform Data Engineering team, you will be responsible for... 
    Full time
    Part time
    Relocation package
    Flexible hours

    Caterpillar Inc.

    Chicago, IL
    15 hours ago
  • $97.53k - $158.48k

    Career Area: Technology, Digital and Data Job Description: Your Work Shapes the...  ...latest technologies to build industry leading digital solutions for our customers and...  ...better, more sustainable world As a Data Engineer on the Platform Data Engineering team,... 
    Full time
    Part time
    Worldwide
    Relocation package
    Flexible hours

    Caterpillar Inc.

    Chicago, IL
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!