Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer

Eliassen Group

Job Description

Job Description

Description:

Hybrid At least 2 days per week in office in Chicago, IL

Our client seeks a Lead Data Engineer to design, build, and optimize large-scale data processing for attribution, measurement, forecasting, and privacy-preserving analytics. The role requires leadership of a major workstream, strong Scala, Spark, SQL, and Python skills, and cloud data platform expertise. The engineer will partner across Product, Data Science, Security, Privacy, and Platform Engineering to deliver secure, scalable, and compliant data solutions.

For our w2 consultants, we offer a great benefits package that includes Medical, Dental, and Vision benefits, 401k with company matching, and life insurance.

Rate: $75.00 to $85.00/hr. w2

Responsibilities:
  • Lead design, implementation, and optimization of large-scale data processing using Scala, Spark, SQL, and modern data platforms for attribution.
  • Design and operate trusted data pipelines handling advertiser, customer, and measurement datasets across AWS, GCP, Azure, and approved ecosystems.
  • Collaborate with Product, Data Science, Security, Privacy, and Platform Engineering to deliver privacy-preserving attribution, measurement, forecasting, and analytics.
  • Own scalable batch and streaming workflows using orchestration frameworks and cloud-native services.
  • Implement data classification, access controls, and privacy-preserving techniques aligned with security and compliance requirements.
  • Drive clean-room and trusted data-sharing environments, exposing only aggregated or privacy-protected outputs.
  • Build observability, monitoring, alerting, and operational tooling for reliability, performance, and compliance.
  • Troubleshoot complex platform, performance, and pipeline issues across distributed systems.
  • Influence technical design, architecture, and best practices in partnership with senior engineering leadership.
  • Mentor engineers, lead design and code reviews, and provide technical leadership.
  • Ensure all sensitive processing occurs within controlled, auditable boundaries with no unintended egress of PII or proprietary signals.
  • Define data handling standards and document trust boundaries, data contracts, lineage, and permitted movement between zones.
  • Apply privacy-preserving computation, including aggregation-before-export, pseudonymization, tokenization, differential privacy concepts, and privacy-aware reporting.
  • Implement encryption, key management, and secure handling using cloud-native security and governance services.
  • Support audits, compliance, governance reviews, and secure data-sharing initiatives.
Experience Requirements:
  • 8+ years in Data Engineering with strong Scala and extensive Apache Spark on AWS and/or GCP.
  • Strong Python for pipelines, tooling, automation, and infrastructure modules.
  • Advanced SQL across relational, cloud warehouses, and lakehouse platforms handling TB-scale datasets.
  • Design, build, and maintenance of batch and streaming pipelines.
  • Data warehousing, dimensional modeling, data quality, partitioning, and performance optimization.
  • Distributed data processing and lakehouse architectures such as Databricks, Delta Lake, or Apache Spark.
  • Operating distributed data platforms at scale with orchestration tools like Airflow, Databricks Workflows, or AWS Step Functions.
  • Git-based workflows and automated testing frameworks.
  • Cloud-native development on AWS and/or GCP with CI/CD, code reviews, observability, and production support.
  • Proven technical leadership, mentorship, and delivery within tight timelines.
  • Trusted environment execution: clean rooms or secure data-sharing platforms, handling PII and regulated data, fine-grained access controls, governance policies, and policy enforcement.
  • Familiarity with tokenization, pseudonymization, aggregation-before-export, and differential privacy concepts.
  • Experience with measurement, attribution, audience analytics, or privacy-preserving reporting solutions.
  • Data lineage and governance tooling such as Unity Catalog, AWS Glue Data Catalog, Apache Atlas, or OpenLineage.
  • Understanding of trust boundaries, secure data-sharing patterns, and zero-trust data architecture.
  • Experience documenting data contracts, flows, lineage, and permitted data movement between zones and domains.
  • Encryption, key management, and secure handling of sensitive data with cloud-native security services.
  • Design of observability and alerting to detect anomalous movement, policy violations, and potential leakage.
  • Experience in environments where only aggregated, anonymized, tokenized, or privacy-protected outputs may leave trusted boundaries.
  • Strong written and verbal English communication and experience in Agile/SCRUM.
  • Good to have: Databricks, AWS Clean Rooms or PETs, advertising and retail media platforms, collaboration with Security/Privacy/Risk/Compliance, ELK/Grafana/OpenTelemetry, Docker/Kubernetes, and secure design reviews.

Recruitment Transparency Notice

 

Eliassen Group values transparency in our recruitment practices. Please be advised that Eliassen Group utilizes artificial intelligence (AI) tools as part of its initial application screening  and hiring process. You may receive email and SMS notifications from the Eliassen Virtual Recruiting Team ( View email address on ziprecruiter.com , View phone number on ziprecruiter.com) inviting you to complete a brief voice screening as part of your application process. These tools assist our hiring teams in different ways, including but not limited to, assistance in reviewing application materials to help identify candidates whose qualifications most closely match the requirements of the position. All AI-assisted evaluations and responses are reviewed by human recruiters before any hiring decisions are made. The use of AI in our process is intended to support fairness, efficiency, and consistency, and Eliassen Group takes measures to prevent bias or discrimination in connection with its hiring practices. By proceeding, you acknowledge, agree, and consent to Eliassen Group’s use of these tools, including AI tools, as part of the application and hiring process.

Skills, experience, and other compensable factors will be considered when determining pay rate. The pay range provided in this posting reflects a W2 hourly rate; other employment options may be available that may result in pay outside of the provided range. W2 employees of Eliassen Group who are regularly scheduled to work 30 or more hours per week are eligible for the following benefits: medical (choice of 3 plans), dental, vision, pre-tax accounts, other voluntary benefits including life and disability insurance, 401(k) with match, and sick time if required by law in the worked-in state/locality. If anyone reaches out to you about an open position connected with Eliassen Group, please ensure that you are working directly with us by confirming the following:

· When you work with Eliassen Group, all email communication will come from an Eliassen.com address, never Gmail, Yahoo, etc.

· Eliassen Group will never ask you for personal information (home address, bank account, or check routing number) until you have worked with someone clearly associated with Eliassen Group.

If you have any indication of fraudulent activity, please contact  View email address on ziprecruiter.com.

About Eliassen Group:

 

Eliassen Group is a strategic consulting firm that helps organizations reach further and achieve more through our technology, business advisory, and life sciences solutions. For nearly 40 years, we have combined exceptional people, deep domain expertise, and intelligent capabilities to expand our clients’ capacity and accelerate meaningful outcomes. We are driven by a purpose to positively impact the lives of our employees, clients, consultants, and the communities we serve.

 

Eliassen is committed to building a diverse and inclusive team from a variety of backgrounds, perspectives, and skills. We are an Equal Opportunity and Affirmative Action Employer and all employment decisions are based on merit, performance, and business needs. Eliassen does not discriminate on the basis of race, color, gender identity or expression, sexual preference or orientation, sex (including pregnancy, childbirth, and related medical conditions), marital status, creed, religion, physical or mental disability, genetic information, military or veteran status, age, ancestry, national origin, citizenship status, prohibited criminal record inquiries of applicants and employees, or any other category protected by federal, state, or local laws.

 

Don’t miss out on our referral program! If we hire a candidate that you refer us to then you can be eligible for a $1,000 referral check!

 

Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in Chicago, IL vacancy
  •  ...Overview DecisionPoint seeks a Lead Data/ETL Engineer to provide technical leadership and hands-on engineering for data ingestion, transformation, integration, and delivery under the USMEPCOM ENLIST program. This SME-level position will maintain and optimize SQL... 
    Suggested
    Contract work
    For contractors
    Local area
    Remote work

    Decisionpoint Corporation

    Chicago, IL
    21 hours ago
  • $92k - $138k

    Lead Data Engineer Lead II - Data EngineeringWho We Are:Born digital, UST transforms lives through the power of technology. We walk alongside our clients and partners, embedding innovation and agility into everything they do. We help them create transformative experiences... 
    Suggested
    Full time
    Temporary work
    Part time
    Work at office
    Local area
    Remote work
    Flexible hours

    UST Global

    Chicago, IL
    2 hours ago
  • $125k - $207k

     ...Lead Data Engineer As a Lead Data Engineer, you will design and implement solutions to ingest, store, transform, and distribute big data on Azure Cloud. Our developers use Databricks (Spark, SQL, PySpark), Databricks Workflows, Python, JSON, and SQL to build products... 
    Suggested

    84.51°

    Chicago, IL
    3 days ago
  • $90k - $200k

     ...Job Description CapTech Machine Learning Engineers are responsible for designing and implementing data-driven solutions for our clients, with a specific...  ...our clients. Specific responsibilities for the Lead Machine Learning Engineer position include:... 
    Suggested
    Work at office
    Remote work
    Visa sponsorship
    Work visa
    Flexible hours

    CapTech Consulting

    Chicago, IL
    3 days ago
  • $137.7k - $229.5k

    Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make a significant...  ..., and fuel growth through innovation.Work You’ll DoAs a Lead Data Engineer II on the team, you will be responsible for:Support... 
    Suggested
    Local area

    Deloitte

    Chicago, IL
    3 days ago
  • $180k - $220k

     ...Role DescriptionWe are seeking a Lead Data Engineer to drive the design, architecture, and delivery of scalable data platforms, AI-powered data products, and enterprise-grade data services. This role goes beyond hands-on engineering to include technical leadership, architectural... 
    Full time
    Work at office

    Harbor Funds

    Chicago, IL
    4 days ago
  • DescriptionThe Data & AI Platform Engineer at Sabert Corporation is a hybrid role that plays a strategic and hands-on role at the intersection of...  ...our benefits click here.Learn More About Sabert:We are a leading global manufacturer of innovative food packaging products... 

    Sabert

    Chicago, IL
    3 days ago
  • $186k - $222k

     ...William Blair, the Premier Global Partnership.We are looking for a Lead AI Engineer to join a newly formed AI Innovation Function, part of the...  ...deployment patterns. You will work across the full stack—from data pipelines and ML models to LLM orchestration and Salesforce... 
    Temporary work

    William Blair

    Chicago, IL
    4 days ago
  • $140k - $175k

    Position SummaryThe Lead Azure Data Engineer will be reporting to VP of Data Engineering and will be responsible for implementing data solutions and collaborating with Business Analyst, Data Analyst, IT Teams and key stake holders. This role will lead projects and coach... 
    Full time
    Work experience placement
    Work visa

    Ryan Specialty Group

    Chicago, IL
    1 day ago
  • $250k

     ...-ups across industries, and partner with leading global research organizations and universities...  ...From optimizing our core trading engine to building custom hardware, we leverage...  ...business needs. What You'll Do: Our data engineering internship is an intensive 10... 
    Full time
    Work experience placement
    Summer work
    Internship
    Visa sponsorship
    Work visa

    Jump Trading

    Chicago, IL
    4 days ago
  • $27 - $42 per hour

     ...mindset, strong communication skills, advanced critical thinking, and the ability to navigate new challenges with confidence. As a Data Analytics Developer in Crowe’s Consulting Practice , your role will focus on designing and implementing data visualization and... 
    Hourly pay
    Internship
    Local area
    Immediate start
    Worldwide

    Crowe

    Chicago, IL
    21 hours ago
  •  ...-day direction to a blended onshore/offshore engineering team. Design and develop Databricks-based batch and event-driven data solutions, including streaming ingestion, event...  ...model development and deployment. Lead design reviews, establish engineering standards... 
    Full time
    Work at office

    84.51˚

    Chicago, IL
    6 hours ago
  • $122.4k - $228k

     ...Application Deadline: 09/29/2026 Address: 320 S Canal Street Job Family Group: Data Analytics & Reporting Lead Data Engineer, Applied AI Data Ingestion & Integration (DII) team Team Overview We accelerate BMO’s AI journey by building enterprise-grade, cloud-native capabilities... 
    Full time
    Contract work
    Part time
    Local area

    Bank of Montreal

    Chicago, IL
    5 days ago
  • $155k - $175k

     ...Walker Sands is one of the fastest-growing integrated B2B marketing agencies in the world. We are looking for a Lead Data Engineer to help fuel that continued growth. The starting salary for this role is expected to be between $155,000 and $175,000 per year. Walker Sands... 
    Remote job
    Summer work
    Flexible hours
    Day shift

    Walker Sands

    Chicago, IL
    2 days ago
  • $125k - $207k

    84.51° Overview: 84.51° is a retail data science, insights and media company. We help The Kroger Co., consumer packaged goods...  ..._______________________________________________________   Lead Data Engineer (P3999) SUMMARY:   As a Lead Data Engineer, you will design... 
    Work at office

    84.51°

    Chicago, IL
    17 hours ago
  • $94.4k - $293.8k

     ...About Accenture: Accenture helps the world's leading enterprises reinvent by building their digital core...  ...Industry and Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve... 
    Full time
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture Global

    Chicago, IL
    5 days ago
  • $140k - $190k

     ...employee engagement across the enterprise.Join our team as the expert you are now and create your future.We are looking for a Lead Data Engineer to provide technical leadership for our engineering team and take on one of our most strategically important challenges:... 
    Full time
    Local area
    Flexible hours

    Huron Consulting Group

    Chicago, IL
    4 days ago
  •  ...by designing and implementing a scalable data architecture that enables better decision...  ...This hands-on individual contributor will lead the design and implementation of Guidant'...  ...technology initiatives.  Data Engineering & Architecture  ~ Design and implement Guidant... 
    Full time

    Guidant Power

    Chicago, IL
    a month ago
  •  ...Title: Senior Data Engineer (Python, AWS, Spark, Unix/Linux) Employment Type: Full-Time Location: Riverwoods, IL (Chicago Area)...  ...Senior Data Engineer (Python, AWS, Spark, Unix/Linux) to join a leading Financial Services organization. This role is ideal for professionals... 
    Full time
    Flexible hours

    Capgemini

    Chicago, IL
    11 days ago
  • $130k - $160k

     ...healthcare experiences at scale. Job Description The Senior Data Engineer designs, builds, and operates data pipelines and platform...  ..., performance tuning, and cost optimization in BigQuery. Lead development of efficient ELT pipelines using dbt Cloud, Python... 

    The Aspen Group

    Chicago, IL
    a month ago
  •  ...Job Title: Senior Data Engineer (Databricks) Primary Location: Chicago, IL (Hybrid – 3 days onsite / 2 days remote) Position Type:...  ...experience WHAT YOU'LL DO. (SKILLS USED IN THIS POSITION) Lead the design and build of scalable data pipelines and models in... 
    Full time
    Remote work
    3 days per week

    BinaryBees Business Solutions LLC

    Chicago, IL
    4 days ago
  •  ...Position: Data Engineer- Snowflake Location: Chicago, IL or Peoria, IL Travel required Duration: 6- 12 months Contract US Citizen/Green Card who are willing to work on W2 are advised to apply Job Description: ~ Looking for 12+ years in data engineering... 
    Contract work

    Delta System & Software, Inc.

    Chicago, IL
    17 hours ago
  •  ..., IL V-Soft Consulting is currently hiring for a Senior Data Center Engineer for our premier client in Chicago, IL. Education and Experience...  ...professional experience ~7 years of managing experience leading IT engineering teams and environment ~10 years of... 
    Currently hiring
    Local area

    V-Soft Consulting Group, Inc.

    Chicago, IL
    24 days ago
  •  ...I'm working with a top-tier hedge fund hiring multiple Data Center Operations Engineers in Chicago and Austin . Exceptional compensation + bonus 5 days onsite Mission-critical environment supporting some of the industry's most advanced infrastructure... 

    Private Company

    Chicago, IL
    11 days ago
  •  ...Data Centre Engineer, Low Latency Trading Location: Chicago, IL Type: Full time, Permanent Team: Infrastructure Operations / Trading Technology / Linux A leading global trading firm is seeking a hands on Data Centre Engineer to support the low latency infrastructure... 
    Permanent employment
    Full time
    Remote work
    Relocation
    Flexible hours

    Autonomai Recruitment

    Chicago, IL
    3 days ago
  •  ...Location: Chicago, IL (On-site) Employment Type: Internship Role Description This Financial Data Engineering Intern role at Solar Digital Agro LLC is an on-site position based in Chicago, IL. The intern will support the development and maintenance of financial... 
    Internship

    Solar Digital Agro LLC

    Chicago, IL
    a month ago
  • $150k - $175k

     ...Senior Data & AI Engineer | Python, Snowflake, dbt & Machine Learning Location: Downtown Chicago, IL | Hybrid, 3 Days Onsite Per Week Employment...  ...@talentfish.com. Lorie Gironda Senior Recruiting Lead | TalentFish ✉️ ****@*****.*** Stay ahead of... 
    Full time
    Remote work
    Visa sponsorship
    2 days per week
    3 days per week

    TalentFish

    Chicago, IL
    17 hours ago
  • $99.83k - $135k

     ...as internal equity and alignment with market data Job Summary: The Senior Research Facility Data Management Engineer is a senior-level individual contributor responsible...  ..., sensitivity, and collaboration needs. Lead technical discovery and requirements gathering... 
    Work at office

    Northwestern University

    Evanston, IL
    3 days ago
  • $175k - $225k

    We are seeking a dedicated and experienced Data Engineer to join our Chicago team. The ideal candidate is energized by working in a cutting-edge environment that enables IMC to continue to be at the forefront of the evolving global financial markets. Your Core Responsibilities... 
    Permanent employment
    Full time

    IMC

    Chicago, IL
    4 days ago
  •  ...Overview DecisionPoint seeks a Journeyman Data/ETL Engineer to develop and maintain extract, transform, and load workflows supporting the USMEPCOM ENLIST program. This position will sustain SQL Server Integration Services (SSIS), build approved integrations, and... 
    Contract work
    For contractors
    Local area
    Remote work

    Decisionpoint Corporation

    Chicago, IL
    21 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!