Sr. Data Engineer

Job Description

Job Description

Job Title: Senior Data Engineer (Databricks)
Primary Location: Chicago, IL (Hybrid – 3 days onsite / 2 days remote)
Position Type: Full-Time, Direct Hire

OVERVIEW

TalentFish is casting a line for a Senior Data Engineer. This is a full-time, direct hire role in Chicago, IL, working a hybrid schedule with a minimum of three days onsite per week. This role exists as part of a multi-year initiative to transform our client's Data & AI function, building a modern data platform that enables speed-to-insight and makes the organization truly data-driven.

You'll be one of the core builders of a brand-new Azure Databricks platform, designing and delivering the pipelines, transformations, and models that everything else runs on. The environment is greenfield: Databricks is stood up, but no production pipelines exist yet, so you'll help set the standards from day one. This is a hands-on senior individual contributor role, not a people management, architecture, AI/LLM, or BI/reporting role. The architecture is defined; your job is to build it well and raise the technical bar of a growing team (expanding to 25–30) that is largely BI-focused today.

The work carries strong executive sponsorship and enterprise-wide visibility, with the chance to influence engineering practices across the organization. The interview process includes a live technical assessment covering Python, SQL, and practical data pipeline logic.

WHAT YOU BRING TO THE ROLE. (IDEAL EXPERIENCE)

• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field
• 6 years of hands-on data engineering experience, with a track record of owning pipelines end to end: ingestion, validation, transformation, and governance
• Real, hands-on Databricks experience (Azure preferred; AWS or GCP acceptable), including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, and Workflows/Jobs
• Strong coding skills in Python, SQL, and Apache Spark that go well beyond basic scripting
• Proven experience designing and building scalable ETL/ELT pipelines, including reusable, metadata-driven pipelines
• Experience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelines
• Solid data modeling, schema design, and performance tuning experience
• Working knowledge of Git-based workflows, CI/CD, test automation, and monitoring and alerting
• Strong troubleshooting skills and a solid grasp of engineering fundamentals: scalability, reliability, and maintainability
• The ability to clearly explain what you built, why it mattered, and how it solved a business or platform problem

Preferred

• Experience building or migrating a data platform from the ground up
• Infrastructure as Code (Terraform, Bicep, ARM) and Governance as Code experience
• Multi-cloud background with depth in Azure
• Legacy data warehousing and/or Hadoop experience
• Deep Databricks exposure beyond basic pipelines, such as enterprise-scale work and optimization
• A track record of mentoring and upskilling peers
• Agile delivery experience

WHAT YOU'LL DO. (SKILLS USED IN THIS POSITION)

• Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yet
• Own pipelines end to end, from ingesting source systems to transforming and modeling data to producing outputs for downstream analytics, BI, and AI/ML use cases
• Refactor and migrate multiple legacy data warehouses into the centralized Databricks lakehouse
• Work at the code level every day in Python, SQL, and Spark, building pipelines with Databricks Asset Bundles, Spark workflows, and API-driven jobs
• Establish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring, and alerting
• Help shape and enforce standards for data modeling and governance
• Troubleshoot and optimize performance across the platform
• Mentor and coach junior engineers, conduct code reviews, and lead by example in design, coding, and testing practices
• Partner with Data Engineering, BI, AI, Finance, and product owners, and help translate technical work into business impact

COMPENSATION INFORMATION

The expected salary range for this position i s $150,000-$164,000 per year, depending on experience and qualifications. This role also qualifies for comprehensive benefits such as health insurance, 401(k), and paid time off. TalentFish is committed to pay transparency and equal opportunity. The salary range provided is in compliance with applicable state and federal regulations.
This role requires authorization to work in the U.S. without current or future visa sponsorship.
All offers are contingent upon the completion of a background check, which may include but is not limited to reference checks, education verification, employment verification, drug testing, criminal records checks, and any required certifications or compliance requirements based on the end client's background check policies and applicable laws.
TalentFish is an employee-owned company pioneering a new realm in talent acquisition. We are redefining IT staffing by evolving AI, video screening, and our unique platform. TalentFish focuses on providing the best employee, consultant, and client experience possible. At TalentFish we are an Equal Opportunity Employer; we embrace and encourage diversity!