Lead Data Engineer

Qualifications 10+ years of hands‑on data engineering experience, including technical leadership of data platform initiatives Strong, hands‑on expertise with the Databricks Lakehouse Platform, including Delta Lake, Delta Live Tables, Databricks Workflows, and Unity Catalog Proven experience designing and implementing the medallion (bronze/silver/gold) architecture for data ingestion, curation, and consumption Strong expertise in distributed data processing using Apache Spark (PySpark and Spark SQL) or equivalent big‑data frameworks Proven experience designing and building ETL/ELT pipelines for both batch and streaming data at scale Expert‑level proficiency in Python and SQL for data transformation, validation, and pipeline development Strong experience building and managing data lakes and data warehouses, including dimensional and lakehouse data modeling Working knowledge of Large Language Models (LLMs) and GenAI concepts — prompts, embeddings, vector databases, and Retrieval‑Augmented Generation (RAG) — and the data pipelines required to support them Hands‑on experience with at least one major cloud platform (AWS, Azure, or GCP) and its core data services Proven experience leading large‑scale data migrations from on‑premise (e.g. Hadoop) and cloud platforms to modern data architectures Strong understanding of distributed data processing, partitioning, and performance optimization techniques Experience implementing data security, governance, lineage, and access control (e.g. IAM, encryption, cataloging) Strong understanding of object‑oriented programming, software design patterns, and CI/CD practices Familiarity with Agile/Scrum delivery methodologies and experience mentoring engineers Excellent problem‑solving, analytical, and stakeholder‑management skills Strong verbal and written communication skills #J-18808-Ljbffr