Lead Data Engineer

Lead Data Engineer (Frisco, TX & Bellevue, WA) Data Platform & Near Real‑Time Analytics Role Overview We are seeking a Lead Data Engineer to design and build a scalable, high‑quality data platform that ingests data from multiple sources, ensures data quality and governance, and delivers near real‑time insights (15‑minute SLA) through Power BI / Microsoft Fabric dashboards. This role will provide technical leadership and drive end‑to‑end data engineering architecture and delivery. Key Responsibilities Architecture & Platform Design Design and implement end‑to‑end data platform architecture (ingestion, processing, storage, serving). Define batch and near real‑time data pipelines ensuring low‑latency and high reliability. Make technology decisions across Databricks, Delta Lake, Snowflake, and Azure ecosystem. Data Engineering & Pipelines Build scalable pipelines using Databricks / Spark for large‑scale processing. Use Delta Lake for ACID‑compliant data storage. Use Snowflake for data warehousing and analytics. Implement ETL/ELT pipelines with strong data modeling practices. Streaming & Real‑Time Processing Design and implement real‑time pipelines using Kafka / Azure Event Hubs. Ensure data freshness within ~15‑minute SLA. Enable incremental processing and efficient data updates. Data Quality & Governance Establish data quality frameworks (validation, completeness, consistency checks). Implement monitoring, alerting, and data observability. Define and enforce data governance, lineage, and metadata standards. Data Serving & Analytics Enable optimized data layers for Power BI / Microsoft Fabric dashboards. Design semantic models and curated data layers for business consumption. Ensure consistent, accurate, and high‑performance reporting. Performance & Scalability Optimize pipelines and storage for large‑scale datasets (TB/PB). Ensure low‑latency query performance and efficient compute usage. Implement partitioning, indexing, caching, and optimization strategies. Leadership & Collaboration Lead and mentor a team of data engineers. Collaborate with Technical Product Managers, BI teams, and business stakeholders. Drive best practices in coding, architecture, and delivery. Manage technical risks, dependencies, and roadmap execution. Required Skills Strong experience in data engineering and platform architecture (Lead level). Expertise in Databricks, Spark, Delta Lake, Snowflake or similar cloud data warehouses. Hands‑on with streaming technologies (Kafka / Event Hubs). Strong knowledge of data modeling, ETL/ELT, and pipeline design. Experience with data quality frameworks and monitoring tools. Familiarity with Power BI / Microsoft Fabric. Strong programming skills (Python, SQL). Experience with Azure ecosystem (ADF, ADLS, AKS – preferred). Nice to have: Experience with real‑time analytics platforms. Exposure to data governance / MDM frameworks. Familiarity with CI/CD and DevOps practices for data platforms. Key Expectations Own and deliver a robust, scalable data platform. Ensure high data quality and near real‑time availability (15‑minute SLA). Drive standardization, reusability, and performance optimization. Enable business‑ready, trusted data for analytics and decision‑making. Business Impact Build and scale a modern data platform that delivers trusted, near real‑time insights, enabling faster decisions and powering analytics across the organization. #J-18808-Ljbffr Apptad Inc