Lead AI Data Engineer

The role You will own the build of two strategic initiatives: A unified operational intelligence platform — an eCommerce Health view that combines system signals (latency, errors, integration and data-flow health) with commercial performance (conversion, checkout, auto-renewal, feature adoption) into one trusted source of truth. AI enablement across engineering — shipping agents and productivity tools that engineering teams actually adopt and use every day. This is a builder's role, not a platform administration role. You will be writing Spark, designing schemas, tuning jobs, and shipping applications that stakeholders open and rely on. What you'll do Build the data foundation Bring data from eCommerce, MarTech, observability, and operational datastores into an analytics-ready and AI-ready state through production ETL/ELT pipelines, expert SQL, and deliberate data modeling — with automated quality validation built in. Design dimensional models (star/snowflake schemas, sensible partitioning and grain) purpose-built to power analytics, KPI reporting, and dashboards. Build gold-layer, summarized datasets — collapsing millions of raw events into one clean summary record per entity per day — the aggregation layer that makes natural-language querying and executive dashboards trustworthy rather than approximately correct. Make performance visible Design and ship the eCommerce Health view: one place where system health and commercial outcomes sit side by side. Surface standardized engineering metrics across teams — delivery velocity (DORA-style), quality, security posture, incident trends, and service ownership. Ship AI that reduces toil Build AI agents that detect anomalies, surface critical issues with actionable context, notify the right stakeholders, and drive intelligent triage that accelerates root‑cause analysis. Build AI productivity tools that help teams ship faster and operate more reliably — RCA assistance, intelligent search across telemetry and code, runbook copilots. Own the full lifecycle of LLM features: prompt design, retrieval and grounding, tool use, guardrails, evals, and operational reliability. Ship applications, not just tables Use Databricks as a product platform, not only a pipeline runtime — Genie, Databricks Apps, and Lakebase to deliver internal tools, chatbots, and operational apps that stakeholders interact with directly. Partner closely with Product, SRE, Security, and Architecture to translate operational pain into solutions teams adopt. Identify and spread AI use cases across engineering teams — this role has an evangelism dimension as well as a build dimension. What you'll bring Experience 8+ years in application development, including designing and architecting large-scale production applications and distributed systems. 5+ years hands‑on production data engineering — building pipelines, ETL/ELT workflows, and analytics platforms that turn raw operational and business data into decisions. A track record as a genuine domain expert on business‑critical data work, and the versatility to carry those skills from one domain and one data platform to another. Expert Spark / PySpark. Performance tuning, Spark SQL, partitioning and shuffle handling at billion‑row scale. This is the foundation the rest of the role is built on. Expert SQL at scale and strong Python. Data modeling for analysis — dimensional modeling, star/snowflake schemas, thoughtful grain and partitioning decisions. Modern lakehouse platforms, with Databricks depth strongly preferred: Delta Lake, Workflows, Unity Catalog, Medallion architecture. Real‑time and event‑driven processing. AI engineering (must‑have) Hands‑on experience shipping LLM‑powered applications: production RAG systems, AI agents, or LLM‑integrated internal tools, with sound judgment on tool use, guardrails, evals, and reliability. At least one AI agent you personally built and can walk through in detail — what it monitored, how it decided to act, what framework and model you used, and what happened when it was wrong. Daily use of AI coding assistants (Claude, Copilot, or similar) to ship real work faster — pipelines, dashboards, tooling. Domain eCommerce or a comparable operational domain — funnel metrics, subscription flows, payment processing. You understand the shape of transactional and operational data before you model it. Ways of working Efficiency‑driven impact: a demonstrable record of using AI to measurably reduce toil and accelerate incident response. Strong cross‑functional partnership across Product, SRE, Security, and Architecture. Work arrangement This is a hybrid role based in Frisco, TX. You'll be on‑site on an as‑needed basis and work from your home office otherwise. Candidates must be within commutable distance of Frisco, TX. Relocation assistance is not available for this position. Benefits and perks Bonus program and 401(k) retirement plan Medical, dental, vision, basic life, short‑term and long‑term disability coverage Paid parental leave 14 paid company holidays Unlimited paid time off for exempt employees Flexible working hours and family‑friendly benefits Community involvement and social programs Our client embraces diversity and inclusion and encourages everyone to bring their authentic selves to work. They prohibit discrimination based on race, colour, religion, gender, national origin, age, disability, veteran status, marital status, pregnancy, gender expression or identity, sexual orientation, or any other legally protected status. #J-18808-Ljbffr Syndesus, Inc.