Data Engineer

Role Description

Our data engineering team is the foundation of this system, ensuring data is accurate, fast, and always available for our models and AI applications. As a Data Engineer, you’ll design, build, and maintain the data pipelines that power our deep learning, video and LLM systems. You’ll work across ingestion, transformation, and orchestration layers — from real-time feeds to analytics-ready datasets. Your mission is to make data reliable, discoverable, and scalable for use by model training, analytics, and AI-driven products across multiple sports. You’ll collaborate closely with our MLOps and Sports Data teams to ensure seamless integration between data and AI.

Responsibilities

• Build and operate robust data pipelines for ingestion, cleaning, and transformation using Databricks, Airflow, or Kubernetes.
• Develop efficient ETL/ELT workflows in Python and SQL to support both batch and streaming workloads.
• Partner with ML/AI teams to make datasets and tools discoverable and safe for autonomous agents, including evaluation and guardrails for AI-generated queries.
• Develop retrieval pipelines (RAG, vector search) over structured stats and unstructured sources (scouting notes, video metadata) to power AI applications.
• Model and maintain structured data assets (Delta, Parquet, Iceberg) for reliability, versioning, and lineage tracking.
• Implement orchestration and monitoring: schedule jobs, track dependencies, and automate recovery from failures.
• Ensure data quality and compliance through validation frameworks, schema enforcement, and audit logging.
• Contribute to data platform evolution: evaluate tools, standardize best practices, and improve developer experience.
• Support performance and cost optimization across compute, storage, and orchestration systems.

Qualifications

• 3–8 years of experience as a Data Engineer or ETL Developer in a production environment.
• Proficiency in Python and SQL; strong familiarity with Databricks, Spark, or equivalent big-data frameworks.
• Experience with workflow orchestration tools such as Airflow, Dagster, Luigi or Prefect.
• Deep understanding of data modeling, data warehousing, and distributed data processing.
• Knowledge of modern data lakehouse architectures.
• Familiarity with CI/CD, GitHub Actions, Infrastructure as Code, and data pipeline testing frameworks.
• Comfort working in a cross-functional environment with ML, product, and analytics teams.
• Exposure to LLM-powered data tools: text-to-SQL, RAG, agent/tool interfaces (e.g. MCP), or natural-language analytics.
• Previous work with cloud infrastructure (AWS, GCP, or Azure) and container orchestration (Docker, Kubernetes).

Preferred

• Previous experience with sports, telemetry, or sensor data pipelines.
• Familiarity with streaming frameworks and event driven data processing (Kafka, Spark Structured Streaming, Flink).
• General knowledge of American football, the NFL, and college football.
• Background in data governance, lineage, and observability tools (Monte Carlo, Great Expectations, Unity Catalog, OpenLineage).
• Experience designing semantic layers or metric definitions consumed by AI and BI tools.
• Exposure to best practices in machine-learning model management and MLOps.

Benefits

• Competitive Salary and Bonus Plan
• Comprehensive health insurance plan
• Retirement savings plan (401k) with company match
• Remote working environment
• A flexible, unlimited time off policy
• Generous paid holiday schedule - 13 in total including Monday after the Super Bowl