Data Engineer II

Disney Entertainment and ESPN Product & Technology Technology is at the heart of Disney's past, present, and future. Disney Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more - all working to build and advance the technological backbone for Disney's media business globally. The team marries technology with creativity to build world-class products, enhance storytelling, and drive velocity, innovation, and scalability for our businesses. We are Storytellers and Innovators. Creators and Builders. Entertainers and Engineers. We work with every part of The Walt Disney Company's media portfolio to advance the technological foundation and consumer media touch points serving millions of people around the world. Here are a few reasons why we think you’d love working here: Building the future of Disney's media: Our Technologists are designing and building the products and platforms that will power our media, advertising, and distribution businesses for years to come. Reach, Scale & Impact: More than ever, Disney's technology and products serve as a signature doorway for fans' connections with the company's brands and stories. Disney+. Hulu. ESPN. ABC. ABC News…and many more. These products and brands – and the unmatched stories, storytellers, and events they carry – matter to millions of people globally. Innovation: We develop and implement groundbreaking products and techniques that shape industry norms, and solve complex and distinctive technical problems. Ad Platforms is responsible for Disney's industry-leading ad technology and products – driving advertising performance, innovation, and value in Disney's sports, news, and entertainment content, across all media platforms. Job Summary: We are seeking a passionate Data Engineer to join the Product Performance Team within the Data Organization at Disney ESPN & Entertainment Technology. Our team is vital, owning the development of reliable data solutions that measure the product performance of Disney+. This role involves designing, building, and testing the core algorithms and data models that generate precise and dependable browse, playback attribution, and search data. This data establishes the governed, source-of-truth solutions essential for measuring Disney+ product engagement and driving critical business decisions. We are looking for an individual excited by the challenge of solving complex business questions through the power of data. Responsibilities and Duties of the Role: Design, develop, and optimize large-scale batch and real-time data pipelines using Spark Structured Streaming on Databricks. Work with Data Analysts to develop reports and dashboards on snowflake Write production-grade, maintainable code primarily in Scala and SQL Implement complex data transformations using Spark SQL and core Spark APIs. Own the end-to-end lifecycle of data products from ingestion, transformation, orchestration, to consumption by analytics, ML, and reporting teams. Design and maintain workflow orchestration using Apache Airflow or Databricks Build and manage infrastructure on AWS with some expertise in cloud-native data architectures. Implement data quality, monitoring, alerting, and lineage solutions. Performance tuning of Spark jobs, cluster optimization on Databricks, and cost optimization on AWS. Required Education, Experience/Skills/Training: 3+ years of hands-on data engineering experience. Proficient in Spark Structured Stream and Databricks platform (Unity Catalog, Delta Lake, Workflows, Cluster management). Strong programming skills in Scala (must have) and Python. Advanced SQL skills with experience in writing complex, optimized queries. Hands-on experience building and scheduling workflows in Apache Airflow. Working knowledge of AWS services for data engineering. Solid understanding of data modeling concepts Experience with CI/CD for data pipelines (Git, Jenkins, GitHub Actions, etc.). Familiarity with Snowflake is a strong plus. Experience with streaming technologies beyond Spark (Kinesis, Flink) is a plus. Excellent problem-solving, communication, and leadership skills. Bachelor's degree in Computer Science or equivalent study required The hiring range for this position in New York City, NY is $112,000 to $150,100 per year. The base pay actually offered will take into account internal equity and also may vary depending on the candidate's geographic region, job-related knowledge, skills, and experience among other factors. A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered. #J-18808-Ljbffr Socket.dev