Senior Data Engineer

Decision Science And Integration Team MemberThe Decision Science and Integration (DDSI) team leverages advanced technologies, data analytics, and scientific approaches such as optimization, and statistical modeling to build innovative tools that shape business decisions across the company. We support Entertainment, ESPN, Experiences, and Corporate Finance with strategic applications that enable data-driven decision-making. The Data Engineering team is responsible for development and sustainment projects, ad-hoc requests, prototyping and research initiatives by providing data pipeline and data engineering services. You will work with business, science, and technology teams to understand scope and requirements, business needs, and data from across the company in order to design and deliver the data pipelines necessary for our solutions. You will have tasks to design, develop, enhance, and implement systems for advanced analytic data solutions. This includes data aggregation and manipulation to make the data usable by science models, visualizations, and applications. You will be expected to leverage GenAI tools to be efficient in your work, and you will be delivering AI-based solutions. You will also support the exploration and integration of GenAI tools and techniques into data engineering processes to enhance automation and analytical capabilities. Other activities could include planning, estimating, design, development, testing, production rollout, and sustainment activities. In addition to technical capabilities, the role is responsible for developing an understanding of the business domain and processes, then applying that knowledge to the assigned deliverables.Work assignments may cover activities such as participation in data requirements gathering, source-to-target mapping, data validation scripting and review, developing and monitoring ETL/ELT data pipelines, and producing datasets as input to science models and visualizations. As part of a data engineering team, this role will be included in technical reviews of the team to ensure quality deliverables. This role communicates data engineering progress to the project leadership team and actively participates in meetings and discussions.Responsibilities:Work assignments may cover activities such as participation in data requirements gathering, source-to-target mapping, data validation scripting and review, developing and monitoring ETL/ELT data pipelines, and producing datasets as input to science models and visualizations.As part of a data engineering team, this role will be included in technical reviews of the team to ensure quality deliverables.This role communicates data engineering progress to the project leadership team and actively participates in meetings and discussions.Experience:5 years of technology experienceSkillsets:Strong experience with ELT/ETL data pipeline development and maintenanceProven expertise using Python and SQLStrong understanding of database and data warehouse design, and proficiency utilizing various data platforms, such as PostgreSQL, Snowflake, etc.Solid understanding of GenAI concepts and tools (e.g., LLMs, embeddings, prompt engineering)Understanding of vector databases or RAG (Retrieval-Augmented Generation) architecturesExperience working with GenAI APIs or frameworks (e.g., OpenAI, LangChain, AWS Bedrock)Experience with developing in a multi-environment (Dev, QA, Prod, etc.) and DevOps procedures for code deployment/promotion using a source control product such as GitLab or GitHub.Hands-on knowledge of job orchestration software like Apache AirflowExperience and understanding of one or more business domains to assist in gathering/refining data requirements and data design solutionsExperience managing and deploying code using a source control product such as GitLab/GitHubAble to effectively formulate solutions and communicate complex technical concepts to non-technical team membersProven ability to work collaboratively in a multi-team environmentEducation:Bachelor's degree (Computer Science, Mathematics, Software Engineering or related field)Additional Qualifications:Experience with analytic systemsExperience with infrastructure as Code, Docker and containerization and elastic scaling with Kubernetes or a similar framework and with AWSExperience working with large datasets and big data technologies such as Snowflake, Databricks, or similarDemonstrated proficiency with API developmentKnowledgeable on cloud architecture and product offerings, preferably AWS.