Data Engineer II

Responsibilities Design, craft and own high quality foundational datasets to produce reliable, meaningful inputs for machine learning models and subscriber experiences Source, analyze and validate diverse upstream data sources from upstream partner teams and raw signals using exploratory data analysis Identify and build new user and content data features for recommendation systems through collaboration with machine learning engineers Enhance existing feature quality through experimentation, analysis and iterative development Build, optimize and maintain large scale data processing jobs, pipelines and workloads Improve data quality and reliability through implementing checks, anomaly detection, testing, monitoring and alerting Contribute to operational excellence by following observability best practices and participating in an on-call rotation to support critical production data pipelines Requirements Bachelor’s degree in Computer Science (or related field), or equivalent work experience 3 years of experience working with large scale data workloads, building and curating datasets and developing data pipelines and processing jobs Strong programming skills in Python, Java, or comparable object-oriented language Experience working with Databricks or equivalent large-scale data processing platforms Demonstrated knowledge of operating within a Public Cloud Provider (e.g. AWS, Microsoft Azure, Google Cloud) Experience working with source control systems and CI/CD pipelines Strong grasp of computer science fundamentals (data structures, algorithms, design patterns, etc.) Core Competencies Demonstrates expertise in designing and maintaining large-scale data processing pipelines and datasets, with a strong foundation in programming and cloud technologies. Proficient in enhancing data quality and reliability through rigorous testing and monitoring practices. Highest-signal resume keywords Python Programming Data Pipeline Development Large Scale Data Processing Cloud Computing (AWS, Azure, Google Cloud) Databricks Experience Hard Skills Data Analysis Exploratory Data Analysis Feature Engineering Anomaly Detection Data Quality Improvement Data Structures Algorithms Design Patterns Industry Keywords Machine Learning Models Recommendation Systems Observability Best Practices Operational Excellence Tools & Technologies CI/CD Pipelines Source Control Systems Data Processing Platforms #J-18808-Ljbffr