Data Engineer

Описание Graphcore is a developer of hardware, software, and systems infrastructure designed to advance artificial intelligence compute and support the adoption of AI solutions across various industries. As a member of the SoftBank Group, the organization focuses on enabling artificial super intelligence and creating accessible, transformative technologies through a multidisciplinary approach involving AI research, silicon design, and systems architecture. Задачи Lead the design, build, and evolution of robust data pipelines and platform services for analytics, reporting, and operational use cases. Own the data engineering stack, planning and delivering improvements to reliability, scalability, maintainability, performance, and security. Build and operate Python‑based batch and streaming workflows, including orchestration, testing, deployment, monitoring, and incident resolution. Design and implement secure, resilient, and cost‑conscious data solutions on AWS using services such as S3, Lambda, Aurora PostgreSQL, Athena, Glue, and Redshift. Define and apply engineering standards for data quality, observability, documentation, release processes, and operational support. Partner with analysts, engineers, and business stakeholders to translate requirements into trusted datasets, data models, and reusable data products. Drive improvements to platform resilience through idempotent processing, retry and recovery mechanisms, buffering strategies, and backfill or replay capabilities. Lead technical decision‐making by reviewing designs and code, sharing expertise, and raising the quality bar for data engineering. Build and maintain CI/CD workflows and development practices for efficient delivery of data infrastructure. Ensure appropriate data protection and access controls, including least‑privilege access, secure secrets handling, and database permissions. Contribute to the development of internal tools and lightweight applications to support self‑serve workflows. Identify opportunities for platform and process improvements to shape the direction of data engineering. Требования Strong experience designing, building, and operating production‑grade data pipelines and platforms in Python. Strong hands‑on experience with modern data orchestration, testing, deployment, and monitoring practices. Experience building solutions on AWS data services including storage, processing, and query technologies. Strong understanding of data modelling, data quality, schema design, and performance optimisation across relational and analytical systems. Experience designing reliable data systems that recover from failure and operate effectively in production. Experience working with batch and streaming data pipelines, including operational support and troubleshooting. Strong knowledge of security and access control principles, including IAM, database permissions, and secure handling of credentials. Experience providing technical leadership as a senior individual contributor through design reviews, code reviews, standards‑setting, and mentoring. Ability to work effectively with technical and non‑technical stakeholders to turn business needs into scalable solutions. Strong communication skills with the ability to explain technical decisions and influence outcomes. Nice to have: Prefect, streaming technologies, PostgreSQL, Redshift, ClickHouse, CI/CD tooling, Infrastructure as Code, Streamlit, Flask, dbt, cloud cost optimisation, experience in fast‑moving engineering‑led environments. Условия No conditions specified. #J-18808-Ljbffr Enfint