Senior Data Engineer

Type: Permanent No Sponsorship Available About the Company Our client is a global AI and data technology business focused on transforming critical industries through advanced data platforms, AI, and intelligent infrastructure solutions. They partner with governments, utilities, telecoms, healthcare providers, and enterprise organisations to deliver scalable, real-world impact across highly complex environments. The organisation operates across the full technology lifecycle, from architecture and engineering through to deployment and operational capability building, leveraging cloud-native technologies and modern data platforms. The Opportunity We’re looking for a Senior Data Engineer to design, build, and optimise scalable data platforms supporting advanced analytics, machine learning, AI, and operational intelligence solutions. This role will focus on building modern cloud-native data architectures capable of handling large-scale, multi-source data workloads across batch, streaming, and near real-time environments. You’ll work closely with Data Scientists, ML Engineers, Product teams, and senior stakeholders to deliver robust, production-grade data capabilities. The environment is highly engineering-led, with a strong focus on: Data as a Product Scalability & reliability Observability & governance Modern lakehouse architectures Distributed processing frameworks AI & advanced analytics enablement Key Responsibilities Design and develop scalable cloud-native data platforms and architectures Build robust ingestion and transformation pipelines across APIs, IoT/telemetry, legacy systems, and structured data sources Develop distributed data processing capabilities using technologies such as Spark and Kafka Implement modern lakehouse/data platform architectures supporting analytics, GIS, AI, and machine learning workloads Establish best practices around observability, monitoring, data quality, lineage, and platform reliability Design secure, governed data environments with strong access controls, encryption, and auditability Collaborate with Data Scientists and ML Engineers to operationalise AI and advanced analytics solutions Optimise platform performance, scalability, and cost efficiency Contribute to architectural strategy and mentor junior engineers within the team Support CI/CD, infrastructure automation, and cloud deployment best practices Required Experience 5+ years’ experience building and operating large-scale data platforms in cloud or hybrid environments Strong experience designing modern data architectures including lakehouse, data mesh, or equivalent frameworks Hands-on expertise with distributed processing technologies such as Spark/PySpark Experience with streaming/event-driven architectures using Kafka or similar technologies Strong Python and SQL development skills Experience working with modern data tooling such as dbt, Docker, and cloud-native platforms Exposure to AWS, Azure, or GCP environments Strong understanding of data governance, security access controls, encryption, and auditability Experience integrating complex and heterogeneous data sources Ability to design highly available, production-grade data systems Experience enabling machine learning and AI workloads through scalable data infrastructure Experience within regulated, public sector, utilities, infrastructure, or government environments Knowledge of Infrastructure-as-Code and CI/CD practices Exposure to AI/ML operationalisation and feature engineering pipelines Experience working within highly distributed, multi-stakeholder organisations Python SQL PySpark / Spark Kafka Databricks dbt Docker Personal Attributes Strong systems-thinking and problem-solving mindset Comfortable operating within fast-paced, complex environments Collaborative and highly communicative Passionate about scalable engineering and platform reliability Proactive, delivery-focused, and accountable Strong stakeholder engagement skills across technical and non-technical teams #J-18808-Ljbffr 83zero