Senior Data Engineer

I am partnered with a Real Time Intelligence Company, who have built a platform that uses AI to transform publicly available data into real time operational insights. With a huge part of their business growing, they are now looking to hire a Senior Data Engineer, who will be an integral part of the team, working on new and exciting projects, building new pipelines, migrating from one service to another service and building a data and analytics infrastructure. Responsibilities Build and maintain data pipelines for large scale data ingestion, real time analytics and Machine Learning Services Develop and optimize NLP, LLMs and other AI powered applications Write CI/CD pipelines and have the ability to do code reviews Design and build a data analytics infrastructure Qualifications Professional Experience in using Apache Storm, Kafka or Flink for real time data or streaming 5+ years experience in Data Engineering and building pipelines using Databricks and Spark Can demonstrate experience in building CI/CD pipelines and DevOps best practises Experience with real time data streaming #J-18808-Ljbffr