Senior/Lead Data Engineer

Data Engineers – Global Investment Firm | New York & London

Apply now, read the job details by scrolling down Double check you have the necessary skills before sending an application.

Build the data platform behind trading, research and risk at a top-tier investment firm.

My direct client, a premier global investment firm is investing heavily in its data. The CTO has a big mandate to modernise the stack and build a more robust platform, and the team is hiring urgently. The team is small and growing, so your work will be visible and you'll have a direct say in the architecture. I placed a key person on this team earlier this year who is doing well and loves the environment and work.

New York: four openings. One Lead Data Engineer and three mid/senior individual contributors. London: one mid/senior individual contributor, working closely with the New York team.

What you'll work on

• Data pipelines for trading, alpha generation, research, risk and accounting
• Golden-source datasets the whole firm relies on: security master, account master and price master
• Shared Python libraries for data APIs, logging and core functions
• The firm's AI tools, including LLM search and summarisation of financial documents and market commentary
• Data quality, observability and governance using modern tools
• Close work with quants, traders, risk managers, the Chief Data Officer and the Head of Development

Must-haves

• Hands-on experience with both Snowflake and Databricks
• Excellent Python (including PySpark) and SQL, plus Spark, dbt and AWS
• 3+ years of development experience, with at least 2 years focused on data engineering
• Deep ETL/ELT experience, strong data modelling, and SQL performance tuning (PostgreSQL or SQL Server)
• Docker and Kubernetes in cloud environments
• Some finance experience. A trading firm or hedge fund is ideal, or another fast-paced financial services firm.
• A degree in Computer Science or a related field

Nice to have

• A software engineering background as well as data engineering. The hiring manager values varied technical experience.
• Data catalogue and data lineage experience (for example OpenMetadata or Great Expectations)
• LangChain/LangGraph, vector databases and RAG
• Market data knowledge across equities, fixed income, futures and options
• FastAPI, Dask, or building BI dashboards

Why this role

• Real modernisation work with backing from a new CTO
• A small team where you'll have a lot of influence
• Bonus buy-outs possible for the right candidates

Process: a Zoom with the hiring manager, a Python-focused take-home on CoderPad, then a half-day onsite that includes a live coding session.

Onsite 4 days a week in both New York and London

Please note:

• Unfortunately, our client can't offer visa sponsorship. Candidates must be authorised to work in the US or UK without sponsorship, now and in the future.
• No third-party recruiters or agencies, please. xhyhwjd We're only considering direct applicants.