Senior/Lead Data Engineer (New York)

Data Engineers – Global Investment Firm | New York & London

Build the data platform behind trading, research and risk at a top-tier investment firm.

My direct client, a premier global investment firm is investing heavily in its data. The CTO has a big mandate to modernise the stack and build a more robust platform, and the team is hiring urgently. The team is small and growing, so your work will be visible and you'll have a direct say in the architecture. I placed a key person on this team earlier this year who is doing well and loves the environment and work.

New York: four openings. One Lead Data Engineer and three mid/senior individual contributors. London: one mid/senior individual contributor, working closely with the New York team.

What you'll work on

• Data pipelines for trading, alpha generation, research, risk and accounting
• Golden-source datasets the whole firm relies on: security master, account master and price master
• Shared Python libraries for data APIs, logging and core functions
• The firm's AI tools, including LLM search and summarisation of financial documents and market commentary
• Data quality, observability and governance using modern tools
• Close work with quants, traders, risk managers, the Chief Data Officer and the Head of Development

Must-haves

• Hands-on experience with both Snowflake and Databricks
• Excellent Python (including PySpark) and SQL, plus Spark, dbt and AWS
• 3+ years of development experience, with at least 2 years focused on data engineering
• Deep ETL/ELT experience, strong data modelling, and SQL performance tuning (PostgreSQL or SQL Server)
• Docker and Kubernetes in cloud environments
• Some finance experience. A trading firm or hedge fund is ideal, or another fast-paced financial services firm.
• A degree in Computer Science or a related field

Nice to have

• A software engineering background as well as data engineering. The hiring manager values varied technical experience.
• Data catalogue and data lineage experience (for example OpenMetadata or Great Expectations)
• LangChain/LangGraph, vector databases and RAG
• Market data knowledge across equities, fixed income, futures and options
• FastAPI, Dask, or building BI dashboards

Why this role

• Real modernisation work with backing from a new CTO
• A small team where you'll have a lot of influence
• Bonus buy-outs possible for the right candidates

Process: a Zoom with the hiring manager, a Python-focused take-home on CoderPad, then a half-day onsite that includes a live coding session.

Onsite 4 days a week in both New York and London

⚠️ Please note:

• Unfortunately, our client can't offer visa sponsorship. Candidates must be authorised to work in the US or UK without sponsorship, now and in the future.
• No third-party recruiters or agencies, please. We're only considering direct applicants.