Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Machine Learning Engineer (Training & Inference Systems)

Jobzhr

A leading global trading firm is building out its machine learning function and hiring engineers to develop the distributed training and inference systems that take models from research into production. This is the most urgent hire on a team that sits unusually close to the trading desk — you'll feel more like part of the trading floor than a traditional ML function.

What you'll do

  • Build and scale distributed training systems for large models
  • Develop low-latency inference pipelines to get models into production
  • Work directly with researchers training the models and the traders/quants relying on them
  • Operate on a rapidly growing GPU footprint (low thousands today, scaling into the tens of thousands over the next few years)

What we're looking for

  • Strong experience building training and/or inference infrastructure at scale
  • Deep PyTorch fluency and strong performance-engineering instincts
  • Distributed systems experience — multi-GPU/multi-node training, model serving
  • A track record of hands-on building (this is an IC-heavy role, not a strategy seat)
  • Background at a top tech firm, trading firm, or a strong big-tech + startup combination
  • 3+ years of relevant experience; flexible on level for the right person

Why it's exciting

  • Real impact, fast. Your systems directly enable production models that drive trading decisions — no research-to-nowhere, no shipping into a void.
  • Scale that's actually scaling. A GPU footprint growing 10x+ over the next two to three years, with the mandate and budget to match.
  • Proximity to the work that matters. You'll sit alongside researchers and traders, not three layers removed. Tight feedback loops, smart people, genuine ownership.
  • Top-of-market comp for engineers who can build at this level.
#J-18808-Ljbffr
Vacancy posted 11 hours ago
Similar jobs that could be interesting for youBased on the Machine Learning Engineer (Training & Inference Systems) in New York, NY vacancy
  •  ...A leading global trading firm is building out its machine learning function and hiring engineers to develop the distributed training and inference systems that take models from research into production. This is the most urgent hire on a team that sits unusually close... 
    Training
    Flexible hours

    Fintal Partners

    New York, NY
    11 hours ago
  •  ...Jobzhr, a leading global trading firm, is expanding its ML infrastructure in New York. We are hiring engineers to build distributed training and low-latency inference systems that move models from research into production. You will work closely with researchers and... 
    Training

    Jobzhr

    New York, NY
    11 hours ago
  • $229k - $343k

     ...live in the moment, learn about the world, and...  ...device and server-side inference. Our team creates intuitive...  ..., and agentic systems that empower creators...  ....We’re looking for a Machine Learning Engineer to join Snap Inc!What...  ...related fieldExperience training large-scale diffusion... 
    Training
    Full time
    Live in
    Work at office
    Local area
    Worldwide

    Snap

    New York, NY
    6 hours ago
  • $127.4k - $191.2k

     ...Vector AI team builds the machine learning systems that decide which ads reach...  ...on the world's leading game engine. Recommendation and ranking...  ...rigorous experiments using causal inference, A/B testing, and offline...  ...the full pipeline from training data to deployed model.... 
    Training
    Full time
    Internship
    Work at office
    Worldwide
    Shift work

    Unity Technologies

    New York, NY
    1 day ago
  • $200.8k - $251k

     ...company in San Francisco seeks a team member to build and optimize a machine learning framework for large language models. Candidates should have system optimization experience and solid software engineering skills, particularly in tools like CUDA and Pytorch. This full-... 
    Training
    Full time

    Scale AI

    New York, NY
    11 hours ago
  • $264.8k - $331k

     ...Machine Learning Systems Research Engineer, Agent Post-training - Enterprise GenAI AI is becoming vitally important in every function of our society. At Scale, our...  ...: Build, profile and optimize our training and inference framework. Post-train state of the art models, developed... 
    Training
    Full time
    Contract work
    For contractors
    For subcontractor
    Work at office

    SCALE INC

    New York, NY
    3 days ago
  • $212.5k - $270k

     ...and deploys state-of-the-art machine learning solutions that predict the...  ...safely and efficiently. The system architecture team handles...  ...-world problems with ML and engineering solutions. Use state of the...  ..., experience, relevant training and education, and skill level... 
    Training
    Full time
    Contract work
    Internship
    Remote work

    Waymo

    New York, NY
    1 day ago
  •  ...Job Title ML Staff Engineer - LLM & Production Systems Job ID 107242 Work...  ...generation of AI- and machine learning-powered solutions across...  ...strategies and optimize inference workloads through capacity...  .../certifications, training, and skill level.In NYC,... 
    Training
    Permanent employment
    Full time
    Work at office
    Local area
    1 day per week

    Bain & Company

    New York, NY
    4 days ago
  • $189.6k - $237k

     ...framework for large language model training and inference. The platform has been...  ...:Strong excitement about system optimizationExperience with...  ...ML systemsStrong software engineering skills, proficient in frameworks...  ..., retirement benefits, a learning and development stipend,... 
    Training
    Full time

    Scale AI

    New York, NY
    2 days ago
  •  ...Jane Street is seeking an engineer with expertise in low-level systems programming and optimisation to join our ML team. The role focuses on optimizing model performance for training and inference, with attention to scalable training, low-latency real-time inference,... 
    Training

    Trading Interview

    New York, NY
    11 hours ago
  •  ...View is hiring for a senior research-engineer role focusing on Gemini Audio. You will design, build, and maintain training infrastructure for Gemini Audio...  ...and help optimize model training and inference across distributed systems. English proficiency required; equity... 
    Training

    Google Inc.

    New York, NY
    11 hours ago
  •  ...the most valuable training data in the world...  ...how well our system understands and protects...  ...Design active-learning loops that...  ...rollout Optimize inference cost, latency, and...  ...Data and Product Engineering on pipeline and review...  ...of professional machine learning or... 
    Training
    Full time
    Shift work

    Sunset

    New York, NY
    19 hours ago
  • $184.05k - $262.93k

     ...By combining cutting-edge machine learning, recommendation systems, and product thinking,...  ...Senior Machine Learning Engineer, you will help shape the...  ...powered user experiences. Train, fine-tune, evaluate, and...  ...worked with large-scale inference systems and understand the... 
    Training
    Remote job
    Full time
    Flexible hours

    Spotify

    New York, NY
    1 day ago
  • $180k - $230k

     ...nobody stops, and billing systems designed to fight over...  ..., and it runs on machine learning at serious scale. We're hiring ML Engineer to build and own the infrastructure...  ...that powers it — from training models on tens of...  ...in seconds against inference-time datasets that run... 
    Training

    Arlo Corporation

    New York, NY
    1 day ago
  • $120k - $240k

     ...simulation software stack for engineering and manufacturing...  ...through AI inference across the entire engineering...  ...For As a Senior Machine Learning Engineer in Delivery,...  ...You’ve shipped ML systems end-to-end and at scale...  ...scalable ML systems, training and inference... 
    Training
    Full time
    Work at office
    Remote work
    Flexible hours

    Physicsx

    New York, NY
    1 day ago
  • $162k - $210k

     ...great dates! We are hiring Machine Learning Engineers to help us build the...  ...including work on recommendation systems, multi-modal embedding, and...  ...both offline and real-time inference endpoints that directly...  ...: Proven track record of training and deploying large scale ML... 
    Training
    Full time
    Work experience placement

    Match Group

    New York, NY
    1 day ago
  •  ...interpretable, and steerable AI systems. We want AI to be safe and...  ...of committed researchers, engineers, policy experts, and business...  ...cutting-edge systems that train AI models like Claude. You're...  ...to work at the frontier of machine learning, implementing and improving... 
    Training
    Work at office
    Visa sponsorship
    Flexible hours

    SignalAI

    New York, NY
    11 hours ago
  • $200k

    Optiver is a seeking a Machine Learning Performance Engineer to join our team, focusing on a pivotal AI...  ...Machine Learning infrastructure, training, and inference challenges to advance our futures...  ...Triton, CUB, cuDNN, and cuBLASLinux system programming experienceExperience... 
    Training
    Work at office

    Optiver

    New York, NY
    2 days ago
  •  ...the DT User Card).As a Senior Machine Learning Engineer, you'll design and scale the ML systems behind ad ranking, bid optimization...  ...engineering, distributed training, deployment, retraining, and monitoring...  ....Optimize training and inference for compute efficiency (CPU/... 
    Training
    Full time
    Local area
    Shift work

    Fyber

    New York, NY
    1 day ago
  • $148.7k - $199.4k

     ...global organization of engineers, product developers,...  ...and advanced machine learning platforms that deliver...  ...for scalable learning, inference, and monitoring, conduct...  ...such as recommendation systems, object detection,...  ...feature engineering, batch training and low-latency... 
    Training
    Local area
    Day shift

    Disney Interactive

    New York, NY
    2 days ago
  • $147.6k - $274k

     ..., we are building the machine learning platforms that enable researchers and engineers to move models from experimentation...  ...real-time and batch inference workloads, GPU-backed...  ..., and distributed systems. Prior inference-...  ..., distributed training, or large-scale data processing... 
    Training
    Full time
    Local area
    Immediate start
    Worldwide
    Relocation package

    Genentech

    New York, NY
    6 hours ago
  • $148.7k - $199.4k

    Job Posting Title:Senior Machine Learning Engineer - ESPNReq ID:10150610Job Description...  ..., feature computation systems, and ML‑adjacent services...  ...‑adjacent services such as inference inputs, feature APIs, and data...  ..., Experience/Skills/Training:Basic Qualification:Experience... 
    Training
    Full time
    Worldwide

    Hulu

    New York, NY
    1 day ago
  • $244k - $293k

     ...We are hiring a Staff Machine Learning Engineer to drive the design, development...  ...of machine learning systems that power our core products...  ...products. Experience with causal inference, uplift modeling, and...  ...work experience, education/training, job-related skills, internal... 
    Training
    Full time
    Work experience placement
    Remote work

    Match Group

    New York, NY
    1 day ago
  • $216k - $270k

     ...is to develop reliable AI systems for the world’s most important...  ...About the Role As a Machine Learning Engineer on Agent Oversight, you will...  ..., reward models, or model training/fine-tuning approaches...  ...optimization (e.g., latency, cost, or inference efficiency) Published... 
    Training
    Full time

    Scale Ai, Inc.

    New York, NY
    1 day ago
  •  ...Fintal Partners in New York is hiring engineers to build the distributed training and inference systems that take models from research into production. You’ll work side-by-side with researchers and traders on a trading floor-like environment, owning critical, IC-heavy... 
    Training

    Fintal Partners

    New York, NY
    3 days ago
  •  ...We are looking for an engineer with experience in low-level systems programming and optimisation to join our growing ML team. Machine learning is a critical pillar of Jane Street's global...  ...of our models – both training and inference. We care about efficient large-... 
    Training
    Work at office

    Trading Interview

    New York, NY
    13 hours ago
  • $180k - $300k

     ...firm’s IT infrastructure and engineering capabilities, positioning us at...  ...large-scale generative AI and machine learning workloads, enabling faster model...  ...and operate distributed systems for model training, hyperparameter tuning, inference, and data preprocessing pipelines... 
    Training
    Work experience placement

    Point72

    New York, NY
    1 day ago
  •  ...We are looking for an engineer with robust experience in machine learning and strong mathematical foundations to...  ...engineering, and for designing APIs and systems that are delightful to use. We'...  ...building and maintaining training and inference infrastructure, with an... 
    Training
    Work at office

    Trading Interview

    New York, NY
    4 days ago
  • $240k - $249.5k

     ...looking for a Senior Staff Machine Learning Engineer to help lead the machine learning...  ..., and retrieval systems that decide what every diner...  ...transfer learning from pre-trained large scale models, calibration...  ...cold start, and real-time inference. Assess rigorously what actually... 
    Training
    Full time
    Temporary work
    Work at office
    Flexible hours
    3 days per week

    GrubHub

    New York, NY
    3 days ago
  •  ...reusable patterns for AI engineering, platform integration, governance...  ...with a background in Machine Learning (deep learning ideal)Hands...  ...AI/ML architecture across training, inference, feature pipelines, model...  ...designVector databases, retrieval systems, knowledge graphs, memory-... 
    Training
    Work experience placement
    Worldwide

    hackajob

    New York, NY
    13 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Machine Learning Engineer (Training & Inference Systems). Be the first to apply!