Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Machine Learning Engineer - Inference

$160k - $230k
Full-time

Together Ai

About the Role


Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models models and ensuring they run efficiently and effectively at scale. If you are passionate about AI inference, PyTorch, and developing high-performance systems, we want to hear from you. This position offers the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions. Join us in shaping the future at Together AI!

Responsibilities



  • Design and build the production systems that power the Together AI inference engine, enabling reliability and performance at scale.

  • Develop and optimize runtime inference services for large-scale AI applications.

  • Collaborate with researchers, engineers, product managers, and designers to bring new features and research capabilities to the world.

  • Conduct design and code reviews to ensure high standards of quality.

  • Create services, tools, and developer documentation to support the inference engine.

  • Implement robust and fault-tolerant systems for data ingestion and processing.

Requirements



  • 3+ years of experience writing high-performance, well-tested, production-quality code.

  • Proficiency with Python and PyTorch.

  • Demonstrated experience in building high performance libraries and tooling.

  • Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, storage, performance, and scale.

  • Preferred: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum

  • Preferred: Knowledge of AI inference techniques such as speculative decoding.

  • Preferred: Knowledge of CUDA/Triton programming.

  • Nice to have: Knowledge of Rust, Cython and compilers.

About Together AI


Together AI is a research-driven artificial intelligence company. We believe open and transparent AI systems will drive innovation and create the best outcomes for society. Together, we are on a mission to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models. We have contributed to leading open-source research, models, and datasets to advance the frontier of AI. Our team has been behind technological advancements such as FlashAttention, Hyena, FlexGen, and RedPajama. We invite you to join a passionate group of researchers and engineers in our journey to build the next-generation AI infrastructure.

Compensation


We offer competitive compensation, startup equity, health insurance, and other competitive benefits. The US base salary range for this full-time position is $160,000 - $230,000 + equity + benefits. Our salary ranges are determined by location, level, and role. Individual compensation will be determined by experience, skills, and job-related knowledge.

Equal Opportunity


Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunities to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.




Please see our privacy policy at 

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Machine Learning Engineer - Inference in San Francisco, CA vacancy
  • $180k - $270k

     ...highest standards of data security and privacy protection. To learn more about Plaud, please visit and follow along on...  ...experience building and deploying high-throughput, ultra-low-latency inference engines for large language models or foundational speech models.... 
    Suggested
    Full time
    Work at office
    Worldwide

    Plaud

    San Francisco, CA
    1 day ago
  • $250k

    Title : ML Inference Engineer Location : San Francisco, CA Salary : $250k base + equity An AI Unicorn startup is hiring a Senior Machine Learning Inference Engineer for a full-time role. You will be responsible for improving efficiency for AI-native infrastructure powered... 
    Suggested
    Full time

    Oscar Technology

    San Francisco, CA
    5 days ago
  • $203.5k - $299.3k

     ...creates a causal question.About the RoleWe are hiring a Causal Machine Learning Engineer to help build the causal ML foundation behind how DoorDash...  ...because you have…Deep practical experience with causal inference, econometrics, experimentation, or causal ML.Experience... 
    Suggested
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    2 days ago
  • $155k - $180k

     ...half the Fortune 100, use Roboflow’s machine learning open source and hosted tools. That includes...  ...on all roles (not only product and engineering), so Roboflow employs developers...  ...At the center of all of this is inference — one of our most important open source... 
    Suggested
    Full time
    Second job
    Remote work
    Work from home
    Relocation package
    Flexible hours
    Night shift

    Roboflow

    San Francisco, CA
    1 day ago
  •  ...a causal question. About the Role We are hiring a Causal Machine Learning Engineer to help build the causal ML foundation behind how DoorDash...  ...re excited about you Deep practical experience with causal inference, econometrics, experimentation, or causal ML. Experience shipping... 
    Suggested
    Hourly pay
    Work at office
    Local area
    Flexible hours

    DoorDash, Inc.

    San Francisco, CA
    5 days ago
  • $203.5k - $299.3k

     ...a causal question. About the Role We are hiring a Causal Machine Learning Engineer to help build the causal ML foundation behind how DoorDash...  ...you because you have… Deep practical experience with causal inference, econometrics, experimentation, or causal ML . Experience... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Visa Hunt

    San Francisco, CA
    5 days ago
  • About the Role We are seeking a Machine Learning Engineer to join our Inference Infrastructure team. You will build and optimize the high-throughput, low-latency distributed systems that power models like GPT-4 and Sora for millions of users worldwide. Responsibilities... 
    Worldwide
    Flexible hours

    AI Breaking Wire

    San Francisco, CA
    5 days ago
  • AI Breaking Wire seeks a Machine Learning Engineer to join our Inference Infrastructure team. You will build and optimize the high-throughput, low-latency distributed systems that power models like GPT-4 and Sora for millions of users worldwide. The role emphasizes C++... 
    Worldwide

    AI Breaking Wire

    San Francisco, CA
    5 days ago
  • A media technology company in San Francisco is seeking a Founding Engineer specializing in ML Inference. This highly technical role requires expertise in the ML infrastructure stack and aims to optimize generative media performance. The ideal candidate will drive innovations... 
    Relocation package

    Reactor.am

    San Francisco, CA
    5 days ago
  • Oscar is hiring a Senior Machine Learning Inference Engineer for a full-time role in the San Francisco Bay Area. You will focus on improving efficiency for AI-native infrastructure powering generative and multimodal models, with significant ownership over production inference... 
    Full time

    Oscar

    San Francisco, CA
    3 days ago
  • Reactor in San Francisco is seeking an ML Inference Engineer to maximize performance of generative media models and push ultra-low-latency, high-throughput inference. You will craft an in-house runtime, implement optimizations with PyTorch tools, and collaborate with partner... 

    Reactor.am

    San Francisco, CA
    18 hours ago
  •  ...requires expertise in deploying GPU systems for high-throughput inference and model performance optimization. The ideal candidate will...  ...inference frameworks and a solid understanding of reinforcement learning technologies. Comprehensive healthcare benefits, parental... 

    Reflection AI

    San Francisco, CA
    4 days ago
  • $150k - $200k

     ...instructional materials with dynamic digital learning. Through unparalleled curriculum...  ...other departments, including Product, Engineering, Machine Learning and Analytics, to understand...  ...execution of code, especially for model inference and lightweight processing tasks.  ~... 
    Permanent employment
    Full time
    Work at office
    Local area
    Remote work

    Kiddom

    San Francisco, CA
    1 day ago
  •  ...is to reinvent the way people learn, starting with language....  ...role We’re hiring an ML Engineer, Assessments to help build best...  ...(Content/Learning Design) , Machine Learning, Product, and Engineering...  ...→ model training → inference → feedback generation) Own... 
    Full time
    Live in
    Immediate start

    Speak

    San Francisco, CA
    1 day ago
  • $150k - $200k

     ...reliable, high-speed robot autonomy software stack optimized for inference performance ● Advance SOTA dexterous manipulation...  ...Required Qualifications ● PhD or MS degree in Computer Science, Machine Learning, Robotics, or equivalent technical discipline ● Deep... 
    Full time

    Deft Ai, Inc.

    San Francisco, CA
    1 day ago
  • $165k - $230k

     ...like. About the role We're looking for exceptional Machine Learning Engineers focused on Ads to help take Higgsfield's advertising...  ...reliably at significant scale, from experimentation through inference and serving. Work closely with Product, Research, Engineering... 
    Full time
    Work at office
    Remote work
    Worldwide
    3 days per week

    Higgsfield

    San Francisco, CA
    1 day ago
  •  ...Francisco, NYC, or London offices. About the Role As a Machine Learning Engineer on the Marketplace team, you will build the models and...  ...not just top-of-funnel engagement • Real-time and batch inference systems embedded in product-critical workflows Example... 
    Full time
    Work at office
    Relocation package

    Mercor

    San Francisco, CA
    1 day ago
  •  ...connect and drive people forward. We are looking for a Machine Learning Engineer to join the growing AI and Machine Learning team at Strava...  ...to shipping production code to scaling and optimizing inference and deployment Shape AI at Strava : Bring your voice and... 
    Full time
    Work at office
    Worldwide
    Flexible hours
    3 days per week

    Strava

    San Francisco, CA
    1 day ago
  • $150k - $190k

     ...-driven simulation software stack for engineering and manufacturing across advanced industries...  ..., multi-physics simulation through AI inference across the entire engineering...  ...goals. Who We're Looking For As a Machine Learning Engineer in Delivery, you are a... 
    Remote job
    Full time
    Flexible hours

    Physicsx

    San Francisco, CA
    1 day ago
  •  ...that runs the real economy. Learn more about our vision in our...  ...Collaborate with product and engineering teams to integrate and deploy...  ...Have Strong experience in machine learning, deep learning, and...  ...generative AI, or real-time inference systems. Hands-on experience... 
    Full time
    Worldwide
    Shift work

    HappyRobot

    San Francisco, CA
    1 day ago
  •  ...We are a small, fast-growing team of engineers in San Francisco powering Fortune 100 enterprises...  ...our San Francisco office ~ Eager to learn and adapt quickly ~ Prior startup or...  ...active learning pipelines Optimize inference, batching, and quantization on GPU Productionize... 
    Full time
    Work at office
    Visa sponsorship
    Relocation package

    The Pulse

    San Francisco, CA
    1 day ago
  •  ...our growing team. About the Role We're looking for a Machine Learning Engineer to design, build, and deploy production-grade ML systems...  ...scalable ML pipelines for training, evaluation, monitoring, and inference Build intelligent services using modern NLP, LLM,... 
    Full time
    Work at office
    Remote work
    Flexible hours
    2 days per week

    Plenful

    San Francisco, CA
    1 day ago
  •  ...is seeking a Member of Technical Staff to design and optimize inference systems. The role involves managing KV cache allocation and improving...  ...components. Ideal candidates should have strong software engineering skills and experience with ML inference systems, particularly... 

    Gimlet Labs

    San Francisco, CA
    2 days ago
  • OpenAI in San Francisco seeks an experienced Software Engineer to help bring inference workloads to AWS Trainium and build the software stack to run frontier models efficiently on the platform. This deeply technical, cross‑stack role covers kernels, compilers, and model... 

    Slope

    San Francisco, CA
    18 hours ago
  • MakerMaker.AI is looking for a Senior Machine Learning Systems Engineer in San Francisco. In this role, you will build and operate production inference systems, optimizing for performance and reliability. The ideal candidate will have 3+ years of experience in production... 

    MakerMaker.AI

    San Francisco, CA
    18 hours ago
  •  ...looking for a Member of Technical Staff focused on ML systems and inference in San Francisco. You will design and build inference systems...  .... Candidates should have strong foundations in software engineering, experience with ML inference systems, and performance tuning... 

    Gimlet Labs, Inc.

    San Francisco, CA
    4 days ago
  • $140k - $200k

     ....About the RoleWe’re looking for an ML Engineer to build the production systems that train...  ..., monitor, retrain, and serve our machine-learning models reliably. You sit between software...  ...stay working.You will build training and inference pipelines, serve predictions through... 
    Temporary work
    Work at office
    Monday to Friday
    Monday to Thursday

    Sprinter Health

    San Francisco, CA
    4 days ago
  • Oscar Associates Limited (US) in San Francisco offers a hybrid Senior ML Inference Engineer role. You will enhance AI-native infrastructure powering generative and multimodal models, with ownership across inference systems and production performance. The ideal candidate... 

    Oscar Technology

    San Francisco, CA
    5 days ago
  • $200k - $400k

     ...Job Description Job Description Machine Learning Infrastructure Engineer San Francisco, CA · On-site (5 days/week) · Full-time Compensation...  ...The Role You would own the distributed training and inference backbone for a foundation model trained from scratch —... 
    Full time
    Visa sponsorship
    Relocation package

    David Joseph & Company

    San Francisco, CA
    14 days ago
  • $200k - $260k

     ...Role Together AI is building the best inference infrastructure for voice applications....  .... We're looking for a Senior ML Engineer to drive the model serving layer for voice...  ...strong plus but not required — you can learn this quickly if you have strong ML engineering... 
    Full time

    Together Ai

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Machine Learning Engineer - Inference. Be the first to apply!