Senior AI Inference Gateway Engineer (Remote)

Rippling

Remote job

Salad is seeking a Staff Software Engineer for the AI Inference Gateway to own the system end-to-end, from request routing to scaling across gateway servers. You will optimize fleet efficiency, operate inference nodes (vLLM, llama.cpp, etc.), and maintain observability with OpenTelemetry. Expect on-call responsibilities and close collaboration with product and marketing teams to shape production deployments. #J-18808-Ljbffr Rippling

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Senior AI Inference Gateway Engineer (Remote) in New York, NY vacancy

Senior ML Systems Engineer, Inference
$150k - $220k
...Runpod is the AI Developer Cloud. More than one million developers... ...more than 20 billion inference requests. We closed a $100M Series... ...will depend on. We're a small, remote-first team. We take ownership... ...We're looking for a ML Systems Engineer, Inference. We want Runpod to...
Remote work
Senior
Full time
Runpod
Remote
19 days ago
Senior Network Engineer Consultant
$155k - $175k
...IWU is seeking a Senior Network Engineer Consultant (Independent... ...re-platforming and an AI-first technology... ...and redundant default-gateway strategies. Design... ...device-ID, GlobalProtect remote access, and... ...for AI training and inference traffic where standards...
Remote work
Senior
Contract work
For contractors
Local area
University Information Technology - Indiana Wesleyan Univers...
Indianapolis, IN
4 days ago
Senior Inference SW Engineer
Senior Inference SW EngineerThis role has been designed as 'Hybrid' with a requirement... ...Description:HPE's Private Cloud AI organization is seeking a Senior Software Engineer to build and evolve the model... ...location in the US; however, remote work options will be considered....
Remote work
Senior
Full time
Work experience placement
Work at office
Local area
Immediate start
2 days per week
Hewlett Packard Enterprise
Durham, NC
2 days ago
AI Systems Engineer - Secure Execution - Senior
$106.9k - $176.5k
...opportunity We are seeking AI Systems Engineers to own the security and... ...roots of trust, and remote attestation. A security... ...policy engines (OPA), and API gateways. Exposure to AI/ML workloads... ...of confidential AI inference (models/secrets inside enclaves...
Remote work
Senior
Work experience placement
Summer holiday
Flexible hours
Ernst & Young
Atlanta, GA
a month ago
Senior LLM Inference Engineer - Hybrid, Impact at Scale
Hewlett Packard Enterprise is seeking a Senior Inference Software Engineer to design and evolve the model runtime for enterprise AI platforms. The role focuses on low tail latency... ...an HPE office in the United States, with remote options considered. Strong expertise in Go...
Remote work
Senior
Work at office
2 days per week
Hewlett Packard Enterprise Company
Spring, Montgomery County, TX
4 days ago
Senior Engineer
$131k - $147k
...Caylent is an AI-first cloud services company... .... We are a fully remote global company with employees... ...The Mission As a Senior CXE Engineer, you are a hands-on... ...Optimize inference cost and latency hands... ...Lambda, EventBridge, API Gateway, DynamoDB, Step Functions...
Remote work
Senior
Work at office
Worldwide
Caylent
Oregon State
22 hours ago
Senior Forward Deployed Engineer - Okta for AI Agents
$200k - $275k
...Secure Every Identity, from AI to Human Identity is... ...platform, and security engineering teams, write production... ...(FGA), MCP Gateway, and agent client registration... ...the patterns. Engage senior leadership. Brief the... ...outcomes end to end. #LI-Remote P25257_3456702...
Remote work
Senior
Local area
Worldwide
Flexible hours
Shift work
Okta, Inc.
San Francisco, CA
2 days ago
Senior AI Systems Engineer
..., and operational support of AI platforms, tools, and services... ...server, cloud, and platform engineering teams.Operationalize machine... ...Experience with model serving, inference optimization, or AI platform... ...position may be performed fully remote, hybrid, or onsite at an ARA...
Remote work
Senior
Full time
Work at office
Applied Research Associates
Raleigh, NC
5 days ago
Senior Backend Engineer, Inference Platform
$150k - $270k
...You.com, we are building the AI Search Infrastructure that powers... .... Our team includes engineers, researchers, product builders... ...through one layer: our inference gateway. It allows users to select a... ...millisecond budget. This is a remote position open to candidates based...
Remote work
Senior
Full time
Immediate start
Work from home
Flexible hours
You.com
Remote
3 days ago
Senior Site Reliability Engineer
...Reference26-27966Remote50% Remote Overview Client US is seeking... ...experienced and pragmatic Senior Site Reliability Engineer to own the reliability,... ...repetitive operational work. - Use AI-assisted development to... ...(AWS Lambda, Fargate, API Gateway, EventBridge). - Experience...
Remote work
Senior
Permanent employment
Work experience placement
Shift work
Mindlance
Louisville, KY
2 days ago
Senior Staff Machine Learning Engineer, Agentic Systems - Moveworks
...we areMoveworks is the Agentic AI Assistant platform that... ...automation with Moveworks’ Reasoning Engine and natural language... ...including distributed training and inference pipeline for large language models... ...trust. Work personas (flexible, remote, or required in office) are...
Remote work
Senior
Permanent employment
Work at office
Flexible hours
Moveworks
Mountain View, CA
22 hours ago
Senior Inference SW Engineer
Senior Inference SW EngineerThis role has been designed as 'Hybrid' with a requirement... ...: HPE's Private Cloud AI organization is seeking a Senior Software Engineer to build and evolve the model runtime... ...location in the US ; however, remote work options will be considered....
Remote work
Senior
Work experience placement
Work at office
2 days per week
HITEC
Spring, Montgomery County, TX
2 days ago
Senior Site Reliability Engineer
...Senior Site Reliability Engineer This is a senior engineering role focused on building... ..., automation, and AI-assisted engineering practices... ...technical bar in a fast-moving, remote-first environment.... ...such as IAM, VPC, Transit Gateway, ALB/NLB, Route53, and cloud...
Remote work
Senior
Full time
Shift work
Jobgether
United States
2 days ago
Senior Site Reliability Engineer-Hybrid
...We have Contract role Senior Site Reliability Engineer-Hybrid for client at Louisville... ...work. ~ Use AI-assisted development to accelerate... ...(AWS Lambda, Far gate, API Gateway, Event Bridge). · Experience... ...be reached on # (63 Remote working/work at home options...
Remote work
Senior
Permanent employment
Contract work
Work experience placement
Work at office
Local area
Immediate start
Work from home
Shift work
Generis Tek Inc
Louisville, KY
5 days ago
Senior Site Reliability Engineer
...The Crown Is Yours At DraftKings, AI is becoming an integral part of both... ...come build the future together. As a Senior Site Reliability Engineer, you'll build and scale the critical... ...implementing and supporting API gateways, reverse proxies, and modern infrastructure...
Remote work
Senior
Immediate start
DraftKings
United States
1 day ago
Senior Site Reliability Engineer
...Senior Site Reliability Engineer Location: Global Remote / San Francisco · Full-Time About Andromeda Andromeda is... ...massive wave of fragmentation, with AI factories of every shape and... ...model builders, research labs, and inference providers when they need it. We...
Remote work
Senior
Full time
Andromeda Cluster, Inc
United States
4 days ago
Senior Inference Engineer — Scale Production AI APIs
AssemblyAI is hiring a Software Engineer to turn cutting-edge AI research into products our customers rely on every day. You'll sit on a fast-paced... .... You’ll design and ship customer-facing APIs, scale inference infrastructure for 1M+ users, and collaborate with researchers...
Senior
AssemblyAI, Inc.
Seattle, WA
4 days ago
Senior Forward Deployed Engineer II (AI/ML)
$167k - $209k
...world. We are looking for a Senior Forward Deployed Engineer II (FDE) who is passionate... ...production AI-native and agentic workloads... ...deployment. You will work across Inference Engine, runtime systems, orchestration... ...- $209,000 *This is a remote role JR: 2026-8207...
Remote work
Senior
Full time
Local area
Worldwide
Flexible hours
DigitalOcean
Remote
1 day ago
Senior Engineer, Machine Learning/AI Ops
$150.5k - $173k
...steps. Our partner is looking for a Senior Engineer, Machine Learning/AI Ops based in the United States. The... ...include developing real-time inference APIs, managing data integration pipelines... ...and training. Work arrangement: Remote position within the United States....
Remote work
Senior
Full time
Jobgether
Remote
2 days ago
Senior Site Reliability Engineering- AI Infrastructure
$78k
...highly talented and self- motivated Senior Site Reliability Engineering– AI Infrastructure to join it in... ...Position Type: Full-time Location: Remote Role/Responsibilities... ...availability issues affecting training jobs, inference services, internal platforms, and...
Remote work
Senior
Full time
Local area
HCLTech
United, PA
5 days ago
Senior Field Application Engineer - AI
...generation computing experiences—from AI and data centers, to PCs,... ...seeking a Field Application Engineer to join the AI and HPC Center... ...-specific training and inference workloads. Understand competitive... ...and training both remotely and in person to our customers...
Remote work
Senior
AMD
Austin, TX
3 days ago
Remote Senior Product Manager, ROCm AI Inference
AMD in Santa Clara is seeking a Senior Product Manager to drive strategy and execution for ROCm, AMD’s open-source GPU software stack. This role involves managing the product roadmap for inference capabilities and engaging with the open-source community to enhance AMD'...
Remote job
Senior
AMD
Santa Clara, CA
3 days ago
Senior/Staff System Research Engineer - LLM Inference Optimization
$236k - $330k
...enterprise. To usher in this new era, we seek AI-native thinkers across every function... ...and advance the state of the art in LLM inference systems and optimization.Our mission is... ...manual tuning. We embrace AI-native engineering, using AI not only as the workload we optimize...
Senior
Snowflake
Bellevue, WA
22 hours ago
Senior Site Reliability Engineer
...Senior Site Reliability Engineer We are looking for a senior Kubernetes-focused DevOps/SRE engineer to... ...you the internal expert on how k0rdent AI is deployed and operated — the person... ...full stack — control plane, identity, gateway, database, workflow engine — with one...
Remote work
Senior
Local area
Mirantis
United States
4 days ago
Senior AI Engineer, MLOps & Distributed Systems
Senior AI Engineer, MLOps & Distributed SystemsThe OpportunityJoin the team building the intelligence... ...yearOperate reliable online and batch inference capabilities that meet clear contracts... ...Health Hybrid ModelWe believe that remote work and in-person work have their own...
Remote work
Senior
Full time
Internship
Work at office
Local area
Worldwide
3 days per week
Hinge Health
San Francisco, CA
4 days ago
Senior LLM Inference Software Engineer (Hybrid/Remote)
Hewlett Packard Enterprise is seeking a Senior Software Engineer to advance the LLM inference runtime within HPE AI Essentials. You will design core components, including... ...supports a hybrid work model with potential remote options; primary location is the US with multiple...
Remote job
Senior
Hewlett Packard Enterprise
Spring, Montgomery County, TX
1 day ago
Senior Site Reliability Engineer (GPU Clusters) - Hosting
$250k
...Join a rapidly scaling AI cloud infrastructure provider... ...training, experimentation, and inference at scale. The company is... ...company is looking for a Senior / Staff Site Reliability Engineer to support and scale large... ...Stock options Bonus Remote working option and...
Remote work
Senior
Full time
San Francisco, CA
more than 2 months ago
Senior Product Manager -AI Inference Software
...generation computing experiences—from AI and data centers, to PCs,... ...owns the framework-layer inference product strategy for ROCm,... ...translating customer, ecosystem, and engineering signals into roadmap... ...LOCATION:Bay Area · Austin · Remote This role is not eligible for...
Remote work
Senior
Shift work
AMD
Santa Clara, CA
1 day ago
Senior Infrastructure Operations Engineer
$160k - $200k
...Who We Are Lightning AI is the company behind PyTorch... ...training, and production inference, with security,... ...is seeking an experienced Senior Infrastructure Operations Engineer to help operate and scale... ...This role may be fully remote within the U.S., or hybrid...
Remote work
Senior
Full time
Work at office
Work from home
Flexible hours
Lightning AI
Remote
9 days ago
Senior Deep Learning Software Engineer, Inference
$152k - $241.5k
NVIDIA seeks a Senior Software Engineer specializing in Deep Learning Inference for our growing team. As a key contributor, you will... ...today’s most sophisticated AI applications. Our team is responsible... ...by law.SummaryLocation: US, CA, Remote; US, DC, Remote; US, TX, Remote;...
Remote work
Senior
Full time
Nvidia
Austin, TX
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior AI Inference Gateway Engineer (Remote). Be the first to apply!