Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Platform Engineer: Scale AI Infra & LLM Routing

$215k - $285k

OpenRouter

About OpenRouter OpenRouter is the leading AI routing and infrastructure layer that enterprises use to access, manage, and optimize the best large language models across providers—without lock-in, capacity constraints, or unnecessary cost. We power the most advanced AI teams in the world by giving them the flexibility to move fast, scale confidently, and stay future-proof as models evolve. As enterprise adoption of AI accelerates, OpenRouter sits at the center of how organizations operationalize LLMs across research, product, and production workloads. About the Role We're hiring a Software Engineer on our Platform team to own and scale the systems that route and serve millions of LLM requests every day. The business is growing at an unbelievable pace and we need help to ensure our platform can keep up. You'll work alongside a small team of engineers on owning our infrastructure, datastores, CI/CD pipeline and everything else at OpenRouter is built on. This is a hands-on IC role with broad surface area and little process between you and shipping. What You'll Do Own and evolve our edge and cloud infrastructure across Cloudflare, Google Cloud, and Vercel. Scale and operate our data layer including Spanner, ClickHouse, and Postgres. Ensure we are optimizing for performance when serving LLM inference as traffic rapidly grows. Partner with engineering leadership on capacity, reliability, and cost across the routing layer, with ownership of the systems carrying production traffic. Set the bar and playbook for how we run infrastructure and operations as the team grows — tooling, observability, on-call, and the patterns other engineers build against. About You 5+ years building and operating production infrastructure at companies where uptime, latency, and cost matter. Proven experience with cloud platforms (GCP, AWS, Azure) and edge-first serverless platforms (e.g. Cloudflare Workers) Deep expertise in operating large scale databases (e.g Postgres, Spanner, etc). A full-stack TypeScript shop won't faze you; you can move across the stack when the platform needs it. High agency and a bias toward action. You don't wait for tickets — you see the bottleneck and fix it. AI-forward in your workflow. You use coding agents, MCPs, and LLMs heavily and have opinions about what works. Pragmatic about tradeoffs between speed and simplicity. Bonus Points Existing user of OpenRouter, or active side projects in AI products/infrastructure or developer tooling. The base salary for this full-time position in the United States, spanning multiple internal levels depending on qualifications, ranges between $215,000 to $285,000 plus benefits & equity. Compensation for internationally based candidates will vary to reflect local market conditions. If you don't think you meet all of the criteria below but still are interested in the job, please apply. Nobody checks every box, and we're looking for someone who is excited to join the team. #J-18808-Ljbffr OpenRouter

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Platform Engineer: Scale AI Infra & LLM Routing in New York, NY vacancy
  •  ...member of the Technical Staff to manage AWS infrastructure and build backend services. The ideal candidate will have deep experience in scaling systems and debugging under pressure, with a strong background in cloud infrastructure. This role offers competitive salary, equity... 
    Suggested

    Anterior

    New York, NY
    1 day ago
  • Phaxis is seeking an experienced AI Platform Engineer in New York to develop and scale core infrastructure for next-generation AI capabilities. The role involves designing and building systems that support AI-powered applications and ensuring operational stability. Successful... 
    Suggested

    Phaxis

    New York, NY
    3 days ago
  • ExaCare AI, based in New York, is searching for a Senior Software Engineer (Platform) to enhance and scale their infrastructure and backend systems. The role demands expertise in TypeScript and PostgreSQL while focusing on operational excellence and system reliability.... 
    Suggested
    Flexible hours

    ExaCare AI

    New York, NY
    2 days ago
  • Atilax Executive Search is looking for a Senior Platform Infrastructure Engineer based in New York, NY. This role involves optimizing the infrastructure for an AI personalization platform and collaborating closely with engineering leadership to build reliable systems.... 
    Suggested

    Atilax Executive Search

    New York, NY
    3 days ago
  • A cutting-edge AI company is seeking a talented engineer to join its team in New York. The ideal candidate will possess a strong background in software engineering and experience with large-scale distributed systems. This role involves architecting multi-cloud systems,... 
    Suggested

    Scale AI

    New York, NY
    3 days ago
  • Senior Software Engineer -Gen AI Platforms Apply (opens in new window) Job Req Id: 26961645 Location(s)...  ...engineering cutting-edge products at planetary scale. We are solely focused on the most...  ..., systematic evaluation of LLM behaviours at scale, and your ability... 
    Full time
    Work at office

    Citi

    New York, NY
    3 days ago
  • $79k - $119k

     ...with confidence. Our AI-powered, cloud platform, RelativityOne, transforms...  ...We Do At Relativity, engineers don’t just write code...  ...workflows at massive scale, enabling customers...  ...systems that route AI traffic across regions...  ...large language model (LLM) APIs or generative AI... 
    Remote job
    Home office

    Relativity

    New York, NY
    3 days ago
  • $230k - $270k

    Betterment is looking for a Sr Staff Engineer in New York City to lead AI product strategy, working closely with engineering teams to build AI capabilities. This role involves defining the technical direction, mentoring engineers, and partnering with product teams to deliver... 
    Flexible hours

    Betterment

    New York, NY
    3 days ago
  • Fora is seeking a Senior/Staff Platform Engineer in New York City to lead and scale core infrastructure. You’ll own reliability, scalability, and developer experience across CI/CD, cloud architecture, observability, and security. This role partners with engineering and... 

    Fora

    New York, NY
    2 days ago
  • $180k - $200k

     ...innovative ML Ops Engineer. The ideal candidate...  ...edge machine learning platform and enhancing our...  ...and efficient AI/ML systems for a production...  ...across Data, Infra and ML Engineering....  ...distributed, and high-scale ML applications....  ...applications using LLM’s and GenAI is a huge... 
    Remote work
    Flexible hours

    Remotive

    New York, NY
    13 hours ago
  • Citadel Securities is seeking a Cloud Platform Engineer for its Research Platform Engineering team based in New York or Miami. The role focuses...  ...Science and over 3 years of experience building large-scale cloud systems. Strong programming skills in Go, Python, or C++... 

    Citadel Securities

    New York, NY
    3 days ago
  • $175k - $200k

     ...for OUTFRONT's internal AI platform. Today, our internal...  ...adoption. Both need a senior engineering owner, and both need...  ...agents safely: model routing, evaluation,...  ...how Agency Connect MCP scales with partner adoption....  ...least 2 years building LLM-backed systems that real... 
    Internship

    OUTFRONT Media

    New York, NY
    2 days ago
  • A leading AI finance company is seeking a Staff Platform Engineer to guide the technical direction of its platform as it scales rapidly. The ideal candidate will have extensive Kubernetes scaling experience and be responsible for ensuring reliability and improving developer... 

    Rogo

    New York, NY
    3 days ago
  • $146.8k - $272.6k

     ...Reuters is building the AI platform that will power...  ...a Lead Software Engineer (Staff Engineer),...  ...grade workflows at scale. This role is for...  ...AI, product, and infra (e.g., a new orchestration...  ...AI orchestration: routing, tool calling, MCP...  ...integrating LLM APIs (e.g., OpenAI... 
    Work at office
    Remote work
    Flexible hours
    2 days per week
    3 days per week

    Thomson Reuters

    New York, NY
    3 days ago
  • $130k - $185k

     ...paired with innovative AI-powered technology...  ..., the Growth Platforms team focuses on building...  .... The Software Engineering Director for AI...  ...execution—designing and scaling modern software...  ..., including LLM frameworks, agent...  ...inquiry which will route you to EY’s Talent... 
    Work experience placement
    Summer holiday
    Work at office
    Flexible hours

    EY

    New York, NY
    2 days ago
  • $175k - $220k

    A leading AI infrastructure company in California is seeking a Member of Technical Staff, Software Engineer to build core systems for their generative AI platform. This role involves designing APIs and distributed infrastructure while addressing customer needs. Candidates... 

    Fireworks AI

    New York, NY
    2 days ago
  • $130k - $185k

     ...paired with innovative AI‑powered technology...  ..., the Growth Platforms team focuses on building...  .... The Software Engineering Director for AI...  ...execution—designing and scaling modern software...  ..., including LLM frameworks, agent...  ...inquiry which will route you to EY’s Talent... 
    Work experience placement
    Summer holiday
    Work at office
    Flexible hours

    EY-Parthenon

    New York, NY
    2 days ago
  • Infinity Constellation is seeking a Staff Software Engineer to lead the evolution of its AI employee platform. This full-time position requires deep expertise...  ...platform performance, Kubernetes management, and scaling issues. Candidates should have over 10 years of experience... 
    Remote job
    Full time

    Infinity Constellation

    New York, NY
    3 days ago
  • Get AI‑powered advice on this job and more exclusive features....  ...quantitative trading and research platforms and AI agents for leading...  ...We’re looking for a part‑time engineer to help develop end‑to‑end quantitative...  ...Hourly pay initially, scaling with workload and contribution... 
    Hourly pay
    Part time
    Remote work

    Batonics AB

    New York, NY
    3 days ago
  • Ramp, based in New York, is looking for a software engineer to build and scale applications that drive growth. This role requires a minimum of 3 years...  ...grasp of modern development tools, data ingestion, and AI applications. Ramp offers a dynamic, collaborative workspace... 
    Flexible hours

    Ramp

    New York, NY
    3 days ago
  • Future Secure AI Pty is looking for a Sr. AI/ML Software Engineer to develop and maintain key components for the FutureSecure.ai platform. You will focus on building and scaling infrastructure for intelligent interactions, requiring strong skills in TypeScript/Node.js and... 

    Future Secure AI Pty

    New York, NY
    3 days ago
  • Capital One seeks a Senior Lead AI Engineer to help build Gen AI platform services and responsible AI systems. You will collaborate with engineers, researchers...  ...tools to optimize training and inference efficiency at scale, shaping the future of AI in banking. #J-18808-Ljbffr... 

    Energy Jobline ZR

    New York, NY
    3 days ago
  • $320k - $405k

     ...New York, is seeking a Senior Kubernetes Engineer. In this role, you will own, operate,...  ...fleets, ensuring that the control plane scales efficiently and reliably. Your deep Kubernetes...  ...and operational teams to deliver robust platform capabilities. This position comes with... 
    Flexible hours

    Anthropic

    New York, NY
    3 days ago
  • $130k - $185k

     ...paired with innovative AI-powered technology and...  ...line, the EY Growth Platforms AI ML Engineering Director will collaborate...  ...to design, build, and scale innovative AI...  ...retrieval systems, and LLM fine‑tuning. Strong understanding...  ...an inquiry which will route you to EY’s Talent... 
    For contractors
    Work experience placement
    Summer holiday
    Flexible hours

    Ernst & Young Oman

    New York, NY
    13 hours ago
  • $200k - $250k

     .... We're building the agentic AI platform designed exclusively for healthcare...  ...’re hiring a Senior Platform Engineer focussed on supporting and...  ...and work to optimize LLM-assisted coding workflows, including...  ...of the mechanics of workload scaling in GKE Strong experience with... 
    Work at office

    Translucent AI

    New York, NY
    2 days ago
  •  ...About the Role We are looking for a Senior Platform Infrastructure Engineer to help scale and optimize the infrastructure powering our AI personalization platform. This is an...  ...infrastructure reliability. Exposure to AI/LLM infrastructure, model-serving platforms, or... 
    Full time

    Atilax Executive Search

    New York, NY
    2 days ago
  •  ...s process orchestration platform for users to design, optimize...  ...of workflow execution engines. Implement best...  ...code. Strong interest in AI‑native development and workflow...  .... Experience with LLM integration, vector databases...  ...and large‑scale observability is a plus.... 

    Sea12

    New York, NY
    2 days ago
  • $133.9k - $154.5k

     ...toward transforming global markets. Overview The AI Platform Engineering Lead drives the AI Platform Operations team,...  ...design patterns and best practices that span LLM, MCP, and agentic capabilities, ensuring the platform scales securely and operates with consistency across... 
    Full time
    Contract work
    Temporary work
    Flexible hours

    Ice Services

    New York, NY
    2 days ago
  • $170k - $297.2k

     ...globe. From data rooms to AI deal sourcing and...  ...private market dealmaking platform. We make it easy to...  ...Director of Platform Engineering to lead the foundational...  ...Product, Data, and AI to scale Grata's technical...  ...autonomous workflows , LLM-powered products, or AI... 
    Work at office
    Flexible hours

    DataSite

    New York, NY
    4 days ago
  •  ...the only first-to-market AI native coaching platform for enterprise, offering personalized...  ...the future at a global scale, this is your chance....  ...Role As a Platform Engineer at Valence, you will build...  ...safety, global access and routing, LLM governance, deployment automation... 
    Full time
    Freelance
    Remote work
    Worldwide

    Valence

    New York, NY
    13 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Platform Engineer: Scale AI Infra & LLM Routing. Be the first to apply!