Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior AI Engineer Inference & Agent Systems

Full-time

Arcana Analytics

Title:  Applied AI Engineer — Inference & Agent Systems

Location: United States

What We're Building

Arcana is building AI agents that synthesize information across heterogeneous sources and deliver structured, reasoned answers in real time. The product only works if the agents are fast, reliable, and correct — not approximately correct.

Our stack: Go + Temporal for orchestration, a Plan-Execute-Synthesize agent architecture, and an evaluation harness we use to measure every regression. The problems are hard. The latency bar is aggressive. The accuracy requirements are unforgiving.

The Work

Inference Optimization

- Drive TTFT below 400ms for multi-step agent pipelines

- Streaming optimization — first token to user while sub-agents are still running

- KV cache strategy, prompt compression, dynamic context window management

- Multi-provider routing: model selection by latency, cost, and task type across OpenAI, Anthropic, Gemini, and open-weight models

Agent Architecture

- Design and implement Plan-Execute-Synthesize pipelines that run sub-agents in parallel DAGs, not sequential chains

- Build reliable orchestration on top of Temporal — retries, timeouts, partial failure recovery, idempotency

- Structured output enforcement: JSON schema validation, retry loops on malformed LLM output, graceful degradation

- Tool call design: schema design that LLMs actually follow reliably across providers

Evaluation & Harness

- Own the eval framework end to end: ground truth datasets, automated scoring pipelines, regression detection on every PR

- LLM-as-judge pipelines for qualitative output assessment

- Latency regression testing — p50/p95/p99 tracked across every deployment

- Adversarial test case design: ambiguous queries, missing data, conflicting sources, malformed tool responses

Infrastructure

- Model serving and cold start optimization

- Async worker architecture for parallel sub-agent execution

- Observability: trace every token, every tool call, every synthesis step

What We're Looking For

You've built something that runs in production at a meaningful scale and you understand why it's fast (or why it isn't).

Strong signal :

- You've worked on inference pipelines where TTFT was the primary metric and you moved it meaningfully

- You've built multi-step agent systems and you know where they break — not from reading papers but from watching them fail in production

- You've written eval harnesses from scratch and you have opinions about what makes a ground truth dataset actually useful

- You've debugged LLM non-determinism in production and built systems resilient to it

- You've worked with streaming LLM responses and built infrastructure around partial output handling

Weaker signal (but not disqualifying):

- You've fine-tuned models but haven't shipped inference systems

- You've used LangChain/LlamaIndex but haven't built the 

layer underneath

- Strong ML research background without systems exposure

Stack familiarity (we care more about depth than match): Go, Python, Temporal, Kafka, PostgreSQL, Docker

Why This Role

The problems here don't have blog posts about them yet. Parallel agent DAG execution under hard latency budgets, streaming synthesis across partial sub-agent results, 

eval harnesses for non-deterministic multi-step systems — these are genuinely unsolved at production quality. Small team. High ownership. Every engineer's decisions ship to production.

Who We Want to Hear From

You've shipped inference systems at:

- A real-time AI product (search, coding assistant, chat at scale)

- A model serving infrastructure company

- An agent platform (any domain)

Or you've built eval/harness infrastructure that a team of 10+ engineers actually trusted to catch regressions.

Apply

Send to: [View email address on jobs.jobcopilot.com]

Include:


  1. One system you built where latency was the primary constraint — 

what you measured, what you changed, what moved


  1. Link to anything public (code, writing, talks) — optional but useful

  2. No cover letter required

We respond to every application.

 

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior AI Engineer Inference & Agent Systems in United States vacancy
  •  ...designing and scaling reusable AI-driven workflows, particularly...  ...models and autonomous agents. You will build foundational components...  ...and deploy intelligent systems. The position emphasizes system...  ...components used broadly across engineering teams. Key Responsibilities... 
    Senior
    Full time

    Calliere

    Remote
    1 day ago
  • NVIDIA is seeking an outstanding High Performance AI Engineer to build groundbreaking multi-agent systems for the CUDA ecosystem. You will develop agentic runtimes and collaborate with NVIDIA’s software and hardware teams to accelerate AI workloads on GPUs. The role requires... 
    Senior

    NVIDIA

    Austin, TX
    4 days ago
  •  ...performance computing, cloud, and AI. Whether you’re designing...  ...AI Group is looking for a Senior Software Development Engineer to own the end-to-end...  ...and high-performance inference serving. THE PERSON: This...  ...production metrics, and built the systems infrastructure (... 
    Suggested

    AMD

    Markham, IL
    4 days ago
  •  ...Senior AI Engineer – Edge Dialog Systems BrightAI is a high-growth Physical AI company transforming how businesses...  ...cannot pass silently. Run SLM inference on-device under memory,...  ...with the on-device voice pipeline agent. Exposure to MCP or agentic tooling... 
    Senior
    Full time
    Contract work
    Local area
    Immediate start

    Brightai

    Remote
    1 day ago
  •  ..., and operational support of AI platforms, tools, and services...  ...compatibility with existing systems and enterprise processes.Design...  ...server, cloud, and platform engineering teams.Operationalize machine...  ...Experience with model serving, inference optimization, or AI platform... 
    Senior
    Full time
    Work at office
    Remote work

    Applied Research Associates

    Raleigh, NC
    8 hours ago
  • $160k - $198k

     ...and celebrates all of our team members.What You’ll DoAs a Senior AI Systems Engineer, you will architect, deploy, and manage the critical infrastructure...  ...services required for large-scale AI model training and inference. You will ensure our machine learning platforms are robust... 
    Senior
    Local area

    Archer Aviation

    San Jose, CA
    8 hours ago
  •  ...Capital One in San Francisco is seeking a Staff AI Engineer to build responsible AI systems, partnering with engineers, researchers, technical program...  ...Capital One. The role covers foundation model training, inference, guardrails, evaluation, and observability, using Open... 
    Senior

    Capital One

    San Francisco, CA
    4 days ago
  • $229.9k - $262.4k

     ...Senior Lead AI Engineer (FM Hosting, LLM Inference) Overview: At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized... 
    Senior
    Full time
    Part time
    Local area

    Capital One National Association

    Cambridge, MA
    4 days ago
  • $229.9k - $262.4k

    Senior Lead AI Engineer (FM Hosting, LLM Inference) Overview: At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time, personalized... 
    Senior
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    5 days ago
  • $169.78k - $351k

     ...We are seeking an experienced and mission-driven Senior/Sr. Staff AI Infrastructure Engineer , Inference & Optimization to lead the performance tuning, deployment...  ...design high-efficiency inference pipelines, build system-level stability frameworks, and optimize hardware... 
    Senior
    Full time

    DiDi Labs

    San Jose, CA
    4 days ago
  • $160k - $198k

     ...Senior AI Systems EngineerSan Jose, California, United StatesArcher is an aerospace company based...  ....What You'll DoAs a Senior AI Systems Engineer, you will architect, deploy, and...  ...for large-scale AI model training and inference. You will ensure our machine learning... 
    Senior
    Local area

    Archer

    San Jose, CA
    4 days ago
  • $184k - $287.5k

     ...millions globally. We seek a Senior Engineer to lead technical efforts in deploying advanced AI agent frameworks and local runtimes...  .... By combining powerful local inference (Nemotron models) with strong...  ...foundation of the desktop AI operating system.What You Will Be Doing:Act as... 
    Senior
    Full time
    Local area
    Shift work

    Nvidia

    Redmond, WA
    3 days ago
  • $184k - $287.5k

    We are seeking highly skilled and motivated software engineers to join us and build AI inference systems that serve large-scale models with extreme efficiency. You’ll architect and implement high-performance inference stacks, optimize GPU kernels and compilers, drive industry... 
    Senior
    Full time

    Nvidia

    Santa Clara, CA
    3 days ago
  •  ...next generation of autonomous system intelligence. As a Model Optimization & Deployment Engineer, you will focus on bringing highly...  ..., and build highly concurrent inference code to ensure real-time,...  ...maximize memory bandwidth on AI accelerators. Write production... 
    Senior
    Temporary work
    Relocation package

    Zoox

    Foster, CA
    20 days ago
  •  ..., we’re building the next generation of AI agents for the insurance industry - a trillion-...  ...Now, we’re looking for an exceptional AI Engineer to join our agent team and help shape...  ...new agentic techniques (e.g. multi-agent systems, reasoning-from-feedback, RFT, etc)... 
    Senior
    Full time
    Work at office

    Further AI

    San Francisco, CA
    1 day ago
  • $190.9k - $334.1k

     ...Description It all started when engineer Fred Luddy wrote code...  ...Today, ServiceNow is the AI control tower for...  ...enterprise-scale search systems that power Now Assist, AI Agents, and the AI-driven experiences...  ...Exposure to LLM fine-tuning or inference optimization in... 
    Senior
    Full time
    Work experience placement
    Work at office
    Immediate start
    Remote work
    Flexible hours

    ServiceNow

    San Francisco, CA
    1 day ago
  •  ...get the support of caring nurses while AI agents handle the tedious backend work — all covered...  ...deeply, navigates the healthcare system on your behalf, and propels you to become...  ...LOOKING FOR We're looking for an AI engineer to build the LLM-powered systems at the... 
    Senior
    Full time

    Mira Mace

    San Francisco, CA
    1 day ago
  •  ...change that. Our multimodal AI models understand video the way...  ...TwelveLabs' unified agentic system that reasons across your videos...  ...whole product. Built for agents, not just people. As AI...  ...About the Role We're hiring a senior AI engineer to own the integration layer... 
    Senior
    Full time
    Live in
    Work at office
    Local area
    Remote work
    Visa sponsorship
    Flexible hours

    Twelve Labs

    Remote
    1 day ago
  • Cloudflare is seeking a highly autonomous systems engineer to design and build the core infrastructure powering AI inference across our global network. You will work with AI/ML engineers, hardware partners, and product teams to tackle sub-second model cold starts, multi... 
    Senior

    Cloudflare

    Austin, TX
    2 days ago
  • $106.9k - $176.5k

     ...world. The opportunity We are seeking AI Systems Engineers to own the security and trust fabric...  ...wide identity model so every workload, agent, and service carries a verifiable, propagated...  ...trust challenges of confidential AI inference (models/secrets inside enclaves).... 
    Senior
    Work experience placement
    Summer holiday
    Remote work
    Flexible hours

    EY

    Mc Lean, VA
    2 days ago
  • $229.9k - $262.4k

     ...Senior Lead AI Engineering (GenAI Platform Services: Agentic Systems) Overview: At Capital One, we are creating responsible and reliable AI systems, changing...  ...foundation model training, large language model inference, similarity search, guardrails, model evaluation... 
    Senior
    Full time
    Part time
    Local area

    Capital One

    New Providence, NJ
    3 days ago
  • $250k - $300k

     ...About the Role Together AI runs one of the largest GPU...  ...fleets in the world. The Infra Agent Systems team builds the software...  ...automation , solving challenging engineering problems with real...  ...developers get high-performance inference that scales reliably, fine-tuning... 
    Senior
    Full time
    Remote work

    Together AI

    Remote
    6 days ago
  •  ...A technology company is seeking an experienced software engineer to design and enhance features for their AI SOC platform. The role involves leading the development of AI agents and architecting scalable systems, with a strong emphasis on backend capabilities, particularly... 
    Senior

    Conifers.ai

    Dallas, TX
    4 days ago
  • Cloudflare is seeking a high-agency, systems engineer to design and build AI inference infrastructure across its global network. You will tackle sub-second model cold starts, multi-accelerator workload scheduling, and efficient cache management alongside AI/ML engineers... 
    Senior

    Webhosting

    Austin, TX
    4 days ago
  • $190.9k - $334.1k

     ...DescriptionIt all started when engineer Fred Luddy wrote code...  ..., ServiceNow is the AI control tower for...  ...ways. Develop novel AI Agent frameworks, architectural...  ...and customer enterprise systems including ServiceNow, Workday...  ...level: Mid-Senior LevelIndustry: Information... 
    Senior
    Work at office
    Immediate start
    Remote work
    Flexible hours
    Day shift

    Moveworks

    San Francisco, CA
    3 days ago
  •  ...role is for one of the Weekday's clients Now Hiring: Senior AI Agent Engineer Remote – United States Contract Position | 6+ Months...  ...• Design, implement, and enhance workflows and AI agent systems powered by Claude • Develop and manage Agent Skills, SKILL... 
    Senior
    Contract work
    Immediate start
    Remote work
    Weekday work

    Weekday AI

    United States
    3 days ago
  •  ...Senior AI Agent Engineer Argentina based Remote Senior AI Agent Engineer Wizeline is a global AI-native technology solutions provider...  ...design multi-step reasoning pipelines, wire them to business systems, and harden them for scale, safety, and maintainability —... 
    Senior
    Remote work
    Flexible hours

    Wizeline

    United States
    1 day ago
  •  ...Texas Sports Academy is on the lookout for a Senior AI Engineer specializing in LLM (Large Language Model) Systems and RAG (Retrieval-Augmented Generation) Optimization. As we continue to push the boundaries of sports technology, your role will be pivotal in developing... 
    Senior
    Remote job
    Full time

    Texas Sports Academy

    United States
    1 day ago
  • $120k - $220k

     ...Liberate builds AI agents to automate manual tasks for the $2.7T insurance industry. We started...  ...The Role We are seeking an AI Agent Engineer to bridge the gap between our cutting‑...  ...with customers to assess their current systems and workflows, identifying integration points... 
    Senior
    Work at office
    Flexible hours
    2 days per week

    Liberate

    Boston, MA
    2 days ago
  • $209.5k

     ...Senior AI Agent Engineer Remote Up to $209,500/ year Oracle Health is seeking a Senior AI Agent Engineer to build production AI agents...  ...implement AI-driven workflows that integrate with enterprise systems and support use cases such as SQL generation, pipeline... 
    Senior
    Temporary work
    Remote work
    Flexible hours

    hackajob

    United States
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior AI Engineer Inference & Agent Systems. Be the first to apply!