Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior AI Agent Engineer - Open Models & Evaluation Systems

Sail Research

Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market. Inference is just one piece of an effective background agent. Let's design and build the rest of the system, that turns billions of tokens into the best possible answers. What you’ll do Design custom evals for multi-turn, massively parallel agents. Build agent harnesses to improve open model (Deepseek, Qwen, Llama) performance. Claude Code is all about agent/harness codesign; let's do the same for open source! Automate prompt optimization techniques like DSPy. What we’re looking for Experience building AI agents. Familiarity with open source models. Interview process Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear. Come in to Sail's SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls :) Life at Sail We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally. #J-18808-Ljbffr Sail Research

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior AI Agent Engineer - Open Models & Evaluation Systems in San Francisco, CA vacancy
  • $200k - $300k

     ...Job Description Senior/Staff Applied AI Engineer, Agent Harness Company: Confidential...  ...with credentials the model never sees, and can operate systems it was never given an...  ...code rather than open-ended tool-chaining....  ...real-task data into evaluation and training What... 
    Senior
    Full time
    Contract work
    H1b
    Work at office
    Visa sponsorship
    Monday to Friday

    Transparent Search Group

    San Francisco, CA
    22 days ago
  • $159.2k - $301.6k

     ...Opportunity We are looking for a Senior AI Systems Engineer with deep C++ expertise to...  ...integration layers, evaluation systems, and reliability...  ...codebases to AI-powered services, agents, and model-backed workflows. Define...  ...NoticeIf this role is open to hiring in Colorado (as... 
    Senior
    Full time
    Temporary work
    Local area
    Remote work
    Worldwide

    Adobe Systems

    San Francisco, CA
    2 days ago
  • $124k - $280k

     ...data and analytics engineering focus on...  ...implementing advanced AI and ML solutions...  ...optimising algorithms, models, and systems to enable...  ...relevant.Initiate open and honest coaching...  ...health plans. As a Senior Manager, you will...  ...team members. We evaluate these factors thoughtfully... 
    Senior
    Full time
    H1b

    PwC

    San Francisco, CA
    4 days ago
  • $250k - $300k

     ...the Role Together AI runs one of the...  ...world. The Infra Agent Systems team builds the software...  ..., orchestration, evaluation, and the tooling...  ...challenging engineering problems with real...  ...graphs or graph data modeling ~ Search, retrieval...  ...of leading open models that teams... 
    Senior
    Full time
    Remote work

    Together AI

    San Francisco, CA
    4 days ago
  • $200k - $300k

     ...is a pre-seed AI startup in San...  ...product where an agent registers as a...  ...approval, and drives systems it was never...  ...that turns model capability into...  ...~4+ years of engineering experience ~...  ...Experience with model evaluation, fine-tuning,...  ...time Level: Senior / Staff... 
    Senior
    Full time
    Contract work
    H1b
    Work at office
    Visa sponsorship
    Monday to Friday

    David Joseph & Company

    San Francisco, CA
    22 days ago
  •  ...pod is creating the systems that determine what message...  ...health outcomes.As a Senior AI Engineer focused on MLOps and...  ...path that turns models and AI capabilities into...  ...expectations, evaluation hooks, and production...  ...AI, LLMs, retrieval, agents, or model evaluation... 
    Senior
    Full time
    Internship
    Work at office
    Local area
    Remote work
    Worldwide
    3 days per week

    Hinge Health

    San Francisco, CA
    3 days ago
  • Workleap seeks a Senior AI Developer to join the Product Ops team and shape the AI-powered agent platform. You will own production-ready multi-agent systems, set architectural standards, and drive cost...  ...partners in Product, Engineering, and Data Science. Remote work... 
    Senior
    Remote job

    Remote

    San Francisco, CA
    1 day ago
  •  ...AI Systems Engineer - Codex Core Agents About The Team The Codex Core Agents team builds...  ...agent harness that turns model capability into real-world...  ...quality. The harness is open source and increasingly part...  ...models are trained and evaluated, making this one of the highest... 
    Full time

    openai

    San Francisco, CA
    7 hours ago
  •  ...we’re building the AI agents that run insurance...  ...Both founders are engineers, and engineers...  ...Engineer to own how we evaluate and continuously...  ...automated grading systems: Develop LLM-as-judge...  .... Drive model evaluation and selection...  ...frontier and open-weight models across... 
    Senior
    Full time
    Work at office

    furtherai

    San Francisco, CA
    7 hours ago
  • $180k - $270k

     ...is building an AI-first company—and...  ...looking for a Senior Lead AI Finance Technology Engineer to be the...  ...Data, and GTM Systems, to ensure our...  ...copilots, and agents that automate and...  ...engineering and evaluation LLM system design...  ...Operating Model Identify high... 
    Senior

    Klaviyo

    San Francisco, CA
    4 days ago
  • $124k - $280k

     ...OpportunityAs an Applied AI Health System Engineer you will lead client-...  ...use cases. As a Senior Manager you will guide...  ...architecture, support model readiness, and help...  ...memory, prompt, and agent orchestration components...  ...-based AI platforms- Evaluating model behavior... 
    Senior
    Full time
    H1b

    PwC

    San Francisco, CA
    1 day ago
  •  ...Senior AI Engineer New York City or San Francisco...  ...advancing how AI systems reason, learn, retrieve...  ...publishing and open sourcing as we go...  ...algorithms, and evaluation infrastructure...  ...experimental or modeling work you can also...  ...which specialized agents and models work together... 
    Senior
    Full time

    Doctronic

    San Francisco, CA
    3 days ago
  • $180k - $200k

     ...insurer, or a health system, learn the work...  ...the newest models, real budgets,...  ...— build beside senior FDEs on a...  ...conference talks, open-source contributions...  ...adopt AI, from strategy...  ...prototype works. You evaluate. You have built...  ...and why. Strong engineering fundamentals —... 
    Senior
    Full time
    Work at office
    Visa sponsorship

    Provectus

    San Francisco, CA
    1 day ago
  •  ...building an agentic AI caregiver...  ...across multiple sub-agents to get things...  .... The AI engineering challenge: build...  ...design agentic systems that reason, plan...  ..., and evaluation infrastructure...  ...testing across model updates. Prompt...  ...Contributions to open-source AI/LLM tooling... 
    Senior
    Immediate start
    Remote work
    Flexible hours

    Citizen Health

    San Francisco, CA
    1 day ago
  • $197.3k - $225.1k

    AI Engineer 4 (Gen AI Platform Services: Agentic AI, Agent Guardrails, Agent Evaluation, Agent Memory) At Capital One,...  ...and reliable AI systems, changing...  ...customers. Our AI models and platforms empower...  ...broad stack of Open Source and SaaS...  ...Principal and Senior Associates on... 
    Full time
    Part time
    Local area

    Capital One

    San Francisco, CA
    7 hours ago
  •  ...This is a backend- and systems-heavy founding engineering role where “full-stack” means...  ...for long-running human and agent work: durable state,...  ...boundary, including validation, evaluations, and release gates...  ...Thinks in invariants, threat models, failure modes, and user outcomes... 
    Senior
    Full time
    H1b
    Work at office
    Visa sponsorship

    Cephas Consultancy Services Private Limited

    San Francisco, CA
    3 days ago
  •  ...of caring nurses while AI agents handle the tedious...  ...navigates the healthcare system on your behalf, and propels...  ...'re looking for an AI engineer who owns a business...  ...start to finish, the evaluation infrastructure that...  ...prompts, retrieval, and models. Make quality a number... 
    Senior
    Remote job
    Full time

    miramace

    San Francisco, CA
    5 hours ago
  • # AI Agent Engineer — Early CareerBuild and verify the agent-native developer systems that help people and coding agents plan...  ...Francisco, CAWork model: Remote-firstEmployment...  ...window: Open until filledApply for...  ...integration tests, and agent evaluations* Investigate failures... 
    Internship
    H1b
    Remote work
    Flexible hours

    CaelumenAI

    San Francisco, CA
    1 day ago
  • $190.9k - $334.1k

     ...DescriptionIt all started when engineer Fred Luddy wrote...  ...ServiceNow is the AI control tower for...  .... Develop novel AI Agent frameworks,...  ...customer enterprise systems including ServiceNow...  ...engineering, and data modeling. Curiosity &...  ...ConsultingExperience level: Mid-Senior LevelIndustry:... 
    Senior
    Work at office
    Immediate start
    Remote work
    Flexible hours
    Day shift

    ServiceNow

    San Francisco, CA
    2 days ago
  •  ...startup (1-10 employees) building AI coworkers for IT teams: a security-focused product where an agent registers as a governed...  ...for human approval, and drives systems it was never given an API for....  ...role builds the layer that turns model capability into systems that actually... 
    Senior
    Full time
    Contract work
    H1b
    Visa sponsorship

    Thomas Talent Network

    San Francisco, CA
    27 days ago
  •  ...employees and AI agents safely reach the...  ...centralized MCP (Model Context...  ...to enterprise systems such as Slack,...  ...AI Enablement Engineer, you'll own the...  ...technical direction. Evaluate emerging agent...  ...decisions with senior ICs and SLG.What...  ...Greenhouse to apply to our open roles.
    Senior
    Remote work

    SoFi

    San Francisco, CA
    1 day ago
  •  ...’s SeniorStaff AI Engineer is a hands-on AI...  ...is a critical, senior role...  ...scale.Distributed Agent Memory & State:...  ...failoversDeep Model Optimization: Pioneer...  ...LLMs, and NLP, evaluating and experimenting...  ...LLM and agentic systems.Experience with...  ...to apply to our open roles.
    Senior
    Remote work

    SoFi

    San Francisco, CA
    12 hours ago
  •  ...Primarily On-site We are seeking an AI Evaluation Engineer – Reinforcement Learning & Agents to build the environments, evaluation systems, and supporting infrastructure...  ...or publication history. Seniority The opportunity is open to exceptional new graduates,... 

    MaxIT Consulting - Max Corporate Group

    San Francisco, CA
    5 days ago
  •  ...Research Scientist, Research Engineer, AI Systems EngineerThe Recursive...  ...feedback loops, designing evaluations, and training models to develop missing...  ...research capabilities through agent harnesses, synthetic data...  ...generalist who can move between open-ended research and... 

    OpenAI

    San Francisco, CA
    2 days ago
  •  ...s most dynamic AI companies, like...  ...bring cutting-edge models into production...  ...the platform engineers turn to ship AI...  ...: Evaluate, run and automate...  ...contribute to open-source tools such...  ...and analysis. System Profiling : Use...  ...This is a mid-senior, high leverage... 
    Full time
    Flexible hours

    Baseten

    San Francisco, CA
    4 days ago
  •  ...Description Founding Applied AI Engineer — Proactive AI Agents Location: San...  ...equity Visa: Open to visa transfers...  ...products, agent workflows, model orchestration, and...  ...and agentic systems that observe user behavior...  .... The team will evaluate hands-on technical depth... 
    Full time
    H1b
    Relocation
    Visa sponsorship
    Relocation package

    Morgan Pinnacle Group

    San Francisco, CA
    8 days ago
  • $293.5k

     ...Title Expert Senior Manager, AI Engineering Job ID 1043...  ...of an integrated system that combines technology...  ..., operating model changes, and...  ...including multi-agent workflows and production...  ..., and evaluation design to production...  ...job postings for open roles. The estimated... 
    Senior
    Permanent employment
    Full time
    Apprenticeship
    Work at office
    Local area
    Work from home
    Home office
    3 days per week

    Bain & Company

    San Francisco, CA
    2 days ago
  • $268k - $358k

     ...platform where AI and automation...  ...AI for Protein Engineering team develops and...  ...and predictive models that move...  ...looking for a senior individual contributor...  ...Lila designs, evaluates, and learns from...  ...software systems and broader reasoning...  ...Publications, open-source contributions... 
    Senior
    Full time
    Work at office
    Local area
    Flexible hours

    Lila Sciences

    San Francisco, CA
    3 days ago
  •  ...builds the layer that turns model capability into systems that work for users. You’ll develop the core agent harness—including the execution...  .... You’ll build and run evaluations against replicas of real customer...  ...Python/TypeScript and modern AI tooling Experience with... 
    Senior
    Full time
    Contract work
    H1b
    Work at office
    Visa sponsorship

    Cephas Consultancy Services Private Limited

    San Francisco, CA
    3 days ago
  •  ...Corporate Group in San Francisco invites applications for the Forward Deployed AI Engineer – Agents & Research Systems. You will build production-grade agent systems, data pipelines, and evaluation tooling for large enterprise deployments. The role blends software... 

    MaxIT Consulting - Max Corporate Group

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior AI Agent Engineer - Open Models & Evaluation Systems. Be the first to apply!