Senior AI Agent Engineer - Open Models & Evaluation Systems
Sail Research
Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market. Inference is just one piece of an effective background agent. Let's design and build the rest of the system, that turns billions of tokens into the best possible answers. What you’ll do Design custom evals for multi-turn, massively parallel agents. Build agent harnesses to improve open model (Deepseek, Qwen, Llama) performance. Claude Code is all about agent/harness codesign; let's do the same for open source! Automate prompt optimization techniques like DSPy. What we’re looking for Experience building AI agents. Familiarity with open source models. Interview process Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear. Come in to Sail's SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls :) Life at Sail We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally. #J-18808-Ljbffr Sail Research
$200k - $300k
...Job Description Senior/Staff Applied AI Engineer, Agent Harness Company: Confidential... ...with credentials the model never sees, and can operate systems it was never given an... ...code rather than open-ended tool-chaining.... ...real-task data into evaluation and training What...SeniorFull timeContract workH1bWork at officeVisa sponsorshipMonday to Friday$159.2k - $301.6k
...Opportunity We are looking for a Senior AI Systems Engineer with deep C++ expertise to... ...integration layers, evaluation systems, and reliability... ...codebases to AI-powered services, agents, and model-backed workflows. Define... ...NoticeIf this role is open to hiring in Colorado (as...SeniorFull timeTemporary workLocal areaRemote workWorldwide$124k - $280k
...data and analytics engineering focus on... ...implementing advanced AI and ML solutions... ...optimising algorithms, models, and systems to enable... ...relevant.Initiate open and honest coaching... ...health plans. As a Senior Manager, you will... ...team members. We evaluate these factors thoughtfully...SeniorFull timeH1b$250k - $300k
...the Role Together AI runs one of the... ...world. The Infra Agent Systems team builds the software... ..., orchestration, evaluation, and the tooling... ...challenging engineering problems with real... ...graphs or graph data modeling ~ Search, retrieval... ...of leading open models that teams...SeniorFull timeRemote work$200k - $300k
...is a pre-seed AI startup in San... ...product where an agent registers as a... ...approval, and drives systems it was never... ...that turns model capability into... ...~4+ years of engineering experience ~... ...Experience with model evaluation, fine-tuning,... ...time Level: Senior / Staff...SeniorFull timeContract workH1bWork at officeVisa sponsorshipMonday to Friday- ...pod is creating the systems that determine what message... ...health outcomes.As a Senior AI Engineer focused on MLOps and... ...path that turns models and AI capabilities into... ...expectations, evaluation hooks, and production... ...AI, LLMs, retrieval, agents, or model evaluation...SeniorFull timeInternshipWork at officeLocal areaRemote workWorldwide3 days per week
- Workleap seeks a Senior AI Developer to join the Product Ops team and shape the AI-powered agent platform. You will own production-ready multi-agent systems, set architectural standards, and drive cost... ...partners in Product, Engineering, and Data Science. Remote work...SeniorRemote job
- ...AI Systems Engineer - Codex Core Agents About The Team The Codex Core Agents team builds... ...agent harness that turns model capability into real-world... ...quality. The harness is open source and increasingly part... ...models are trained and evaluated, making this one of the highest...Full time
- ...we’re building the AI agents that run insurance... ...Both founders are engineers, and engineers... ...Engineer to own how we evaluate and continuously... ...automated grading systems: Develop LLM-as-judge... .... Drive model evaluation and selection... ...frontier and open-weight models across...SeniorFull timeWork at office
$180k - $270k
...is building an AI-first company—and... ...looking for a Senior Lead AI Finance Technology Engineer to be the... ...Data, and GTM Systems, to ensure our... ...copilots, and agents that automate and... ...engineering and evaluation LLM system design... ...Operating Model Identify high...Senior$124k - $280k
...OpportunityAs an Applied AI Health System Engineer you will lead client-... ...use cases. As a Senior Manager you will guide... ...architecture, support model readiness, and help... ...memory, prompt, and agent orchestration components... ...-based AI platforms- Evaluating model behavior...SeniorFull timeH1b- ...Senior AI Engineer New York City or San Francisco... ...advancing how AI systems reason, learn, retrieve... ...publishing and open sourcing as we go... ...algorithms, and evaluation infrastructure... ...experimental or modeling work you can also... ...which specialized agents and models work together...SeniorFull time
$180k - $200k
...insurer, or a health system, learn the work... ...the newest models, real budgets,... ...— build beside senior FDEs on a... ...conference talks, open-source contributions... ...adopt AI, from strategy... ...prototype works. You evaluate. You have built... ...and why. Strong engineering fundamentals —...SeniorFull timeWork at officeVisa sponsorship- ...building an agentic AI caregiver... ...across multiple sub-agents to get things... .... The AI engineering challenge: build... ...design agentic systems that reason, plan... ..., and evaluation infrastructure... ...testing across model updates. Prompt... ...Contributions to open-source AI/LLM tooling...SeniorImmediate startRemote workFlexible hours
$197.3k - $225.1k
AI Engineer 4 (Gen AI Platform Services: Agentic AI, Agent Guardrails, Agent Evaluation, Agent Memory) At Capital One,... ...and reliable AI systems, changing... ...customers. Our AI models and platforms empower... ...broad stack of Open Source and SaaS... ...Principal and Senior Associates on...Full timePart timeLocal area- ...This is a backend- and systems-heavy founding engineering role where “full-stack” means... ...for long-running human and agent work: durable state,... ...boundary, including validation, evaluations, and release gates... ...Thinks in invariants, threat models, failure modes, and user outcomes...SeniorFull timeH1bWork at officeVisa sponsorship
- ...of caring nurses while AI agents handle the tedious... ...navigates the healthcare system on your behalf, and propels... ...'re looking for an AI engineer who owns a business... ...start to finish, the evaluation infrastructure that... ...prompts, retrieval, and models. Make quality a number...SeniorRemote jobFull time
- # AI Agent Engineer — Early CareerBuild and verify the agent-native developer systems that help people and coding agents plan... ...Francisco, CAWork model: Remote-firstEmployment... ...window: Open until filledApply for... ...integration tests, and agent evaluations* Investigate failures...InternshipH1bRemote workFlexible hours
$190.9k - $334.1k
...DescriptionIt all started when engineer Fred Luddy wrote... ...ServiceNow is the AI control tower for... .... Develop novel AI Agent frameworks,... ...customer enterprise systems including ServiceNow... ...engineering, and data modeling. Curiosity &... ...ConsultingExperience level: Mid-Senior LevelIndustry:...SeniorWork at officeImmediate startRemote workFlexible hoursDay shift- ...startup (1-10 employees) building AI coworkers for IT teams: a security-focused product where an agent registers as a governed... ...for human approval, and drives systems it was never given an API for.... ...role builds the layer that turns model capability into systems that actually...SeniorFull timeContract workH1bVisa sponsorship
- ...employees and AI agents safely reach the... ...centralized MCP (Model Context... ...to enterprise systems such as Slack,... ...AI Enablement Engineer, you'll own the... ...technical direction. Evaluate emerging agent... ...decisions with senior ICs and SLG.What... ...Greenhouse to apply to our open roles.SeniorRemote work
- ...’s SeniorStaff AI Engineer is a hands-on AI... ...is a critical, senior role... ...scale.Distributed Agent Memory & State:... ...failoversDeep Model Optimization: Pioneer... ...LLMs, and NLP, evaluating and experimenting... ...LLM and agentic systems.Experience with... ...to apply to our open roles.SeniorRemote work
- ...Primarily On-site We are seeking an AI Evaluation Engineer – Reinforcement Learning & Agents to build the environments, evaluation systems, and supporting infrastructure... ...or publication history. Seniority The opportunity is open to exceptional new graduates,...
- ...Research Scientist, Research Engineer, AI Systems EngineerThe Recursive... ...feedback loops, designing evaluations, and training models to develop missing... ...research capabilities through agent harnesses, synthetic data... ...generalist who can move between open-ended research and...
- ...s most dynamic AI companies, like... ...bring cutting-edge models into production... ...the platform engineers turn to ship AI... ...: Evaluate, run and automate... ...contribute to open-source tools such... ...and analysis. System Profiling : Use... ...This is a mid-senior, high leverage...Full timeFlexible hours
- ...Description Founding Applied AI Engineer — Proactive AI Agents Location: San... ...equity Visa: Open to visa transfers... ...products, agent workflows, model orchestration, and... ...and agentic systems that observe user behavior... .... The team will evaluate hands-on technical depth...Full timeH1bRelocationVisa sponsorshipRelocation package
$293.5k
...Title Expert Senior Manager, AI Engineering Job ID 1043... ...of an integrated system that combines technology... ..., operating model changes, and... ...including multi-agent workflows and production... ..., and evaluation design to production... ...job postings for open roles. The estimated...SeniorPermanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week$268k - $358k
...platform where AI and automation... ...AI for Protein Engineering team develops and... ...and predictive models that move... ...looking for a senior individual contributor... ...Lila designs, evaluates, and learns from... ...software systems and broader reasoning... ...Publications, open-source contributions...SeniorFull timeWork at officeLocal areaFlexible hours- ...builds the layer that turns model capability into systems that work for users. You’ll develop the core agent harness—including the execution... .... You’ll build and run evaluations against replicas of real customer... ...Python/TypeScript and modern AI tooling Experience with...SeniorFull timeContract workH1bWork at officeVisa sponsorship
- ...Corporate Group in San Francisco invites applications for the Forward Deployed AI Engineer – Agents & Research Systems. You will build production-grade agent systems, data pipelines, and evaluation tooling for large enterprise deployments. The role blends software...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Agent Engineer - Open Models & Evaluation Systems. Be the first to apply!
- agent San Francisco, CA
- operations agent San Francisco, CA
- recovery agent San Francisco, CA
- registered agent San Francisco, CA
- tsa agent San Francisco, CA
- sourcing agent San Francisco, CA
- right of way agent San Francisco, CA
- work from home chat agent San Francisco, CA
- import export agent San Francisco, CA
- commissioning agent San Francisco, CA







