Senior AI Agent Engineer - Open Models & Evaluation Systems
Sail Research
Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will become an expert; nothing less will do in our immensely competitive market. Inference is just one piece of an effective background agent. Let's design and build the rest of the system, that turns billions of tokens into the best possible answers. What you’ll do Design custom evals for multi-turn, massively parallel agents. Build agent harnesses to improve open model (Deepseek, Qwen, Llama) performance. Claude Code is all about agent/harness codesign; let's do the same for open source! Automate prompt optimization techniques like DSPy. What we’re looking for Experience building AI agents. Familiarity with open source models. Interview process Meet the CEO. This is the first step because we respect your time. Ask any question and get a definitive answer immediately. Meet the CTO, who will ask about your experience, and share as much technical detail about Sail as you want to hear. Come in to Sail's SF office for an interview day. Meet the whole team, then you'll have 3-4 hours to work on a problem that closely simulates the work we do daily. It's an objectively scored task, so you'll have immediate feedback on how well your code is working - just like we do in production! AI assistance is highly encouraged, and we'll provide a laptop with all the best tools set up. Finish with a short presentation describing your process, learnings, and results. Offer. Once the team decides we want to work with you, we make a strong offer quickly and will be quite persistent over email/text/calls :) Life at Sail We work out of a beautiful, sunny office in downtown San Francisco. All meals are on us (and actually great; SF is a food paradise and it would be a shame to eat only bowl slop). Everyone gets a Studio Display at their desk. We are serious about investing in anything that saves us time or energy. There are six different ways to make coffee or tea in the office. A friendly (hypoallergenic) black cat named Coco visits occasionally. #J-18808-Ljbffr Sail Research
- B Capital seeks a talented individual for an AI Evaluation role in San Francisco. This position involves conducting... ...comparative analysis, refining evaluation systems, and collaborating with various teams to enhance model capabilities. The ideal candidate will have strong...Suggested
- Cline is building open-source AI agents for developers and power users. We seek an experienced engineer to design and implement the agent systems behind the platform—runtime, orchestration, tools,... ...across teams, and contribute to a model-agnostic, open-weight capable agent...Senior
- ...caring nurses while AI agents handle the tedious... ...the healthcare system on your behalf, and... ...looking for an AI engineer to build the LLM-powered... ...to finish, the evaluation infrastructure... ...prompts, retrieval, and models. Make quality a... ...Passionate and open. You exhibit enthusiasm...SeniorFull time
- ...and optimizing features for an AI runtime and SaaS platform. The... ...years of experience in backend systems, proficiency in Python and C++,... ...teams. You will contribute to open-source initiatives and help shape... ...position offers a hybrid working model with a hands-on approach to AI...Senior
- Sail is advancing AI with a focus on agentic systems and scalable evaluation. This role centers on designing custom evals for multi-turn, massively parallel agents and building harnesses to improve open models. You’ll also help automate prompt optimization to push performance...Suggested
- AI Systems Engineer - Codex Core Agents Location San Francisco Employment Type Full time Department... ...Codex agents interpret model outputs, use tools,... ...development environments. Develop evaluation, experimentation, and... ...for other teams and open-source users to build on....Full timeWork at officeLocal areaRelocation packageFlexible hours
$166.9k - $225.9k
...thoughtful hybrid model because we believe... ...intelligent AI capabilities into... ...We are seeking a Senior AI Engineer to help design, build... ...robust, high-impact AI systems that improve... ...LLM + retrieval + agent systems in production... ...reliability, and evaluation in real-world enterprise...SeniorFull timeWork at officeImmediate startWorldwideMonday to FridayFlexible hours$200k - $300k
...startup building AI “coworkers” for IT... ...product in which an agent operates as a... ..., and can operate systems it was never handed... ...one of the earliest engineers alongside a small... ...layer that turns raw model capability into... ...experience across model evaluation, fine-tuning, and/...SeniorFull timeContract workH1bWork at officeVisa sponsorshipMonday to Friday$139k - $257.55k
..., reusable design systems, and collaboration... ...next generation of AI-native creative... ...both users and AI agents. Our mission is to... ...forward-thinking engineers who are excited to... ...experience.Develop evaluation and quality frameworks... ...this role is open to hiring in Colorado...SeniorFull timeTemporary workLocal areaWorldwide- Arcadia is looking for a Senior Software Engineer (AI Agents) in San Francisco. In this pivotal role, you will build core systems for an intelligent automation platform used in operations-heavy industries. Your responsibilities will include designing production-grade LLM...Senior
- Cacheflow is seeking a Senior Applied Research Engineer to enhance the effectiveness of our AI systems through focused research and experimentation. This role involves designing information retrieval strategies and collaborating with engineers to turn validated approaches...SeniorFlexible hours
$240k - $280k
...software monitoring company is seeking a Senior Software Engineer on its AI/ML team to build evaluation infrastructure for measuring the performance of AI systems. This role involves designing... ...The position offers a hybrid work model and a salary range of $240,000 to $...Senior- ...building the next generation of AI agents for the insurance industry -... ...for an exceptional AI Engineer to join our agent team and help... ...techniques (e.g. multi-agent systems, reasoning-from-feedback, RFT... ...Note that this is not a model-training role - you’ll be building...SeniorFull timeWork at office
$200k - $400k
...Job Title: Founding AI Engineer Location: San Francisco, CA (On-site... ...design, build, and launch a multi-agent AI system that will redefine the future of talent evaluation. You’ll join at a pivotal... ...DNA — hackathon wins, notable open-source contributions, or personal...SeniorFull timeImmediate startRelocationVisa sponsorshipRelocation packageFlexible hours$190.2k - $360.5k
...looking for a Principal AI Systems Engineer with deep C++... ...integration layers, evaluation systems, and reliability... ...-powered services, agents, and model-backed workflows. Define... ...teams. Mentor senior engineers and raise... ...NoticeIf this role is open to hiring in Colorado...Full timeTemporary workLocal areaRemote workWorldwide$163k - $246.5k
...as you build, with AI that learns your context... ...role As an AI engineer, you’ll apply LLM... ...and rigorously evaluate the efficacy of different prompts and models through experimentation... ...provided you are open to learning them quickly... ...employee with a system that equally...SeniorFull timeCurrently hiringLocal areaRemote workWeekend work3 days per week$314.8k - $359.3k
{"description": "Senior Distinguished AI Engineer At Capital One... ...responsible and reliable AI systems, changing banking... ...customers. Our AI models and platforms... ..., guardrails, model evaluation, experimentation, governance... ...a broad stack of Open Source and SaaS AI...SeniorFull timePart timeLocal area- ...scale autonomous AI agents with full... ...looking for a Senior Applied AI Engineer to join our on-... ...improving production systems . You will work... ...including agent evaluation, observability,... ...maintain pipelines for model evaluation,... ...LLMs (APIs or open-source models) in...SeniorFull time
- ...part of Schwab’s AI Strategy &... ...together product, engineering, strategy, and... ...We’re seeking a Senior AI Engineer who... ...ensure that the systems you build are... ...leveraging AI models for measurable... ...proprietary or open-source LLMs (e.... ...writing and running evaluations for quality and...SeniorFull timeWork at office
$190.9k - $334.1k
...DescriptionIt all started when engineer Fred Luddy wrote... ...ServiceNow is the AI control tower for... .... Develop novel AI Agent frameworks,... ...customer enterprise systems including ServiceNow... ...engineering, and data modeling. Curiosity &... ...ConsultingExperience level: Mid-Senior LevelIndustry:...SeniorWork at officeImmediate startRemote workFlexible hoursDay shift$2,000 per month
...is the leading AI analytics... ...With powerful AI Agents embedded across... ...looking for a Senior AI Engineer to help advance... ..., build the systems that deliver it... ...orchestration, evaluation, retrieval,... ...AI ecosystem: model APIs, agent frameworks... ...empathy and openness, seeking to...SeniorHome office$128k - $252.5k
...covering valuation modeling, cost... ...scientists to AI strategists, machine... ...specialists, and data engineers. SFL Scientific... ...to add a Senior AI Engineer to... ...cloud/enterprise/open-source solutions... ..., autonomous systems and edge AI, and... ...understand and evaluate new and essential...SeniorLocal areaVisa sponsorship- ...building intelligent systems that sit at the... ...of AI, biology, chemistry... ...and large-scale engineering. Our goal is to... ...The Role As a Senior AI/ML Engineer,... ...learning models that form the core... ...Design, train, and evaluate large-scale... ...Contributions to open-source ML frameworks...SeniorFull timeRemote workFlexible hours
- Senior AI Architect - Multi-Agent Systems & Platform Infrastructure Senior AI Architect - Multi... ...Orchestration / Head of Engineering Seniority: Senior-Level (... ...and refine test plans, evaluation pipelines, and debug tools... ...LLMs • Contributions to open-source AI orchestration or...SeniorFull timeWork at officeRemote work
$200k - $250k
...building the AI Search Infrastructure... ...modern AI systems. Our goal is... ...layer that agents, applications... ...language models, and large-scale... ...includes engineers, researchers,... ...vertical feeds, evaluation tooling, and... ...credibility with senior engineers and... ...footprint — open-source,...SeniorFull timeImmediate startRemote workWork from homeFlexible hoursEarly shift$180k - $215k
...flagship product—an AI-driven, non-... ...-coded, 3D model of a patient’... ...both the Senior and Staff levels... ...product engineering. You won't just... ...intelligent agents to execute multi... ...agentic systems from scratch,... ...guardrails, evaluation frameworks, and... ...solving complex, open-ended...SeniorLocal areaWorldwideRelocation$229k - $281k
MissionAs a AI Accelerated Engineering Lead, you will guide... ...where AI agents and engineers... ...practices such as evaluation harnesses,... ...leadership in system or platform architecture... ...databases, or model routing... ...to open source or thought... ...San Francisco* Senior Principal: $22...SeniorTemporary workLocal areaShift work$124k - $280k
...in data and analytics engineering focus on leveraging advanced... ...implementing advanced AI and ML solutions to... ...algorithms, models, and systems to enable intelligent... ...staying relevant.Initiate open and honest coaching conversations... ...with team members. We evaluate these factors...SeniorFull timeH1b$293.5k
...Title Expert Senior Manager, AI Engineering Job ID 1043... ...of an integrated system that combines technology... ..., operating model changes, and... ...including multi-agent workflows and production... ..., and evaluation design to production... ...job postings for open roles. The estimated...SeniorPermanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week- ...AI Systems EngineerTransluce is a fast-moving research... ...industry standards for evaluation. We are a non-profit with... ...AI systems engineer to lead the design and... ...organisational reach (open-source tools the entire... ...enough to allow complex model introspection and intervention...Flexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Agent Engineer - Open Models & Evaluation Systems. Be the first to apply!
- fbi agent San Francisco, CA
- booking agent San Francisco, CA
- tsa agent San Francisco, CA
- state farm agent San Francisco, CA
- import export agent San Francisco, CA
- remote chat agent San Francisco, CA
- agent San Francisco, CA
- executive protection agent San Francisco, CA
- right of way agent San Francisco, CA
- showing agent San Francisco, CA



