Senior AI Engineer, Agent Platform & Evaluation Harness
Jobleads-US
GEICO is seeking an experienced Sr. Staff AI Engineer for its AI Agent Platform to shape agent design, context strategies, and evaluation methodology at scale.
You will own the intelligence layer across enterprise workflows, partnering with backend engineers to ensure reliability and performance in production environments. The role emphasizes experimentation, data-backed decisions, and rapid prototyping of GenAI capabilities used by thousands of associates and millions of customers.
#J-18808-Ljbffr Jobleads-USVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior AI Engineer, Agent Platform & Evaluation Harness in New York, NY vacancy
$115k - $260k
...Culture, Great Rewards, and Great Careers. Sr. Staff AI Engineer, AI Agent Platform (Agent & Evaluation Harness) Why Join GEICO? GEICO is transforming how AI is... ...across the AI Agent Platform, in partnership with senior AI and engineering leaders. Mentor engineers and...SeniorHourly payWork experience placementLocal area$197.3k - $225.1k
AI Engineer 4 (Gen AI Platform Services: Agentic AI, Agent Guardrails, Agent Evaluation, Agent Memory) At Capital One, we are creating responsible and reliable AI systems, changing... ...standards are met Mentor Principal and Senior Associates on scalable design, performance...SuggestedFull timePart timeLocal areaCapital One
New York, NY8 hours ago- ...GEICO is seeking a Sr. Staff Software Engineer to lead the design and delivery of the AI Agent Platform core systems. You will own architecture... ...emphasizes platform work across agent harness, RAG, insight engines, and evaluation tooling, with hands‑on coding and technical...Senior
$229.9k - $262.4k
Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create...SeniorFull timePart timeLocal areaCapital One
New York, NY8 hours ago- ...Capital One is seeking an AI Engineer 5 for Gen AI Platform Services - Agentic Systems to design, build, and deploy responsible AI systems with broad impact. You will work with engineers, researchers, and PMs to advance foundation models, multi-model orchestration, and...Senior
$286.2k - $326.7k
Senior Staff AI Engineer - Agentic AI Platform (Remote Eligible) At Capital One, we are creating responsible and reliable... ..., large language model inference, agents and multi-agent workflows, similarity search, guardrails, model evaluation, experimentation, governance, and...SeniorFull timePart timeLocal areaRemote workCapital One
New York, NY8 hours ago$130k - $300k
...Careers. Sr. Staff Software Engineer, AI Agent Platform We are looking for a Sr... ...core systems: an agent harness that runs agentic... ...running offline and online evaluations, simulating scenarios, capturing... ...enterprise, partnering with senior AI and engineering leaders...SeniorHourly payWork experience placementLocal area$229.9k - $262.4k
...Overview Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning...SeniorFull timePart timeLocal area- ...its-kind centralized platform. With BlinkRx, never... ...us!We are hiring a Senior AI Security Engineer II to lead the security... ...identity management.Evaluate, deploy, and operate... ...and define agent-to-agent authentication... ...tooling, evaluation harnesses, and automated testing...Senior
Blink Health
New York, NY15 hours ago $125.8k - $209.7k
...OpportunityAs a Cyber AI Engineer at the Senior Consultant level in... .... You will build, evaluate, and deploy AI solutions... ...datasets and test harnesses to measure accuracy,... ..., tool-calling agents, evaluation harnesses... ...and secure security-platform connections.Ability...SeniorImmediate startEY (Ernst & Young)
New York, NY4 days ago- As an AI Platform Engineer for AI & Emerging Tech, you will drive AI platform... ..., access patterns, skills, agents, MCP integrations, platform... ...Automated testing and evaluation frameworks (Ragas, Playwright... ...Seniority Senior...Senior
Luxoft
New York, NYmore than 2 months ago $150k - $350k
...millions of people around the world to run AI on their own computers, and now with... ...The Role We are looking for an Agent Harness Software Engineer to build the core agent system behind... ..., and recovery Build repeatable evaluations and diagnostics for understanding where...Work at officeLocal areaWork from homeFlexible hoursLM Studio
New York, NYa month ago- ...Capital One is building an AI-native platform to empower our business teams with AI-assisted analytics and production-grade AI systems. The team collaborates across engineering, research, and product disciplines to deliver scalable AI-powered products for associates and...SeniorRemote job
- ...Capital One is hiring an AI Engineer 5 in New York, NY to partner across engineering, research, and product teams. You will design and... ...AI software components, including model training, inference, agents, and multi‑model workflows, leveraging Open Source and SaaS AI...Senior
- ...Compass, Inc. is seeking a security engineer to own the AI product security roadmap, enforce threat modeling, and build real-time protections for AI features. You will work with the AI platform and product teams to ensure secure, compliant, and scalable AI capabilities...Senior
- ...we are expanding the platform to include agentic workflows... ..., deploys, and scales AI-driven identity... ...the Head of New Product Engineering, you'll join a new Internal... .... Production Agent Design & Development... ...Excellence – Leverage the evaluation harness and tracing substrate...Senior
Socure Inc
New York, NY15 hours ago $200k - $240k
...is building the AI operating system... ...our suite of AI agents to help logistics... ...agents themselves—the harnesses, evals,... ...entrepreneurial Senior AI Agent Engineer to join as a founding... ...product—an agentic platform that synthesizes... ...you scope, build, evaluate, and ship without...SeniorTemporary workLocal areaDay shiftTraba
New York, NY4 days ago- ...As the Lead AI Engineer for our next-generation TCO Agent Platform, you will serve as the technical lead and multi-agent system architect for a greenfield,... ...Platform, and Enterprise Architects. You will also have a Senior Software Engineer under your direct subordination...
EPAM Systems
New York, NY18 days ago $65k - $102k
...We require 8+ years of software engineering experience, including 3+ years in... ...leadership role building multi-agent AI systems, FinOps tools, or LLM-powered platforms. We seek advanced proficiency... ...repositories. You will mentor Senior and Mid-level AI engineers and...Full timeEPAM Systems
New York, NY12 days ago- ...Bot Jobs is seeking an Agent Deployment Engineer in New York City to own the end-to-end build and launch of AI agents for enterprise customers. This hands-on role blends technical... ...responsibility inside a growing agent platform team. You will write agent-building artifacts...Senior
- ...Senior AI Agents Developer - New York City Salary Details: $500,00... ...strategy. Our expanding AI Engineering team, working directly alongside... ...-grade AI agents, evaluating emerging technologies, and... ...benchmark external AI models, platforms, and services Work closely...SeniorFull timeContract workNew York, NY4 days ago
- ...LatentBridge is seeking an experienced AI Engineer to join a platform engineering initiative focused on building Micro UIs across an enterprise platform. The role blends applied AI engineering with full-stack delivery (.NET, Angular) and backend document intelligence...Senior
- ...engineeringjobs.net, Inc. is hiring a full-stack engineer to extend Hazel’s action layer and build AI-driven workflows across Shopify, Klaviyo, Meta, and Amazon. You will own API-driven features, optimize LLM performance, and ensure a superb user experience in a fast-...Senior
- ...Capital One is building responsible AI systems to transform banking experiences. The... ...Experiences team leads state-of-the-art AI engineering across models, pipelines, and governance.... ..., researchers, and PMs, and mentor senior engineers while shaping the roadmap for scalable...Senior
$130k - $147k
...the center of every decision. Our AI-first platform transforms proprietary data, advanced... ...Description We are seeking a Senior ML/AI Platform Engineer to help build and operate Curinos'... ...business outcomes. Implement scalable evaluation and monitoring frameworks for LLMs,...SeniorPart timeWork at officeRemote workWork from homeFlexible hoursCurinos Inc
New York, NYa month ago- ...Capital One is seeking a Sr. Staff AI Engineer to join the Intelligent Foundations and Experiences... ...training, LLM inference, and multi-agent workflows, while advancing model safety... ...available, with compensation aligned to seniority and location. #J-18808-Ljbffr Jobleads...SeniorRemote job
- Title: Senior AI Platform Engineer Client: CPPIB Location: Hybrid - New York City, NY Visa: USC/GC... ...architectures, including building and supporting agents, skills, and MCP servers. Gen AI... ...optimization. Automated testing and evaluation frameworks (Ragas, Playwright,...Senior
$143.2k - $243.4k
...Description It all started when engineer Fred Luddy wrote code that... .... Today, ServiceNow is the AI control tower for business reinvention. Our ServiceNow AI platform brings together any AI, any data... ...About the Role : As a Senior AI Agent Engineer / Federal at Moveworks...SeniorPermanent employmentWork at officeImmediate startRemote workFlexible hours- About the job Senior AI Platform Engineer Job Title: Senior AI Platform Engineer Location: New York (... ...Troubleshoot issues related to skills, agents, MCP integrations, and API usage... ...PGVector, Neo4j, Neptune) Automated testing/evaluation frameworks (Ragas, Playwright,...Senior
- ...Description Job Description Senior Software Engineer Job Type: Contractor (~15... ...Engineers to support an AI training project by creating... ...learning environments that evaluate AI models on complex software... ...reference solutions. Evaluate AI agents' ability to reason through...Remote jobFor contractors
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Engineer, Agent Platform & Evaluation Harness. Be the first to apply!
Related searches
- senior ai engineer New York, NY
- machine learning ai engineer New York, NY
- ai ml engineer New York, NY
- ai developer New York, NY
- ai prompt engineer New York, NY
- ai engineer New York, NY
- ai engineer remote New York, NY
- operations agent New York, NY
- work from home chat agent New York, NY
- freight broker agent New York, NY