Remote Senior AI Agent Evaluation Engineer
YO AI Labs
Boston, MA
- Remote job
YO AI Labs is seeking a Senior Software Engineer contractor to help build reinforcement learning environments evaluating AI models on software tasks. The role focuses on creating reproducible environments, deterministic verification, and reference solutions for code challenges. Responsibilities include designing tasks such as bug fixing, feature development, refactoring, and performance optimization, while evaluating agents' reasoning with MCP tools. Remote, ~15 hours/week. #J-18808-Ljbffr YO AI Labs
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote Senior AI Agent Evaluation Engineer in Boston, MA vacancy
- ...leading security-first enterprise AI company, is seeking a Member... ...Staff in the Safety for Agents team to advance safer LLMs through... ...post-training algorithms, and evaluation methods. You'll collaborate with... ...across global offices, while remote-friendly policies support...Remote workSeniorFlexible hours
- YO AI Labs is seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using MCP tools. You will design reproducible environments, deterministic...Remote jobSenior
- ...Motors in the United States seeks a Senior Engineer on the Embodied AI Scaling Foundations team to measure... ...will design, implement, and iterate evaluation and introspection tools, collaborate... ...and launch decisions. This role is remote with travel up to 25% and relocation...Remote jobSeniorRelocation package
$144.7k - $261.3k
...DescriptionAbout the TeamThe Evaluation Foundations team, part of Embodied AI's Scaling Foundations,... ...vehicle development. We engineer high-performance tools... ....About the RoleAs a Senior Engineer in the Embodied... ...1 This role is based remotely, but if the selected candidate...Remote workSeniorFull timeLocal areaWork from homeRelocation packageFlexible hours$200k - $275k
...Secure Every Identity, from AI to Human Identity is... .... About Okta for AI Agents Okta secures access... ...platform, and security engineering teams, write production... ...patterns. Engage senior leadership. Brief the... ...outcomes end to end. #LI-Remote P25257_3456702...Remote workSeniorLocal areaWorldwideFlexible hoursShift workOkta, Inc.
San Francisco, CA2 days ago$152k - $241.5k
...the unlimited potential of AI to define the next era of computing... ...team. We are looking for a Senior Software Engineer to join our DGX Cloud /... ...Intelligence team and build agent-side systems that collect GPU... ...US, CA, Santa Clara; US, NY, Remote; US, RemoteType: Full timeRemote workSeniorFull timeLocal area$80 per hour
...ethically shape the future of AI. What We Do The... ...security researchers, engineers, and penetration... ...tools for running and evaluating agent behavior. You’ll implement... ...Take part in a flexible, remote, freelance project... ...expertise Other Information Seniority level: Mid‑Senior...Remote workHourly payPart timeFreelanceFlexible hours- ServiceNow is seeking a Senior Staff engineer to shape the next generation of agentic systems. You will own cross-cutting architecture, tool boundaries... ...and React/TypeScript frontend. Strong capability in multi-agent coordination, context management, and real-time execution is...Remote jobSenior
- BigCommerce Pty is seeking a Senior Software Engineer (AI Specialization) based in Austin, TX. You'll design and build AI agents, collaborating with product designers and NLP experts to create intelligent product experiences. This role requires a strong background in backend...Remote jobSenior
- BigCommerce Pty. is looking for a Senior Software Engineer (AI Specialization) based in Austin, TX to design and develop AI agents that enhance user interactions across its platform. The ideal candidate will have extensive experience in backend development, particularly...Remote jobSenior
- Commerce Inc is hiring a Senior Software Engineer with AI Specialization in Austin, TX. The ideal candidate... ...help design and build intelligent AI agents for e-commerce platforms. This role requires... ...for local applicants, promoting both remote and in-office collaboration. #J-1880...Remote jobSeniorWork at officeLocal area
- EPAM Systems is seeking a Senior Full-Stack Engineer to design, build and enable AI agents for QA, testing and software delivery. You will work with client champions... ...to deliver robust AI-assisted QA solutions. Remote work from Georgia is supported. #J-18808-Ljbffr...Remote jobSenior
$156.85k - $187.68k
...breakthroughs at the Intelligent Edge. ADI combines analog, digital, AI, and software technologies into solutions that combat... ...LinkedIn and X.Employer: Analog Devices, Inc. Job Title: Senior Engineer, Design Evaluation EngineeringJob Requisition: R262582Job Location: San Jose,...Remote workSeniorPermanent employmentFull timeWork at officeLocal areaWork from homeDay shift2 days per week$216k - $270k
About Scale AIScale AI is the data foundation for AI, helping... ...to build production AI agents that automate complex workflows... ...remains one of the hardest engineering challenges.As a Senior Frontier Agent Engineer (... ...customers to design, evaluate, and deploy intelligent systems...SeniorFull time- Sail is the foundation of useful, agentic AI. We are here to take a big swing at the most ambitious engineering challenge of our careers. Everyone working at Sail will... ...is just one piece of an effective background agent. Let's design and build the rest of the system,...SeniorWork at officeImmediate start
- Zillow Group’s Agentic AI team seeks a Principal... ...reasoning and long-running agent systems. You will partner with researchers, engineers, product leaders, and... ...capabilities. This senior individual contributor... ...emphasizes leadership, rigorous evaluation, and cross-functional...Remote jobSenior
- Smartsheet is seeking a Senior Product Manager for its Applied AI organization. You will own the roadmap for specialized AI agents and evaluation infrastructure, shipping complex agent capabilities... ...office, with the option to work remotely from anywhere in the US where...Remote jobSeniorWork at office
$184k - $287.5k
...unlimited potential of AI to define the next era... ...seeking an AI Compiler Engineer with deep expertise in... ....Lead experimentation, evaluation, and deployment with strong... ...LLM-based systems, agents, or developer tooling in... ...US, TX, Austin; US, TX, Remote; US, CA, Remote; US, WA...Remote workSeniorFull time$149k - $198.5k
Mission Summary:The AV Behavior Design and Evaluation Team is responsible for owning the... ...competencies, we are looking for passionate Engineers to help build a robust framework to... ...collaboration, or this role can be fully remote.The salary range for this role is an estimate...Remote workSeniorWork at office$186.1k - $292.81k
...Senior Engineering Manager CaptivateIQ is transforming the way companies... ...revenue performance. AI agents are becoming central to how... ...orchestration layer, and the evaluation and observability... ...cross-functional partners in a remote-first environment. High...Remote workSeniorFlexible hoursShift workCAPTIVATEIQ INC
United States2 days ago- YO AI Labs is seeking experienced Senior Software Engineers to support an AI training project remotely. You will create reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools. Responsibilities...Remote jobSenior
- Zscaler Senior Sales Engineer role in the South Majors team. Based in Texas, you will architect technical... ..., gather customer requirements, lead evaluations and configurations, run Proof of Value... ...with internal teams to drive success across deals. #J-18808-Ljbffr Remote JobsRemote jobSenior
- Asana is seeking a Privacy Engineer to lead privacy-by-design reviews for new features and major architectural changes, embedding privacy... ...regulatory requirements into concrete engineering standards, evaluate AI/ML privacy risks, and build systems enabling data rights and...Remote jobSenior
- Zscaler is seeking a Senior Sales Engineer to join our South Majors team. This role is based in Florida,... ...delivering technical product presentations, gathering customer requirements, leading evaluations, and running Proof of Value while collaborating #J-18808-Ljbffr Remote JobsRemote workSenior
- A leading AI consultancy is expanding by hiring 25 Senior Electrical Engineering Experts for part-time, fully-remote roles. This position focuses on designing complex engineering problems for AI and evaluating models in electrical engineering. Ideal candidates will have...Remote jobSeniorPart time
- ...our customer-facing Agentic AI product leveraging LLMs, agents, and internal APIs to create... ...to rollout. ~Partner with engineers to design prompt strategies, evaluation frameworks, and guardrails—balancing... ..., and beyond. We’re fully remote with offices in Palo Alto,...Remote workSeniorFull time
Jerry.ai
Remote10 days ago - Zscaler seeks a Senior Sales Engineer to join the North Carolina Majors team, guiding technical solutions... ...requirements and drive successful evaluations. You will lead Proof of Value,... ...outcomes for enterprise accounts in a remote/NC‑based setup. #J-18808-Ljbffr ZscalerRemote jobSenior
- ...seeking an experienced AI Technologist to drive the... ...with business units, engineers, and functional teams,... ...augmented generation (RAG), AI agents, and automation... ...Essential Functions:Identify, evaluate, and prioritize... ...may be performed fully remote, hybrid, or onsite at an...Remote workSeniorFull timeWork at office
- ...is for one of the Weekday's clients Now Hiring: Senior AI Agent Engineer Remote – United States Contract Position | 6+ Months An innovative... ...and manage Agent Skills, SKILL.md workflows, and evaluation suites • Contribute to Managed Agents, Claude Cowork...Remote workSeniorContract workImmediate startWeekday work
Weekday AI
United States3 days ago - ...Salary$1-$92 / hourSummary:Work Mode: Remote, EST requiredDuration: 12/31/2026... ...a zero-defect delivery posture.Evaluate automation requests and emerging AI capabilities to determine the optimal... ...to junior and mid-level engineers, establishing code quality standards...Remote workSenior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Senior AI Agent Evaluation Engineer. Be the first to apply!
