Senior LLM Evaluation & Agent Reliability Lead
Shopint
Shopint is seeking a part-time Senior LLM Evaluation / Agent Reliability Advisor based in Seattle, Washington. This role focuses on reviewing evaluation approaches and case structures for AI agents in commerce. The ideal candidate should have experience in LLM evaluations and agent reliability. The position requires roughly 2–5 hours per week, with flexible compensation options including paid advisory roles and potential equity. #J-18808-Ljbffr Shopint
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior LLM Evaluation & Agent Reliability Lead in Seattle, WA vacancy
- Position Summary Shopint is looking for a part‑time Senior LLM Evaluation / Agent Reliability Advisor. We are building a focused evaluation product for... ...Ideal Background Senior Applied Scientist, LLM Eval Lead, Agent Evaluation Lead, AI Quality Lead, or similar Experience...SeniorPart timeFlexible hours
$202.16k - $368.22k
...hotly contested space amongst leading Internet companies, and its future... ...and development of multi-agent tools, models, engines, and platforms... ...roadmap, and build a stable, reliable Agent infrastructure. What You... ..., multimodal, search, graph, LLM, Agent etc. to provide support...SeniorTemporary workLocal areaOverseasTik Tok
Seattle, WA2 days ago- Releady is seeking a Senior Observability Engineer to support a major airline in Washington. This client-facing role involves designing... ...Grafana, defining SLOs/SLIs, and contributing to enterprise reliability practices. The ideal candidate has over 5 years in SRE or...SeniorRemote job
- ...capacity ingestion and incident management within their infrastructure team. The role includes mentoring team members, ensuring reliability of infrastructure, and implementing automation standards to enhance efficiency. The ideal candidate will have a strong background...Senior
- ...Senior Principal AI Agent / ML Software Engineer The Senior Principal... ...technical strategy, lead multi-team execution... ..., memory, retrieval, evaluation, guardrails, and... ..., scale, and operate reliable, secure, observable,... ...Deep understanding of LLM application patterns,...Senior
Oracle
Seattle, WA4 days ago - ...Senior Software Engineer - AI Coding Agents At NiCE, we don't limit our challenges. We challenge... ...decision-making Work on LLM integrations, prompt engineering... ...Improve system performance, reliability, and observability Build evaluation and observability systems —...Senior
NICE Actimize
Seattle, WA2 days ago - ...A leading technology company in Seattle is seeking a seasoned software engineer with over 10 years of experience, specifically in distributed... ...the organization builds and scales agentic AI while ensuring reliability, governance, and security across various teams. Experience...SeniorRemote work
F5 Networks
Seattle, WA15 hours ago $40 - $45 per hour
Triple Canopy is seeking skilled Executive Protection Agents based in Seattle, WA. Successful candidates must have a minimum of 5 years of experience in executive protection, possess a valid driver's license, and demonstrate exceptional critical thinking and communication...SeniorFlexible hours- Expedia, Inc. is seeking a Senior Machine Learning Scientist to drive innovations in AI for travel technology. You'll design cutting-edge agentic systems and frameworks while collaborating with diverse teams to enhance user experience and solve real-world tasks. Ideal...Senior
$204k - $259k
...Senior Machine Learning Engineer – VLM/LLM Evaluation Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver... ...the quality, safety, and realism of embodied AI agents Partner with cross-functional teams within the organization...SeniorFull timeTemporary workRemote workWaymo
Kirkland, WA3 days ago$160k - $200k
...passionate about AI to design and build LLM powered systems in Seattle, Washington. The... ...production AI systems, designing reliable LLM architectures, and maintaining strong... ...language models, retrieval systems, and evaluation frameworks. Flexible time off and meaningful...SeniorFlexible hours- Apple Inc. is hiring a Sr. Research Manager in Seattle to lead an ML research team focused on advancing evaluation methods for AI systems. You will collaborate closely with applied and measurement scientists to develop methodologies and ensure they can be applied effectively...Senior
$130k - $300k
...information, please .Senior Staff Machine Learning Engineer, AI Agent Platform page is... ...deployment, and hosting of LLM-based AI agents.... ...management, evaluation frameworks, skill registries... ...Engineering:** Lead design of... ...that makes AI agents reliable for long-running workflows...SeniorHourly payWork experience placement$117.2k - $313.7k
...Salesforce is the #1 AI CRM, where humans with agents drive customer success together. Here,... .... This role focuses on designing reliable telemetry pipelines, improving monitoring... ...operate accurately and reliably. Critically evaluate code (Human or AI-generated) for...Senior- ...LLM/Prompt-Context Engineer – Fullstack Python (AI Agents, LangGraph, Context Engineering) Location – 1st Atlanta, 2nd Dallas, 3rd Seattle... ...Engineering: Design, optimize, and evaluate prompts for LLMs to achieve precise, reliable, and contextually relevant outputs...Remote work
Diversity Nexus
Seattle, WA15 hours ago $140k - $150k
...responsibilities related to platform development and administration. The role entails managing project development, responding to incidents, and leading a team of engineers. Important qualifications include 5+ years of Salesforce experience, strong leadership skills, and relevant...Senior$180.2k - $243.34k
A leading data and AI infrastructure platform is seeking a Senior Staff Technical Program Manager for Reliability in Seattle. This high-visibility role will lead critical initiatives across infrastructure and product engineering, focusing on enhancing the reliability and...Senior- ...global communities. The Senior Software Engineer - AI... ...powered automation and agent‑based systems across... ...engineering excellence. Evaluates emerging AI, GenAI, and... ...development 2 years of leading teams of four or more... ...AI concepts including LLM orchestration, autonomous...SeniorLive inLocal areaRemote workFlexible hours2 days per week
$55k - $151.47k
...experiences you need to lead and deliver value at this... ...validate Generative AI agents and data pipelines, promoting reliability, scalability, and alignment... ...PwC standards. As a Senior Associate you will analyze... ...platform Executing LLM evaluation frameworks using defined...SeniorFull timeWork experience placementH1bRemote workPwC
Seattle, WA15 hours ago$139.5k - $258.1k
Senior Applied Scientist - AI Evaluation & Quality Systems Seattle, Washington, United States... ...Engineering (ASE) powers the AI and LLM features behind... ...and tooling that generates reliable ground truth and detects quality... ...is the autonomous QA agents that make those...SeniorRelocationShift work$201.3k - $302.2k
...Staff Applied Scientist in Seattle, WA, focused on AI Quality and Meta Evaluation. This role involves designing and building a Data Quality Validation framework, ensuring the trustworthiness of LLM evaluations. Key qualifications include a Master’s degree in a related...Senior- ...We're building the next generation of AI evaluation systems — and we're looking for a hands-on... ...AI/ML Evaluation organization, seeking a Senior or Staff-level Applied ML Engineer with... ...Experience working on AI evaluation systems, LLM-based simulations, or agentic AI...Senior
Apple
Seattle, WA4 days ago - ...Senior Lead Cybersecurity Architect Play a vital role in shaping the future of an iconic company and make a direct impact in a dynamic... ...Ping, ForgeRock, CyberArk, Hashicorp Vault, and Dileania. Evaluate and recommend IAM products and integrations for cloud environments...Senior
Chase
Seattle, WA2 days ago - ...Senior Principal Software Engineer We're looking for a tech leader... ..., and deliver trusted market-leading technology products in a... ...AWS, ensuring observability, reliability, and cost efficiency. Leads... ...success architecting and deploying LLM & GNN solutions on AWS (e.g.,...Senior
Chase
Seattle, WA2 days ago $150k - $180k
...Senior Solution Lead, Pre-Sales & Solution Strategy (remote) • Seattle United States Please note... ...positioning solutions with AI integration , agent workflows, automation, or model-... ...stacks (Snowflake, Databricks, vector DBs, LLM orchestration frameworks) Background...SeniorRemote workFlexible hoursMonks Associates
Seattle, WA15 hours ago- A leading environmental consulting firm in Washington seeks a Senior Wetland Scientist to manage projects involving wetland delineations and ecological fieldwork... ...The successful candidate will lead critical area evaluations, navigate regulatory processes, and collaborate...Senior
- A technology company in Bellevue, Washington, is seeking a skilled Site Reliability Engineer (SRE) to join their engineering team. The role focuses on ensuring the reliability and performance of services, designing AWS infrastructure, and driving automation. Ideal candidates...Senior
- ...., based in Seattle, is searching for a Principal Architect to lead its CRM platform, oversee architectural integrity, and integrate... ...role emphasizes strategic technological direction, platform reliability, and the introduction of agentic AI solutions. A comprehensive...SeniorRemote jobWork at office
$148.7k - $201.2k
...Video Playback is seeking a Senior Technical Program Manager to... ...If you are interested in leading key strategic programs for Prime... ...to make Prime Video even more reliable for customers. In that capacity... ...- Experience building and evaluating system-level technical design...SeniorTemporary workWorldwideFlexible hoursAmazon.com Services LLC
Seattle, WA15 hours ago- ...aviation with cutting-edge solutions The Role As a Senior Software Engineer Evaluation, you will design and implement systems that measure and... ...and monitoring tools that ensure our AI systems perform reliably in real-world environments. You will help establish the...Senior
VTI Aerospace
Seattle, WA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior LLM Evaluation & Agent Reliability Lead. Be the first to apply!
