AI Evaluation Systems Engineer — Scale & Tooling
$214k - $300kMonograph
Monograph is seeking an engineer to build and improve AI evaluation systems aimed at increasing shipping quality for AI tools. You will enhance scalable eval runners, improve benchmarks, and ensure reliability in distributed systems. Strong engineering fundamentals and proficiency in TypeScript or Python are crucial. The position is based in San Francisco with a hybrid work model and a competitive salary range of $214,000 - $300,000 per year. #J-18808-Ljbffr Monograph
Vacancy posted 13 hours ago
Similar jobs that could be interesting for youBased on the AI Evaluation Systems Engineer — Scale & Tooling in San Francisco, CA vacancy
$320k - $405k
...Machine Learning Systems Engineer, Research Tools San Francisco, CA | New York City... ...interpretable, and steerable AI systems. We want AI to be... ...analytical skills and can evaluate the impact of engineering changes... ...team on just a few large-scale research efforts. And we...SuggestedWork at officeVisa sponsorshipFlexible hoursAnthropic
San Francisco, CA13 hours ago- A leading AI contracting platform is seeking an AI-native builder for its GTM AI & Systems team in San Francisco. This role focuses on replacing manual... ...integration with existing tools, and collaborating with various teams to scale solutions. Candidates should have...Suggested
$172.5k - $210k
A cutting-edge AI infrastructure firm located in San Francisco is seeking a Senior Systems Performance Engineer. This role involves leading hardware evaluations and optimizing AI systems for performance. Candidates should have over 5 years of experience, proficiency in...Suggested- ...Software Engineer Opportunity at Abridge Abridge's... ...This is a distributed systems oriented role and is approximately... ...and profiling tools to identify performance... ...will be under tremendous scale, and presents many... ...research as we pioneer new AI-first cloud-native-first...Suggested
Abridge
San Francisco, CA3 days ago - ...Our applied research engineers sit side-by-side with... ...believe it takes to bring AI to the enterprise. We... ...data foundations at Scale AI, and systems at Together, Two... ...techniques, and build the tools that let us... ...RL environments and evaluations for language models...SuggestedDaily paidWork at officeVisa sponsorshipRelocation package
$335k
...infrastructure that powers large-scale AI systems. We design and deliver next-... ...We are seeking a System Engineer (Network / Storage / Systems... ..., and cost efficiency. Evaluate vendor hardware and infrastructure... ...external vendors. Build tools and automation that improve...Work at officeRelocation package$266k
...develops silicon and system-level solutions... ...demands of advanced AI workloads. The team... ...creates custom design tools and methodologies that... ...Operating System Firmware Engineer to help bootstrap and scale the switching layer... ...baselines. Evaluate switch silicon SDK releases...Work at officeRemote workRelocation packageOpenAI
San Francisco, CA2 days ago- ...is looking for a candidate to drive research initiatives that influence engineering solutions. You'll build evaluations using real tool data, tackle search challenges for tools, and train systems for improved accuracy. Ideal candidates will have research experience, the...
$142.6k - $261.5k
...Using our product-driven, AI-centric approach, we... ...designers, and software engineers enable our clients to... ...methods, techniques, and evaluation criteria for obtaining... ...application development tools. Strong... ...practices. Knowledgeable in system development lifecycle and...Summer holidayFlexible hoursEY
San Francisco, CA1 day ago- ...building the automation engine that powers the... ...as the GenAI team scales. If you’re passionate... ...automation systems and want to shape the... ...expose data to internal tools and user‑facing... ...monitoring, debugging, and evaluating LLM behavior in... ...complex data and AI systems through well...Home officeFlexible hours
$250k - $350k
...AI is becoming vitally important in every... ...of our society. At Scale, our mission is to... ...an ML Sys Research Engineer, you'll work on building... ...to optimize our ML system. Your customer will... ...multi-agent/multi-tool rollouts.... ...fair and thorough evaluation of all applicants....Full timeScale AI
San Francisco, CA2 days ago- ...-site Department Engineering Our Mission Reflection... .... Our team of AI researchers and... ...throughput data ingestion tooling enabling... ...move faster as we scale. What This Team Does... ...environments. These systems form the... ..., deployment, and evaluation, unifying data, compute...Full timeRelocation package
$172.5k - $260.1k
...Salesforce is the #1 AI CRM, where humans... ...build the autonomous engines that power... ...technical depth to build systems from the ground up... ...Responsibilities*** Autonomous Scaling: Architect and... ...to ensure deployed tools are well governed.... ...assess and evaluate candidates’ resumes...Shift work$117.2k - $313.7k
...Category Software Engineering Job Details... ...Salesforce is the #1 AI CRM, where humans with... ...to provide scale and efficiency. Design... ...innovations that improve system scalability,... ...intelligence (AI) tools to help our recruiters assess and evaluate candidates' resumes...Immediate startRemote workSalesforce
San Francisco, CA2 days ago$150k - $180k
...technology company is seeking a skilled infrastructure engineer to build and maintain AI evaluation tooling. The ideal candidate has over 5 years of experience in software engineering with a focus on backend systems and production-grade infrastructure. This role offers...Remote jobFlexible hours$150k
Tzafon is seeking a skilled engineer to enhance their machine intelligence systems in San Francisco. As part of the team, you'll be responsible for building evaluation infrastructure, designing data pipelines, and implementing fine-tuning processes. Ideal candidates have...$120k - $170k
...manufacturing base. Our AI-powered robots automate food... ...Amazon Robotics, Chef is rapidly scaling with multiple multi-year... ...the Role Chef is seeking Systems Support Engineers to serve as a vital link... ...API integrations for Support tooling development Demonstrated...Work at officeFlexible hoursNight shiftChef Robotics
San Francisco, CA3 days ago- A leading AI research company in San Francisco is seeking a Systems Engineer focused on kernel optimization and AI-assisted workflows. You'll develop tooling to improve performance, debug infrastructures, and collaborate across teams to enhance production kernels. The ideal...
- ProducePay is seeking a Senior Storage Systems Engineer in San Francisco. This role involves high-performance data layer management, focusing... ...in storage administration, ensuring reliable data storage for AI workloads. Benefits include competitive compensation, comprehensive...
- ...leading tech company in California is looking for exceptional AI engineers to join their team as Members of the Technical Staff. The ideal... ...the forefront of artificial intelligence and has experience scaling backend services in production. This position offers an exciting...
$250k - $330k
...Principal AI Systems Engineer - Agentic Platforms The Moment The... ...reliably, securely, and at scale. Our platform supports deployment... ...-model execution, secure tool-calling integrations, and the... ...operability foundations, including evaluation, observability, auditing,...Remote workShift workKindo
San Francisco, CA2 days ago$140k - $185k
...power trusted, internet‑scale financial innovation.... ...for Marketing’s AI transformation by designing... ...insights into working systems—building automations, integrating tools, and shipping scalable... ...AI use cases (prompt engineering, chaining, evaluation, and optimization) Own...Flexible hours- ...Zensors Infrastructure Engineer Zensors is the spatial... ...the physical world. Our AI platform provides real-time... ...accuracy. To do this at scale, we rely on cutting-edge... ...not just in cloud-native tools, but also in the foundational Linux systems and networking required to...Work at office
Zensors
San Francisco, CA1 day ago - ...Ando is building AI-native workforce infrastructure... ...across static tools and manual... ...is rebuilding this system from first... ...term success. ML Engineer (AI-Native Systems... ...prompt systems, and evaluation frameworks Make... ...in mid-maturity or scaling organizations Deep...Hourly payContract work
Ando Technologies, Inc
San Francisco, CA2 days ago - ...Machine Learning Systems Engineer, RL Engineering San Francisco, CA | New... ...interpretable, and steerable AI systems. We want AI to be... ...Like working on systems and tools that make other people more productive... ...High performance, large scale distributed systems Large...Work at officeVisa sponsorshipFlexible hours
Anthropic
San Francisco, CA1 day ago - ...building next-generation AI creative tools. We are dedicated... ...re looking for an ML Engineer to architect and... ...personalization and recommendation systems from scratch. You'll... ...with large-scale data systems and... ...ranking, or aesthetic evaluation Interest in creative...H1b
Krea.ai, Inc
San Francisco, CA1 day ago $185k - $227k
...hiring the world’s best engineers, scientists, designers,... ...The Enterprise Systems Engineer will own the technical... ...the data pipelines and AI agentic workflows... ...Identify, pilot, and scale low-friction AI use cases... ...specifications documents using AI tooling - reducing manual...Remote work- ...AI & Partner Systems Strategy Manager The AI & Partner Systems... ...Partner Operations at scale. This role focuses on... ...generative AI, rules engines, and event-driven architectures... ...AI effectiveness Evaluate, prototype, and... ...insights, self-service tools, and intelligent user...Work experience placementWork at officeRemote workFlexible hours
NICE Actimize
San Francisco, CA1 day ago $170k - $220k
Senior AI Systems Engineer — Agentic Platforms The Moment The role of the software... ...reliably, securely, and at scale. Our platform supports... ...multi‑model execution, and tool‑calling integrations that interact... ...foundations, including evaluation, observability, failure isolation...Remote workShift work- ...using technology and AI. We empower personal injury... ...Machine Learning Engineer and build the future... ...architecture of large-scale ML systems for RAG, vector search... ...embedding pipelines and evaluation frameworks. Provide... ...gaps in ML tooling, infrastructure, and developer...Full timeTemporary workWork at officeLocal areaHome officeFlexible hours
EvenUp Inc.
San Francisco, CA1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Systems Engineer — Scale & Tooling. Be the first to apply!
Related searches
- machine learning ai engineer San Francisco, CA
- senior ai engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- ai ml engineer San Francisco, CA
- ai engineer San Francisco, CA
- ai developer San Francisco, CA
- ai research engineer San Francisco, CA
- ai prompt engineer San Francisco, CA
- operations support system engineer San Francisco, CA
- microsoft systems engineer San Francisco, CA