Evaluations Lead AI Model Progress & Metrics
Cartesia, Inc.
Cartesia is looking for an Evaluations Lead to design frameworks that measure AI model interactions, focusing on understanding, naturalness, and adaptability in real-world settings. The role blends research, product, and infrastructure, requiring a candidate with a scientific mindset and technical skills to create impactful evaluation systems. Join our in-person team in San Francisco and dive into pioneering AI methodologies while enjoying competitive benefits. #J-18808-Ljbffr Cartesia, Inc.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Evaluations Lead AI Model Progress & Metrics in San Francisco, CA vacancy
- ...interpretable, and steerable AI systems. We want AI to... ...Engineers to build the evaluations that tell us — and the... ...into clear, defensible metrics that researchers,... ...leadership use to monitor model health during training,... ...results as training progresses Run experiments to characterize...SuggestedFull timeWork at officeVisa sponsorshipFlexible hours
$200k - $400k
Simile is seeking a Member of Technical Staff, Model Evaluations, to develop measurement systems that gauge the accuracy of simulations of human behavior. You will design evaluation metrics and collaborate with modeling teams to ensure models are effective. Ideal candidates...Suggested- Anthropic is seeking a Research Lead for the Training Insights team to shape the evaluation of model capabilities. This hands-on leadership role involves developing innovative... ...You will play a crucial role in transforming how AI capabilities are assessed, working...SuggestedRemote work
$176k - $253k
Harper Group, an AI-native commercial insurance company in San Francisco, is looking for a Senior Member of Technical... ...agent quality assessments from vague estimations to concrete metrics, ensuring agents are evaluated, tested, and monitored effectively. Candidates should...Suggested$50 - $75 per hour
A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in...SuggestedHourly payContract work$350k
...Our first goal is to democratize frontier AI R&D across scientific disciplines. We... ...AI research company and training our own models end-to-end. Our work spans areas such as... ...looking for a research engineer to build the evaluation infrastructure that tells us whether our...- Obsidian is seeking an independent contractor for a remote role focused on benchmark dataset projects that evaluate AI models on visual document understanding in the Energy/Utilities domain. The role requires authoring complex tasks and providing objective rubrics to assess...Remote jobFor contractorsFlexible hours
$25 per hour
Prolific is seeking AI Training Experts to assist in training and evaluating cutting-edge AI models. The role involves completing tasks such as analyzing and writing annotations, and judging AI performance. Candidates should have professional experience as an AI Trainer...Remote jobHourly payWork from homeFlexible hours- Welocalize is seeking a Data Quality Associate to evaluate AI model outputs and provide structured feedback. This is a full-time, onsite role located in San Francisco. The ideal candidate possesses a Bachelor's degree and has 1-2 years of professional writing experience...Full time
- The Consulting Solutions is seeking a specialist to define AI model behavior in high-risk contexts, focusing on safety and ethical standards. The role involves developing policies to balance the benefits and risks of AI systems, ensuring they adhere to safety norms. Key...
- .... is seeking a Technical Business Development professional in San Francisco, CA. You will work directly with partners on AI infrastructure and model launches, serving as their primary contact and ensuring successful integration. Your role involves significant collaboration...
- Welocalize is seeking a detail-oriented Data Quality Associate in San Francisco. This role involves evaluating AI model outputs, providing feedback, and ensuring data quality. Candidates must have native-level language proficiency and a Bachelor's degree. The position requires...
$146.2k - $261.4k
...Position Description RAND’s Center on AI, Security, and Technology (CAST), part... ...research and policy analysis projects, and leading multidisciplinary teams of policy... ...scientists. Your team will build systems to evaluate how AI models perform across the full attack...Fixed term contractWork experience placementRemote workWork from homeDormont Manufacturing Company
San Francisco, CA4 days ago$300k - $320k
...We are seeking a Technical Program Manager to lead our AI model evaluation initiatives across multiple workstreams. This role... ...projects to build new processes, align metrics with policy, and track measurable progress. You will help build and adapt the model evaluation...Work at officeHome officeVisa sponsorshipRelocation package- ...world. Backed by leading investors like PayPal... ...a future where AI is embedded into how... ...communicate progress, risks, and outcomes... ...each org’s operating model Establish Measurement... ...Define success metrics for each... ...Standardize how we build, evaluate, and scale AI solutions...Work at officeRemote work2 days per week
$90k - $110k
Common Sense Media is seeking an Evaluations Partner Manager in San Francisco, California. This role involves managing the operational execution of the Youth AI Safety Institute's evaluation work, with responsibilities such as coordinating workflow between internal teams...Full time- A cutting-edge AI technology firm in San Francisco is seeking an Evaluation Lead to drive the assessment of AI model performance. You will design evaluation methodologies, automate evaluation processes, and oversee various evaluation strategies. The ideal candidate has...
- ...for a builder to help lead our ‘AI for Work’ efforts. Together... ...That means building, evaluating vendors, and... ...Integrate large language models, agentic frameworks, and... ...signals that demonstrate progress, partnering with... ...experiments, interpret metrics, and distinguish meaningful...Hourly payWork at officeShift work
- About Archetype AI Archetype AI is developing the... ...building a foundation model for the physical world... ...Series A stage and is progressing rapidly to develop... ...is seeking a hands‑on Evaluation Lead to build and assess model... ...refining evaluation metrics Deep understanding of...
$180k - $260k
A tech-focused investment firm in San Francisco is seeking a Model Behavior Architect to enhance their answer engine by collaborating with... ...teams. Candidates should demonstrate a strong passion for AI, be familiar with Python, and possess a high-level understanding...Full time- About the Role We’re looking for a Model Behavior Architect to help build the company’s AI products and evaluations. You’ll sit within our AI team and collaborate closely... ...Experience designing evaluations, benchmarks, or metrics for AI systems. Strong written and verbal...
$315k
...interpretable, and steerable AI systems. We want AI to be safe... ...the role Anthropic's production models undergo sophisticated post-... ...pipelines for model fine-tuning and evaluation Develop tools to measure and... ...navigate ambiguity and make progress in fast-moving research environments...Work at officeVisa sponsorshipFlexible hours$405k
...interpretable, and steerable AI systems. We want AI to... ...systems, tooling, and evaluation infrastructure that... ...frameworks that measure model capabilities across... ...our research roadmap Lead the design of infrastructure... ...and how we measure progress against them Mentor and...Visa sponsorship- Xcede is looking for a Member of Technical Staff focused on AI Safety to lead red-teaming efforts and ensure the robustness of next-generation AI systems. The selected candidate will design scalable safety frameworks, partner with researchers to define production safety...
$207k - $285k
OpenAI is seeking a Technical Program Manager in San Francisco to lead initiatives that ensure the safety and robustness of its AI models. The role involves collaborating with diverse teams to turn risks into actionable plans. Ideal candidates will have experience in technical...- ...Designing and running experiments to improve model capabilities Developing training methods, architectures, and evaluation metrics Iterating quickly on research ideas using large... ...on ambitious problems at the intersection of AI, research, and science , we’d love to hear...
$100 per hour
Turing, based in San Francisco, is seeking finance experts to improve AI model performance. Candidates with a background in capital markets, portfolio management, or related fields are invited to apply. This role offers flexible hours (10-30 hrs/week) and allows you to...Remote jobFlexible hours- ...Hopper is looking for a Lead, Finance Analytics & Enablement AI/ML to own our data warehouse... ...infrastructure and metrics across finance and the business... ...data pipeline and AI model foundation. Understand and... ...prompt design, and AI model evaluation frameworks. Familiarity...Work experience placementWork from home
Drive Capital
San Francisco, CA2 days ago $127.49k - $262.2k
...training facility, and leading market tools, we help our... ...delivery of data / AI engagements (including... ...mitigating risk in data, modeling, validation, delivery)... ...design through training, evaluation, validation, implementation... ...report key business metrics to management; design,...Full timeLocal areaKPMG
San Francisco, CA4 days ago- ...Lead Product Manager, Agentic AI As the Lead Product Manager for Agentic AI on Hinge Health's Intelligent... ...outcomes. Own company-level metrics tied to Robin's success: including engagement... ...and iteration. Own evaluation framework: LLM-as-a-judge, golden datasets...Work at officeLocal area3 days per week
Hinge Health
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Evaluations Lead AI Model Progress & Metrics. Be the first to apply!
