Staff Engineer - AI Evaluation & Metrics Platform
Kindredventures
Kindredventures is seeking research engineers to build a central evaluation framework and scalable evaluation pipelines for models and datasets. You will design benchmarks, implement baselines, and create dashboards that translate results into actionable insights for the team. The role emphasizes measurable rigor, collaboration with research and domain teams, and end-to-end ownership of evaluation tooling from data collection to automated metrics. #J-18808-Ljbffr Kindredventures
Vacancy posted 14 hours ago
Similar jobs that could be interesting for youBased on the Staff Engineer - AI Evaluation & Metrics Platform in San Francisco, CA vacancy
$200k
...Manufacturing Co is seeking a Member of Technical Staff to manage our internal evaluations platform, essential for improving AI model performance. You will be responsible... .... The ideal candidate has strong software engineering fundamentals and experience with machine learning...Platform$200k - $400k
...that. We have built the first AI simulation of society,... ...Role As a Member of Technical Staff in Evaluations Engineering, you will build the systems... ...systems, benchmark platforms, regression suites, experiment... ...intervals, and distributional metrics. Sensitive Data and Access...PlatformFlexible hours- ...the world come to our platform to find creative ideas... ...Possible. At Pinterest, AI isn't just a feature,... .... We’re looking for a Staff ML engineer to develop core... ...systems: problem framing, metrics, data/feature design, model training, evaluation, and online...PlatformFull timeWork at officeRelocationRelocation package
Pinterest
San Francisco, CA14 hours ago $252k - $315k
About Scale AIScale AI is the data foundation... ...one of the hardest engineering challenges.As a Staff Frontier Agent... ...customers to design, evaluate, and deploy intelligent... ...customer intelligence platform, a healthcare... ...using quantitative metrics.Excellent communication...PlatformFull time$242.8k - $357k
...experiences. As a Senior Staff Engineer, you will lead the technical direction for AI-first experiences,... ...understanding. Rigorously evaluate ML and LLM models... ..., designing metrics and experiments that clearly... ...the fastest growing Ads platforms in the world and we're...PlatformHourly payWork at officeLocal areaRemote workFlexible hoursDoordash Usa
San Francisco, CA14 hours ago- Causal Labs in San Francisco is building a central evaluation framework for a Large Physics foundation Model, enabling measurable, comparable... ...teams. The role emphasizes end-to-end ownership and rapid execution in a hands-on engineering culture. #J-18808-Ljbffr Causal LabsPlatform
$170.4 per hour
...about enabling Data & AI teams to solve the world... ...and AI infrastructure platform so our customers can... ...business. Founded by engineers, we leap at every opportunity... ...and profiling (using metrics and heap/thread dumps)... ...of model training, evaluation, and deployment in...PlatformLocal areaWorldwide$190.72k - $290k
...combining frontier agentic AI, an enterprise-grade platform, and deep domain... ...started.Role OverviewAs a Staff Database Engineer on the Engineering team... ...database engineering in vendor evaluations, build-vs-buy decisions,... ..., and you know which metrics actually predict incidentsExperience...PlatformRelocation packageHarvey
San Francisco, CA5 days ago- ...We are seeking a Staff Machine Learning Engineer with deep expertise in agentic AI — and a true passion for... ...design, build, test, evaluate, and productionize next... ...observability (tracing, logging, metrics), and human-in-the-... ...or advanced analytics platforms) — designing,...PlatformFull time
Medeloop
San Francisco, CA14 hours ago $220.8k - $298.8k
# Staff Applied Research EngineerHybrid -... ...path to redefine how AI and General AI... ...seeking an Applied AI Engineer to drive the... ...experimentation, and evaluation. In this role, you... ...owning the quality metrics and evaluation systems... ...'s compliance platform is document-heavy:...PlatformWork at officeImmediate startWorldwideMonday to FridayFlexible hours$264.8k - $331k
Scale AI is the data foundation for AI, helping... ...our data annotation platform, generative AI... ...the RoleAs a Senior/Staff Machine Learning Engineer (MLE) on the General... ...and system design to evaluation, deployment, and iteration... ..., environments, and metrics to measure agent...PlatformFull time- ...started Handshake AI and built the fastest... ...to create evaluations, publish benchmarks... ...institutions Work alongside engineers, scientists,... ...is hiring a Staff Machine Learning Engineer... ...the Handshake platform. You'll operate as... ...marketplace's core metrics, and you'll be a key...PlatformFull timeWork at officeRemote workFlexible hours
$228.6k - $342.8k
...the world's best data and AI infrastructure platform so our customers can use... ....We are hiring a Senior Staff Applied AI Engineer to own context retrieval... ...ranking, retrieval, and evaluation — across the Enterprise... ...beyond ranking metrics to measure subagent decision...PlatformWork at officeLocal areaImmediate startWorldwide$230k - $322k
...find them useful. As a Staff Machine Learning Engineer on Shopping Ads, you... ...selection, offline evaluation, online... ...Foundational Insights, ML Platform, Ads Serving, Auction... ...other outcome-based metrics. Strong hands-on experience... ...intelligence (AI). You will have the...PlatformFor contractorsWork experience placementFlexible hoursShift work- ...Kastle is building the AI employee for consumer lending... ...a Founding Applied AI Engineer, you'll be instrumental... ...of Kastle's AI platform. You'll fine-tune large... ...performance and compliance Evaluate LLMs and AI Agents:... ...Optimization: Monitor performance metrics and continuously...Platform
Kastle AI Inc
San Francisco, CA2 days ago - ...Intelligence Arena Intelligence is the open platform for evaluating how AI models perform in the real world.... .... We’re a team of researchers, engineers, academics, and builders from places... ...preference dimensions Develop new metrics, methodologies, and evaluation protocols...PlatformPermanent employmentWork at office
$207k - $275k
...Essential Cloud for AI™. Built for... ...CoreWeave delivers a platform of technology, tools... ...You’ll Do The Data Engineering Team builds and... ...Role We’re seeking a Staff Data Engineer to... ...layers, enterprise metrics, and reusable data... ...platform strategy. Evaluate and standardize...PlatformPermanent employmentTemporary workCasual workWork at officeFlexible hours$240k - $295k
...use advanced physics and AI to model catastrophic... ...product is a scalable risk engine, our Stand World Model... ...it. The result is a platform that can underwrite what... ...or wiring up the right metric when one doesn't exist... ...designed to manage agents and evaluations at scale. This includes...PlatformFull timeContract workTemporary workH1bWork at officeVisa sponsorshipWork visaFlexible hours$134k - $184.8k
Secure Every Identity, from AI to HumanIdentity is the... ...) organization is the engine that powers Okta's... ...do their best work.The Staff Security Engineer OpportunityWe... ..., and ensure our platforms—from AWS and to... ..., develop and maintain metrics, reporting, and executive...PlatformLocal areaWorldwideFlexible hoursShift work- ...P-1504 The Applied AI team at Databricks sits at the forefront... ...are seeking multiple GenAI Engineers from junior levels to more... ...products, and strengthening our platform architecture to enable... ...hyperparameter tuning, and model evaluation, enabling rapid experimentation...PlatformFull timeWork at office
Databricks
San Francisco, CA14 hours ago $168.91k - $253.37k
Afresh, the AI platform for grocery, began by tackling the most... ...you feed them is not.As a Staff AI Platform Engineer, you build the AI and data... ...built on top of it, and the evaluation and serving infrastructure... ...observability, plus the metrics (faithfulness, accuracy, hallucination...PlatformFull timeLive inWork at officeLocal areaRemote workWork from homeHome officeFlexible hours3 days per week$251k - $310k
...applied to a range of vehicle platforms and product use cases. The... ...diverse set of sensors, enabling engineers like you to (1) develop... ...issues or regressions, develop AI-aided analysis and debugging... ...tooling. Develop and maintain metrics for ADV relevant issues,...PlatformFull timeTemporary workRemote workWaymo
San Francisco, CA14 hours ago- ...and running the world’s best data and AI infrastructure platform so our customers can use deep data... ...to improve their business. Founded by engineers — and customer obsessed — we leap at... ...production at enterprise scale, including evaluation, cost and performance tuning, and the...PlatformWorldwide
$175k - $270k
...payments and financial platform for global businesses.... ...done” end-to-end. You use AI to work smarter and... ...partners closely with engineering, IT, and other stakeholders... ...blocker.Your roleAs a Staff Corporate Security Engineer... ..., develop, test, and evaluate new corporate security...PlatformTemporary workLocal areaWorldwide$141.7k - $250.8k
...San Francisco Bay Area, CA As a Sr. Staff Technical Solutions Engineer and tech subject matter expert,... ...features using tools like Spark UI metrics, Mosaic AI Model Service, DAGs, and event logs... ...the Databricks Data Intelligence Platform to unify and democratize data, analytics...PlatformLocal areaWorldwideNight shift$220k - $280k
...About the Role Together AI is building the best... ...applications. Our Voice AI platform powers production-grade, real... ...reliability. We're looking for a Staff ML Engineer to drive the model serving... ...infrastructure. Build quality evaluation frameworks that guide model...PlatformFull timeTogether Ai
San Francisco, CA14 hours ago$200k - $300k
...leading provider of cloud-based AI solutions to understand,... ...the future of AI! Staff Machine Learning Engineer In order to execute our... ...that can be shared across platforms Contribute meaningfully... ...for use at scale Conduct metric-driven research experiments...PlatformFull timeHive
San Francisco, CA14 hours ago$251k - $310k
...to a range of vehicle platforms and product use cases.... ...sets and high quality metrics cannot scale to Waymo's... ...you will report to an Engineering Director. You will... ...production-grade agentic AI systems to solve complex, open-ended evaluation challenges. Develop...PlatformOdd jobFull timeRemote workWaymo
San Francisco, CA2 days ago$137.1k - $201.6k
...optimization, and systems engineering to the core decisions... ...We’re looking for a Staff Machine Learning... ...Science, Engineering, and Platform teams to improve delivery... ...’s cutting-edge AI vision for logistics:... ...our recruiting team by evaluating job related qualifications...PlatformHourly payWork at officeLocal areaRemote workFlexible hoursDoordash
San Francisco, CA14 hours ago$215k - $322k
...Join GoFundMe as our next Staff Machine Learning Engineer (Pricing) . In this role,... ...from problem framing and metric definition (e.g., donation... ...engineering, training, and evaluation. Build low-latency real... ...diverse set of tools and platforms, including Python, AWS, Databricks...PlatformFull timeTemporary workWork at officeFlexible hoursGofundme
San Francisco, CA14 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Engineer - AI Evaluation & Metrics Platform. Be the first to apply!
Related searches
- assistant engineering manager San Francisco, CA
- assistant mechanical engineer San Francisco, CA
- assistant engineer San Francisco, CA
- staff engineer San Francisco, CA
- staff data engineer San Francisco, CA
- software engineer staff San Francisco, CA
- assistant electrical engineer San Francisco, CA
- staff design engineer San Francisco, CA
- senior staff engineer San Francisco, CA
- senior staff systems engineer San Francisco, CA