Remote AI Evaluation & Benchmark Engineer — Equity & Growth

$180k - $230k

Themeframer

Remote job

Themeframer, a San Francisco–based AI & ML company, is hiring for a full-time role in AI & Machine Learning Generative Systems. You will build automated sandbox testing frameworks and ensure models meet benchmarks, with a compensation range of $180,000–$230,000 USD plus equity. You will work on 4+ years of Python, CI/CD, and evaluation tooling, with remote-first setup and a learning budget. The role emphasizes testing across large test suites and collaboration with research teams. #J-18808-Ljbffr Themeframer

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Remote AI Evaluation & Benchmark Engineer — Equity & Growth in San Francisco, CA vacancy

Evaluation and ML Systems Engineer, AI Safety and Security Engineering
$152k - $241.5k
...foundational to American AI leadership and... ...AI Safety & Security Engineering team builds and evaluates AI-powered tooling that... ...program grows. Your benchmarks will define what... ...also be eligible for equity and benefits.Applications... ...Clara; US, NC, Remote; US, NY, Remote; US,...
Remote work
Employment Equity
Full time
Nvidia
Santa Clara, CA
3 days ago
AI Evaluation & Benchmark Engineer
$180k - $230k
...AI & Machine Learning Generative Systems... ...· 0.15% – 0.3% Equity Ensure Theme Framer... ...the highest benchmarked accuracy in the industry... ..., and LLM evaluation tooling. ~ Strong... ...Equity Ownership Remote-First Setup Stipend... ...Annual Company Engineering Retreats $2,000...
Remote work
Employment Equity
Full time
Flexible hours
Themeframer
San Francisco, CA
1 day ago
Senior Compiler Engineer - AI
$184k - $287.5k
...unlimited potential of AI to define the... ...an AI Compiler Engineer with deep... ...workloads and benchmark suites.Lead experimentation, evaluation, and deployment... ...unprecedented growth, our exclusive... ...be eligible for equity and benefits.Applications... ...; US, TX, Remote; US, CA, Remote...
Remote work
Employment Equity
Full time
Nvidia
Austin, TX
5 days ago
Forward Deployed Engineer, Enterprise AI
...started Handshake AI and built the... ...researchers to create evaluations, publish benchmarks, and push the... ...Work together with engineers, scientists, operators... ...Ownership: Equity in a fast-growing... ...wellness stipend Growth: $2,000 learning... ...development Remote & Office:...
Remote work
Employment Equity
Full time
Work at office
Flexible hours
Handshake
San Francisco, CA
1 day ago
Distinguished Engineer, AI Safety and Security Engineering
$320k
...Distinguished Engineer to serve as the... ...leader for our AI Safety & Security... ...partners, and evaluation teams to build... ...support their growth.What We Need To... ...evaluation or benchmarking infrastructure.... ...be eligible for equity and benefits.Applications... ...Clara; US, NC, Remote; US, NY, Remote...
Remote work
Employment Equity
Full time
Nvidia
Santa Clara, CA
2 days ago
Senior Engineering Manager, Agentic & Generative AI Benchmarking and Evaluations
$201.3k - $352.3k
...DescriptionIt all started when engineer Fred Luddy wrote code... ..., ServiceNow is the AI control tower for... ..., Agentic & GenAI Benchmarking and Evaluations to establish and lead... ...300 - $352,300, plus equity (when applicable),... ...personas (flexible, remote, or required in office...
Remote work
Employment Equity
Work experience placement
Work at office
Immediate start
Flexible hours
Shift work
ServiceNow
Santa Clara, CA
1 day ago
Distinguished Engineer, Frontier Agentic AI (Remote)
$246k
...world’s most advanced AI-native platform. We work... ...a Distinguished Engineer to research, define, and... ...prototypes, establish rigorous evaluations, and help turn... ...rigorous evaluations. Build benchmarks, cyber environments,... ...in compensation and equity awards...
Remote work
Employment Equity
Permanent employment
Work experience placement
Work at office
Local area
Remote
7 days ago
Staff Test & Evaluation Engineer - Networked Battlefield Systems
$166k - $220k
...is powered by Lattice OS, an AI-powered operating system... ...months, not years.The Test & Evaluation Engineering Team at Anduril works across... ...conducting test activities in remote, austere, or tactical operational... .... Highly competitive equity grants are included in the majority...
Remote work
Employment Equity
Full time
Work experience placement
Immediate start
Anduril Industries
Costa Mesa, CA
3 days ago
Senior Forward Deployed Engineer (AI Enablement)
$180.2k - $252.3k
...Senior Forward Deployed Engineer (AI Enablement) As... ...time off #LI-Remote Pay... ...eligible to receive equity in the form of restricted... ...country location, benchmarked against similar stage growth companies. Final offer... ...information to evaluate your application for...
Remote work
Employment Equity
Full time
For contractors
Work experience placement
Local area
Home office
Flexible hours
Reddit
Remote
4 days ago
Non-G7 Central Bank Policy Communications Analyst, AI Evaluation
$150 per hour
...forecasting and research data that helps evaluate and improve AI models performing real-world... ...interest rates, foreign exchange, and equities. Review and grade model-generated analyses... ...material non-public information. Work Terms Remote, hourly engagement. Compensation $150...
Remote work
Employment Equity
Hourly pay
Local area
SaidGig
New York, NY
23 hours ago
Distinguished Engineer, AI Security Systems (Remote)
$246k
...world’s most advanced AI-native platform.... ...learning, evaluation, and secure deployment... ...seeking a Distinguished Engineer who has built AI... .... Establish benchmarks, cyber ranges, and... ...are stopped. #LI-Remote #LI-CS1 Benefits... ...compensation and equity awards...
Remote work
Employment Equity
Permanent employment
Work experience placement
Work at office
Local area
Remote
11 hours ago
Remote Lead, Legal AI Evaluation & Benchmarking
SaidGig invites a remote, full-time researcher to advance AI-powered legal reasoning. You will design rigorous evaluation methods for legal AI agents and work with AI researchers to... ...create structured evaluation methods. Equity compensation and performance bonuses may...
Remote job
Employment Equity
Full time
SaidGig
New York, NY
23 hours ago
Solutions Engineer (Application Engineer): End-of-Line Automation
$115k - $155k
...The Solutions Engineering Team is responsible... ..., including:Equity in Formic: Participate... ...for fully remote and hybrid... ...are thoughtfully benchmarked against peer companies... ...at a similar growth stage. Equity... ...@formic.co.AI Use:At Formic,... ...help our teams evaluate candidate responses...
Remote work
Employment Equity
Full time
Temporary work
Work at office
Local area
Immediate start
Flexible hours
Shift work
Formic Technologies
Chicago, IL
5 days ago
Senior Backend Engineer, Compliance Engineering
$190.8k - $267.1k
...The Compliance Engineering team is responsible... ...in a fully-remote environment, and... ...to receive equity in the form of... ...country location, benchmarked against similar stage growth companies. Final... ...intelligence (AI). You will have... ...information to evaluate your application...
Remote work
Employment Equity
For contractors
Work experience placement
Flexible hours
Reddit, Inc.
San Francisco, CA
1 day ago
GPU and Numerical Methods Engineer, Continuum Engine
$100k - $350k
...Numerical Methods Engineer, Continuum Engine... ...elasticity share, the benchmarks that keep it from... ...works with our AI systems daily. What... ...be rewarded with equity in a company we believe... ..., warm sessions, remote accelerators... ...participation in a high-growth startup ~...
Remote work
Employment Equity
Work from home
Nebula
El Segundo, CA
1 day ago
Senior Forward Deployed Engineer (AI)
...MathCo is an AI and analytics consultancy... ...Forward Deployed Engineering is our answer to... ...technology ecosystems. Evaluate technical trade-... ...in distributed, remote-first environments... ...to company growth initiatives while... ...Support diversity, equity, inclusion, and belonging...
Remote work
Employment Equity
Full time
MathCo
Chicago, IL
2 days ago
Sr. Principal Controls Engineer- Data Center Design and Innovation
$146.3k - $306.4k
...and supply chain engineering teams to develop... ...cloud and AI infrastructure growth. Establish... ...controls management, remote operations, and... ...Leadership Evaluate and qualify controls... ..., performance benchmarks, and technology... ...eligible for bonus, equity, and...
Remote work
Employment Equity
Temporary work
Flexible hours
Shift work
Oracle Corporation
United States
2 days ago
Senior Systems Engineer -- Agentic AI
$224k - $356.5k
...unlimited potential of AI to define the next... ...Harness & Runtime Engineering team to build... ...runtimes, inference, evaluation, and large-scale... ...in AI evaluation, benchmarking, experimentation,... ...also be eligible for equity and benefits.... ..., Austin; US, CA, Remote; US, CA, Santa ClaraType...
Remote work
Employment Equity
Full time
Nvidia
Austin, TX
1 day ago
Martech/ Growth Engineer
$1,000 per month
...RoleWe’re looking for a MarTech / Growth Engineer to sit at the intersection of... ..., advertising platforms, and AI-powered workflows.You’ll be a... ...arise that require remote work or adjusted schedules. Our... ...participate in the bonus and equity programs. You will also have...
Remote work
Employment Equity
Temporary work
Work at office
Immediate start
Flexible hours
Creditly Corp
Philadelphia, PA
2 days ago
Senior TPRM Lead- Tooling and Process Engineering (Remote)
$140k - $252k
...world’s most advanced AI-native platform.... ...for vendor risk evaluations. The ideal candidate... ...for ways to engineer a stronger, more... ...fast-paced, high-growth, global technology... ...components. #LI-CS1 #LI-Remote Benefits of... ...compensation and equity awards...
Remote work
Employment Equity
Permanent employment
Work experience placement
Work at office
Local area
Remote
14 days ago
Staff Backend Engineer, User Safety
$217k - $303.9k
...role is completely remote friendly within... ...Staff Backend Engineer , who has interest... ...to receive equity in the form of restricted... ...location, benchmarked against similar stage growth companies. Final... ...intelligence (AI). You will have... ...information to evaluate your application...
Remote work
Employment Equity
Full time
For contractors
Work experience placement
Work at office
Flexible hours
Reddit
Remote
2 days ago
Growth Engineer
$160k - $280k
...advantage. Cresta’s unified AI platform combines... ...As Cresta’s Growth Engineer , you'll own the systems... ...production, including the evaluation criteria, guardrails,... ...you plan for the future Remote work setup budget to... ...includes a Base Salary + Equity. Base Salary Range: $1...
Remote work
Employment Equity
Work at office
Local area
Home office
Flexible hours
Cresta
Remote
23 hours ago
Senior Systems Software Engineer, Workload Systems Engineering
$184k - $287.5k
...systems software engineer to help engineering... ...application configuration, remote execution, workers... ...direct system evaluation: diagnosing causes... ...and reliability benchmarks for representative... ...be eligible for equity and benefits.... ...vacancy. NVIDIA uses AI tools in its recruiting...
Remote work
Employment Equity
Full time
Nvidia
Santa Clara, CA
1 day ago
Research Lead - AI Cyber Testing & Evaluation
$150.1k - $268.5k
...RAND’s Center on AI, Security, and... ...researchers, engineers, and scientists... ...build systems to evaluate how AI models... ...include developing benchmarks for fully... ...Program (CNODP), Remote Interactive Operator... ...Readiness Growth and Enrichment... ...; and internal equity. Successful candidates...
Remote work
Employment Equity
Fixed term contract
Work experience placement
Work from home
RAND
Washington DC
1 day ago
Senior Machine Learning Systems Engineer, ML Platform
$216.7k - $303.4k
...teams such as Growth, Ads, Feeds, and... ...Senior Software Engineer, ML Platform,... ...Leave #LI-Remote Pay Transparency... ...to receive equity in the form of... ...country location, benchmarked against similar... ...intelligence (AI). You will have... ...information to evaluate your...
Remote work
Employment Equity
Full time
For contractors
Work experience placement
Flexible hours
Reddit
Remote
8 days ago
Senior Software Engineer, Growth
...started Handshake AI and built the... ...researchers to create evaluations, publish benchmarks, and push the... ...Work together with engineers, scientists, operators... ...Engineer on the Growth team, you'll... ...Ownership: Equity in a fast-growing... ...development Remote & Office: Internet...
Remote work
Employment Equity
Full time
Work at office
Flexible hours
Handshake
San Francisco, CA
1 day ago
Senior System Software Engineer, SoC Power and Performance - O-RAN Infrastructure
$152k - $241.5k
...unlimited potential of AI to define the next... ...System Software Engineer to develop and... ...measurement tools, benchmarks, and automated test... ...issues.Evaluate system behavior against... ...also be eligible for equity and benefits.Applications... ...Clara; US, CA, Remote; US, CO,...
Remote work
Employment Equity
Full time
Nvidia
Santa Clara, CA
4 days ago
QA Engineer
$70k - $85k
...Four days remote. Wednesdays in Cincinnati. $70,... ...new phase of private-equity-backed growth. Join a small team, learn... ...number of years. Engineering, technical support, implementation... ...thoughtful use of AI-assisted testing... ...role. Candidates are evaluated only on job-related...
Remote work
Employment Equity
Permanent employment
Full time
Work at office
Urrly
Cincinnati, OH
2 days ago
AI Sales Engineer
$140k - $230k
...Primis is partnering with a high-growth AI observability and evaluation company to find an AI Sales Engineer. About the Company As... ...230,000 , plus a competitive equity package ~ Final... ...Location The company is remote-first, with optional offices...
Remote work
Employment Equity
Work from home
Primis
United States
2 days ago
Growth Engineer
...decision systems by combining: AI agents that execute data... ...Role We’re hiring a Founding Growth Engineer to build Zerve’s growth engine... ...technical walkthroughs, and benchmark reports Decision workflow... ...Competitive salary Meaningful equity Remote-first team Opportunity to...
Remote work
Employment Equity
Full time
Ignite Next GmbH
Washington DC
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Remote AI Evaluation & Benchmark Engineer — Equity & Growth. Be the first to apply!