Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Director, Research - AI Evals

$258k - $348k

Figma

Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma's platform helps teams bring ideas to life-whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world. If you're excited to shape the future of design and collaboration, join us!

The Figma Research team is hiring an Director, Research - AI Evals to own how we measure the quality of Figma's AI-powered experiences. As Figma ships more AI capabilities across our products, the question "is this really good?" has never mattered more - and answering it rigorously is what this role exists to do. You'll define what "good" means for our AI features, build the frameworks and quality bars to measure it, and turn that into trusted signal that product teams rely on to decide what to ship.

The ideal candidate brings deep, hands-on experience evaluating AI/LLM-powered products - blending human evaluation with automated, model-based approaches - along with the product instinct and communication skills to make evaluation genuinely useful. Partnering with Product, Design, Engineering, and Data Science, you'll sit upstream of nearly every AI shipping decision at Figma and directly shape the quality of features used by millions of people.

This is a full time role that can be held from one of our US hubs or remotely in the United States.

What you'll do at Figma:
  • Own AI evaluation methods and operations for Figma's AI-powered experiences - define quality dimensions, design how we measure them, and turn results into decision-ready signal
  • Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars, combining human evaluation with automated/model-based approaches (e.g., LLM-as-judge) where appropriate
  • Partner with engineering to stand up repeatable, reproducible evaluation pipelines and regression testing, so evaluation is a routine part of how AI features are built and shipped
  • Produce clear readouts and dashboards that let stakeholders confidently make go/no-go and prioritization decisions
  • Socialize a shared definition of quality so evaluation standards are adopted across teams rather than re-invented - and advocate for evaluation as a strategic partner in the product process
  • Manage a small team to execute our AI evals in partnership with contractors, internal staff, and/or LLMs
We'd love to hear from you if you have:
  • 10+ years of experience in product, research, applied research, or a closely related field, including 2+ years of management experience
  • Direct, hands-on experience owning the evaluation of AI/LLM-powered products
  • Expertise designing and running AI evaluation - human evaluation programs, rubric and benchmark/golden-dataset construction, inter-rater reliability - and sound judgment about when and how to apply automated/model-based approaches (e.g., LLM-as-judge), including their limitations
  • Strength across both qualitative and quantitative methods, comfort with data and metrics, and the ability to reason about model behavior
  • Demonstrated success in identifying the riskiest assumptions behind an ambiguous quality question, prioritizing them, and designing right-sized evaluation to build confidence
  • A proven track record of gaining buy-in from executive and cross-disciplinary stakeholders - transcending methodology to articulate a larger user story and the "so what" to inspire action
While it's not required, it's an added plus if you also have:
  • Experience building or co-building automated evaluation pipelines and regression testing in partnership with engineering, or familiarity with eval tooling (e.g., Braintrust, LangSmith, DeepEval, or equivalents)
  • Experience standing up a new function, practice, or discipline from scratch
  • 2+ years in product design, user-centric product management, data science, product development, and/or front-end engineering
  • A familiarity and depth of experience using Figma's products
At Figma, one of our values is Grow as you go. We believe in hiring smart, curious people who are excited to learn and develop their skills. If you're excited about this role but your past experience doesn't align perfectly with the points outlined in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.

Pay Transparency Disclosure

Job level and actual compensation will be decided based on factors including, but not limited to, individual qualifications objectively assessed during the interview process (including skills and prior relevant experience, potential impact, and scope of role), market demands, and specific work location.


Figma offers equity to employees, as well as a competitive package of additional benefits, including health, dental, and vision coverage; retirement benefits with company contributions; parental leave and reproductive or family planning support; mental health and wellness benefits; and paid time off. Figma provides paid sick leave, holidays, and other leave benefits in compliance with applicable federal, state, and local laws, including the requirements of the Washington Minimum Wage Act and related regulations. Exempt employees are eligible for employer-provided paid flexible PTO in addition to flexible paid sick leave. PTO is subject to manager approval. Additional benefits may include company recharge days, cell phone and home internet reimbursements, and a number of lifestyle spending accounts. Figma also offers sales incentive compensation for most sales roles and an annual bonus plan for eligible non-sales roles. All compensation and benefits are subject to applicable plan terms and may be modified by Figma at any time, consistent with applicable law.

Annual Base Salary Range:

$258,000-$348,000 USD

At Figma we celebrate and support our differences. We know employing a team rich in diverse thoughts, experiences, and opinions allows our employees, our product and our community to flourish. Figma is an equal opportunity workplace - we are dedicated to equal employment opportunities regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity/expression, veteran status , or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

We will work to ensure individuals with disabilities are provided reasonable accommodation to apply for a role, participate in the interview process, perform essential job functions, and receive other benefits and privileges of employment. If you require accommodation, please reach out to View email address on click.appcast.io. These modifications enable an individual with a disability to have an equal opportunity not only to get a job, but successfully perform their job tasks to the same extent as people without disabilities.


Examples of accommodations include but are not limited to:
  • Holding interviews in an accessible location
  • Enabling closed captioning on video conferencing
  • Ensuring all written communication be compatible with screen readers
  • Changing the mode or format of interviews
To ensure the integrity of our hiring process and facilitate a more personal connection, we require all candidates keep their cameras on during video interviews. Additionally, if hired you will be required to attend in person onboarding.

By applying for this job, the candidate acknowledges and agrees that any personal data contained in their application or supporting materials will be processed in accordance with Figma's Candidate Privacy Notice.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Director, Research - AI Evals in San Francisco, CA vacancy
  • $250k - $325k

     ..., 08/31/2026 - 04:43 EPAM's new Frontier AI business unit partners directly with leading...  ...AI organizations, translating their research and post-training objectives into technically...  ..., and training data. The Senior Director, Frontier AI Research is EPAM's first dedicated... 
    Suggested
    Temporary work
    H1b
    Remote work
    Flexible hours

    EPAM Systems

    San Francisco, CA
    2 days ago
  • $175k - $250k

     ...Research Engineer About Scorecard We’re a small, nimble team backed by top-tier investors...  ...engineers to help shape the future of AI development. This role is part of the "...  ...knowledge, quirks, and memory. Design and run evals . We need to validate that our approach... 
    Suggested
    Work at office

    Kindredventures

    San Francisco, CA
    5 days ago
  •  ...Ando is a messaging platform where AI agents take on work alongside their human teammates...  ...about as well. If you want to have your research come into contact with reality, Ando is the...  ..., and/or failure analysis. You've built evals you trusted enough to make decisions with.... 
    Suggested
    Work from home

    Ando

    San Francisco, CA
    2 days ago
  •  ...targets that other methods cannot reach. AI is reinventing life sciences the same way...  ...About the role We are seeking an AI Research Engineer to help design, train, evaluate,...  ...depends. Build, run and continuously improve evals and data sources, continually improving... 
    Suggested
    Shift work

    Chai Discovery, Inc

    San Francisco, CA
    2 days ago
  •  ...We are making AI that can build megaprojects: power plants, factories, data centers, and...  ...directly. What you'll do The research that matters most to us comes straight out...  ...designing the benchmarks, environments, and evals that tell us whether it's working. You'll... 
    Suggested

    Orin Labs

    San Francisco, CA
    5 days ago
  • $250k - $290k

     ...Research Engineer - Benchmarks Every week, someone asks which data provider is actually best...  ...for which job. This isn't our internal evals role. You're measuring the market, in public...  ...the easiest way to turn the web into data AI agents can use. One API call converts any... 
    Full time
    Temporary work
    For contractors
    Remote work
    Work from home
    Visa sponsorship
    Flexible hours
    3 days per week

    Firecrawl

    San Francisco, CA
    3 days ago
  • $250k

    Our client, an AI-driven Healthcare company, are hiring a Head of AI Research to join their team in San Francisco. The successful candidate will work on building high-impact, complex AI systems across clinical intelligence, outcome modelling and advanced learning frameworks... 
    Full time

    Alldus International Consulting Ltd

    San Francisco, CA
    more than 2 months ago
  • $130k - $190k

     ...poster from Two Point Consulting Vice President of Recruitment at Two Point Consulting Responsibilities Legal/professional services AI tools Creating AI automated workflows Working with IT and knowledge management teams Vetting new AI tools Providing trainings and pilots... 
    Full time

    Two Point Consulting

    San Francisco, CA
    5 days ago
  •  ...Involves gathering, analyzing, and interpreting a wide variety of research data. Designs and conducts research including selecting data...  ...streamlined drug development, the market for advanced analytics and AI in clinical research is expanding exponentially. What You'll... 
    Traineeship
    Worldwide

    University of California , San Francisco

    San Francisco, CA
    2 days ago
  •  ...create reliable, interpretable, and steerable AI systems. We want AI to be safe and...  ...is a quickly growing group of committed researchers, engineers, policy experts, and business...  ...methodology, and help eval authors bring new evals up to the bar for production Investigate... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    4 days ago
  • $65 - $75 per unit

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    2 days ago
  •  ...chain management, and scalable manufacturing. As robotics, Physical AI, AI infrastructure, and advanced manufacturing continue to...  ...frontier technology initiatives. Track market and technology trends, research startups and technology companies, prepare management briefings,... 

    Minth North America

    San Francisco, CA
    a month ago
  •  ...Design, train, ship, iterate on, and innovate on the AI brains behind The Path’s AI Therapist. Combine research, data science, and engineering to create models,...  ...rather than chasing generic benchmarks. Can look at evals, transcripts, and metrics and quickly form grounded... 

    The Path

    San Francisco, CA
    2 days ago
  • $300k

     ...Founding Research Engineer - AI & Defence San Francisco | Onsite I’m working with a highly ambitious defense AI startup building technology to detect and conduct a novel class of AI-powered cyber operation. This is not conventional cybersecurity, vulnerability research... 

    Society of Defense Financial Management

    San Francisco, CA
    5 days ago
  •  ...About the Role Factory is seeking innovative Research Engineers to design and integrate advanced AI and ML capabilities that revolutionize productivity and accelerate innovation within software organizations.What you will do and achieveDesign, develop, and deploy AI-driven... 
    Work at office

    The San Francisco AI Factory

    San Francisco, CA
    3 days ago
  •  ...Get AI-powered advice on this job and more exclusive features. About Resolve AI At Resolve, we’re building Agentic AI that empowers...  ...What You’ll Do Develop AI‑powered workflows end‑to‑end, balancing research and engineering to create production‑ready AI models Build and... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours

    Resolve AI

    San Francisco, CA
    5 days ago
  • $100k - $300k

     ...Company Overview At Skild AI, we are building the world's first general purpose robotic intelligence that is robust and adapts to...  ...contribute to our innovative projects. Position Overview We are hiring Research Engineers to develop scalable robotic systems aimed at achieving... 
    Full time

    Skild AI

    San Francisco, CA
    5 days ago
  •  ...and post-training of LLMs Ability to come up with and evaluate research ideas Experience working with large distributed systems...  ...engineer, you’ll work on training, evaluating, and serving large AI models and new inference-time compute techniques, build internet... 

    Magic Inc

    San Francisco, CA
    5 days ago
  •  ...Proximal is building the research systems needed to identify what models can’t yet do, build the tasks required to teach them, measure whether...  ...the data that frontier models need. We work with frontier AI labs to provide the data and evaluations behind their most capable... 

    Proximal LLC

    San Francisco, CA
    5 days ago
  • $180k - $250k

     ...Open role Research Engineer San Francisco (On-site), Full-time About Us Constellation is creating the AI-human translation layer that ensures humanity evolves alongside our technology. Our mission is to leverage AI towards addressing deep and meaningful problems... 
    Full time
    Work at office
    Relocation package

    Breakout Ventures

    San Francisco, CA
    5 days ago
  • $180k - $280k

     ...About SuperAnnotate SuperAnnotate helps the world’s leading AI teams build responsible, next-generation models powered by high-...  ...consecutive years, including 2025. The Impact You’ll Make Our research team is expanding to keep pace with a wave of frontier‑facing... 
    Full time

    SuperAnnotate AI

    San Francisco, CA
    5 days ago
  •  ...one of the great unsolved problems in science, and one we think AI finally makes tractable. We believe that understanding this transition...  ..., and bench scientists. We hold ourselves to the rigor of a research institute, but we ship like an engineering firm. Global team,... 
    Visa sponsorship
    Shift work

    Dayhoff Labs

    San Francisco, CA
    2 days ago
  • $250k - $290k

     ...work with teams across the United States to help them hire. Research Engineer Location San Francisco, CA / Bay Area, CA...  ...Backed Technology Company Industry Artificial Intelligence, AI Infrastructure, Developer Tools, Machine Learning, LLMs, Web Data... 
    H1b
    Remote work

    Recruiting from Scratch

    San Francisco, CA
    3 days ago
  •  ...By applying to this role, you will be considered for Research Engineer roles across all teams at OpenAI. About the Role As a Research Engineer here, you will be responsible for building AI systems that can perform previously impossible tasks or achieve unprecedented... 

    OpenAI

    San Francisco, CA
    2 days ago
  •  ...foundation models. The company is operating in stealth mode by a highly experienced founding team, who are well known in the AI community for seminal research accomplishments at top AI labs, have run AI departments at top AI x Biology organizations, have exited a past company... 
    Full time

    Menlo Ventures

    San Francisco, CA
    2 days ago
  • $9.7k - $19k

     ...About The Center for AI Safety (CAIS) The Center for AI Safety (CAIS) is a leading research and advocacy organization focused on mitigating societal-scale risks from AI. Some of our past achievements include: releasing the most widely used measure of AI capabilities... 
    Full time
    Summer work
    Internship
    Local area
    Flexible hours

    Center for AI Safety

    San Francisco, CA
    4 days ago
  •  ...Join AfterQuery AfterQuery is an applied research lab curating data solutions for foundation model development. We serve every frontier AI lab with the mission of delivering the best data to power the best models. In doing so, we can make expertise that once took... 
    Local area

    AfterQuery

    San Francisco, CA
    3 days ago
  •  ...software vulnerabilities. We are training and scaling security AI agents to discover zero-days vulnerabilities across large customer...  .... About this role We’re seeking an experienced Research Engineer to join our effort in building and training AI agents for... 
    Full time
    Work at office

    depthfirst Inc.

    San Francisco, CA
    2 days ago
  •  ...members are remote, but this role is currently on-site) Industry: AI infrastructure / Reinforcement Learning (RL) training data &...  ...craftsmanship. The Opportunity Our partner is hiring a Research Engineer to help scale the quality assurance (QA) systems behind... 
    Remote work

    talentpluto

    San Francisco, CA
    2 days ago
  • $120k - $200k

     ...We are actively seeking a Research Engineer specializing in Machine Learning and AI to play a pivotal role in pioneering advanced solutions. In this role, you will lead end-to-end research projects and contribute technical expertise to build scalable systems, all within... 
    Casual work
    Work at office

    Erth.AI Inc.

    San Francisco, CA
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Director, Research - AI Evals. Be the first to apply!