Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Director, Research - AI Evals

$258k - $348k

Figma

Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma's platform helps teams bring ideas to life-whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world. If you're excited to shape the future of design and collaboration, join us!

The Figma Research team is hiring an Director, Research - AI Evals to own how we measure the quality of Figma's AI-powered experiences. As Figma ships more AI capabilities across our products, the question "is this really good?" has never mattered more - and answering it rigorously is what this role exists to do. You'll define what "good" means for our AI features, build the frameworks and quality bars to measure it, and turn that into trusted signal that product teams rely on to decide what to ship.

The ideal candidate brings deep, hands-on experience evaluating AI/LLM-powered products - blending human evaluation with automated, model-based approaches - along with the product instinct and communication skills to make evaluation genuinely useful. Partnering with Product, Design, Engineering, and Data Science, you'll sit upstream of nearly every AI shipping decision at Figma and directly shape the quality of features used by millions of people.

This is a full time role that can be held from one of our US hubs or remotely in the United States.

What you'll do at Figma:
  • Own AI evaluation methods and operations for Figma's AI-powered experiences - define quality dimensions, design how we measure them, and turn results into decision-ready signal
  • Build and maintain evaluation frameworks, rubrics, golden datasets, and quality bars, combining human evaluation with automated/model-based approaches (e.g., LLM-as-judge) where appropriate
  • Partner with engineering to stand up repeatable, reproducible evaluation pipelines and regression testing, so evaluation is a routine part of how AI features are built and shipped
  • Produce clear readouts and dashboards that let stakeholders confidently make go/no-go and prioritization decisions
  • Socialize a shared definition of quality so evaluation standards are adopted across teams rather than re-invented - and advocate for evaluation as a strategic partner in the product process
  • Manage a small team to execute our AI evals in partnership with contractors, internal staff, and/or LLMs
We'd love to hear from you if you have:
  • 10+ years of experience in product, research, applied research, or a closely related field, including 2+ years of management experience
  • Direct, hands-on experience owning the evaluation of AI/LLM-powered products
  • Expertise designing and running AI evaluation - human evaluation programs, rubric and benchmark/golden-dataset construction, inter-rater reliability - and sound judgment about when and how to apply automated/model-based approaches (e.g., LLM-as-judge), including their limitations
  • Strength across both qualitative and quantitative methods, comfort with data and metrics, and the ability to reason about model behavior
  • Demonstrated success in identifying the riskiest assumptions behind an ambiguous quality question, prioritizing them, and designing right-sized evaluation to build confidence
  • A proven track record of gaining buy-in from executive and cross-disciplinary stakeholders - transcending methodology to articulate a larger user story and the "so what" to inspire action
While it's not required, it's an added plus if you also have:
  • Experience building or co-building automated evaluation pipelines and regression testing in partnership with engineering, or familiarity with eval tooling (e.g., Braintrust, LangSmith, DeepEval, or equivalents)
  • Experience standing up a new function, practice, or discipline from scratch
  • 2+ years in product design, user-centric product management, data science, product development, and/or front-end engineering
  • A familiarity and depth of experience using Figma's products
At Figma, one of our values is Grow as you go. We believe in hiring smart, curious people who are excited to learn and develop their skills. If you're excited about this role but your past experience doesn't align perfectly with the points outlined in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.

Pay Transparency Disclosure

Job level and actual compensation will be decided based on factors including, but not limited to, individual qualifications objectively assessed during the interview process (including skills and prior relevant experience, potential impact, and scope of role), market demands, and specific work location.


Figma offers equity to employees, as well as a competitive package of additional benefits, including health, dental, and vision coverage; retirement benefits with company contributions; parental leave and reproductive or family planning support; mental health and wellness benefits; and paid time off. Figma provides paid sick leave, holidays, and other leave benefits in compliance with applicable federal, state, and local laws, including the requirements of the Washington Minimum Wage Act and related regulations. Exempt employees are eligible for employer-provided paid flexible PTO in addition to flexible paid sick leave. PTO is subject to manager approval. Additional benefits may include company recharge days, cell phone and home internet reimbursements, and a number of lifestyle spending accounts. Figma also offers sales incentive compensation for most sales roles and an annual bonus plan for eligible non-sales roles. All compensation and benefits are subject to applicable plan terms and may be modified by Figma at any time, consistent with applicable law.

Annual Base Salary Range:

$258,000-$348,000 USD

At Figma we celebrate and support our differences. We know employing a team rich in diverse thoughts, experiences, and opinions allows our employees, our product and our community to flourish. Figma is an equal opportunity workplace - we are dedicated to equal employment opportunities regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity/expression, veteran status , or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

We will work to ensure individuals with disabilities are provided reasonable accommodation to apply for a role, participate in the interview process, perform essential job functions, and receive other benefits and privileges of employment. If you require accommodation, please reach out to View email address on click.appcast.io. These modifications enable an individual with a disability to have an equal opportunity not only to get a job, but successfully perform their job tasks to the same extent as people without disabilities.


Examples of accommodations include but are not limited to:
  • Holding interviews in an accessible location
  • Enabling closed captioning on video conferencing
  • Ensuring all written communication be compatible with screen readers
  • Changing the mode or format of interviews
To ensure the integrity of our hiring process and facilitate a more personal connection, we require all candidates keep their cameras on during video interviews. Additionally, if hired you will be required to attend in person onboarding.

By applying for this job, the candidate acknowledges and agrees that any personal data contained in their application or supporting materials will be processed in accordance with Figma's Candidate Privacy Notice.

Application limits: To help candidates focus on the opportunities most relevant to their skills and experience, Figma limits the number and frequency of applications. These limits apply to all roles:
  • Candidates may submit up to 5 applications during any rolling 30-day period.
  • Candidates may not reapply to the same role while an application is active or within 90 days of receiving a rejection.

Once the applicable days has passed, candidates are welcome to apply again.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Director, Research - AI Evals in San Francisco, CA vacancy
  • $80k - $230k

     ...About HUD About the role Were looking for a Research Engineer for our Robotics team to develop the datasets and evals that make robotics data useful for training and evaluating embodied AI systems. Youll translate open-ended research needs into data specifications... 
    Suggested
    Full time
    Work at office
    Remote work
    Relocation
    Visa sponsorship

    HUD

    San Francisco, CA
    10 days ago
  •  ...energy programs. Our customers include Anduril, Shield AI, Hermeus, Albedo, Shinkei, and Pratt Miller Motorsports,...  ...engineers who rely on them. The Role: As a Staff Research Engineer supporting agent evals & post-training, you'll define how Nominal measures its... 
    Suggested
    Permanent employment

    Nominal

    San Francisco, CA
    4 hours ago
  •  ...EPAMs new Frontier AI business unit partners directly with leading AI labs and advanced AI organizations translating their research and post-training objectives into technically rigorous...  ...environments and training data. The Senior Director Frontier AI Research is EPAMs first... 
    Suggested
    Full time

    EPAM Systems

    San Francisco, CA
    15 days ago
  • $150k - $250k

    About Distyl AI Distyl is an applied AI technology company partnering with the world’s...  ...goods, and global social organizations.We research and deploy technologies that power AI-native...  ..., reward modeling, synthetic data, evals, or related post-training techniquesStrong... 
    Suggested
    Work at office
    3 days per week

    Distyl AI

    San Francisco, CA
    3 days ago
  •  ...Ando Research Team MemberAndo is a messaging platform where AI agents take on work alongside their human teammates. We're rebuilding Slack from the ground up around...  ..., and/or failure analysis. You've built evals you trusted enough to make decisions with.Evidence... 
    Suggested
    Work from home

    ASARI S.A de CV

    San Francisco, CA
    5 days ago
  •  ...We are making AI that can build megaprojects: power plants, factories, data centers, and...  ...directly. What you'll do The research that matters most to us comes straight out...  ...designing the benchmarks, environments, and evals that tell us whether it's working. You'll... 

    Orin Labs

    San Francisco, CA
    4 days ago
  •  ...scientists and engineers move faster and solve core R&D problems. What you’ll do: Translate cutting-edge AI research into production‑grade systems. Build evals and benchmarks to rigorously measure model and agent performance across a wide variety of scientific tasks... 

    Altara

    San Francisco, CA
    1 day ago
  • $250k - $290k

     ...Research Engineer - Benchmarks Every week someone asks which data provider is actually best...  ...for which job. This isnt our internal evals role. Youre measuring the market in public...  ...the easiest way to turn the web into data AI agents can use. One API call converts any... 
    Full time
    Temporary work
    For contractors
    Remote work
    Work from home
    Visa sponsorship
    Flexible hours
    3 days per week

    Firecrawl

    San Francisco, CA
    15 days ago
  • $135k - $265k

     ...team empowers organizations to build deeper relationships with customers through innovative strategies, advanced analytics, Generative AI, transformative technologies, and creative design. We can enhance customer experiences and drive sustained growth and customer value... 
    Local area
    Visa sponsorship

    Deloitte

    San Francisco, CA
    4 days ago
  • $270k - $340k

    Principal AI Research Scientist, Research Director - AI ScalingP-1227About Databricks AIAt Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development, by building and running the... 
    Local area
    Worldwide

    DataBricks

    San Francisco, CA
    3 days ago
  • $231k - $340k

     ...services operate. By combining frontier agentic AI, an enterprise-grade platform, and deep...  ...at legal work. We are looking for a research engineer who can help scale that loop: defining...  ...those findings into training data, evals, or harness changes. Work with Harvey researchers... 

    Harvey, Inc.

    San Francisco, CA
    4 days ago
  • $250k

    Our client, an AI-driven Healthcare company, are hiring a Head of AI Research to join their team in San Francisco. The successful candidate will work on building high-impact, complex AI systems across clinical intelligence, outcome modelling and advanced learning frameworks... 
    Full time

    Alldus International Consulting Ltd

    San Francisco, CA
    more than 2 months ago
  •  ...Involves gathering, analyzing, and interpreting a wide variety of research data. Designs and conducts research including selecting data...  ...streamlined drug development, the market for advanced analytics and AI in clinical research is expanding exponentially. What You'll... 
    Traineeship
    Worldwide

    University of California , San Francisco

    San Francisco, CA
    1 day ago
  •  ...Research Engineer Post-Training Data Location: San Fran Employment Type: Full-time...  ...Generation Research Environments Frontier AI Profile: Researcher-engineer with strong...  ...Experience with post-training data RL environments evals model behavior or interpretability... 
    Full time

    NxT Level

    San Francisco, CA
    6 days ago
  • $200k - $300k

     ...product ownership. You’ll be in the agent code, the prompts, the evals, the integrations, and the product surfaces, talking to users and...  ...senior to staff level role. We’re looking for someone who has shipped AI products people use every day, at a frontier AI lab, an AI... 
    Full time
    Remote work

    Pally

    San Francisco, CA
    3 days ago
  • $138.1k - $196.5k

    The application window is expected to close on: 10/13/2026Meet the TeamCisco Foundation AI is a frontier AI research team advancing foundational model development, evaluation, and engineering innovation. Based in our San Francisco hub, our team operates in a collaborative... 
    Full time
    Temporary work
    Local area
    Monday to Friday
    Flexible hours

    CISCO Systems

    San Francisco, CA
    5 days ago
  • $124k - $155k

    About the TeamAI Research Lab is one of DoorDash’s frontier innovation hubs, focused on building the foundation for an AI-native future across our three core audiences: consumers, merchants, and dashers. As AI capabilities accelerate, the AI Research Lab operates at the... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    5 days ago
  • $193.3k - $261.5k

     ...the Quick Science team in building the next generation enterprise AI assistant.Quick is AWS’s enterprise generative AI assistant that...  ...enterprise systems. As a key member of this team, you will lead research and development efforts in generative AI and Agentic AI to... 
    Internship
    Local area
    Flexible hours

    AmazonWebServices

    San Francisco, CA
    6 days ago
  • $110.7k - $379.2k

    Position Summary Research Engineer — Post-Training & Small Language Models (SLMs), Healthcare AI Three hundred fifty million Americans rely on a healthcare system whose decision-making has become slow, costly, and adversarial — care delayed by prior authorization... 
    Local area
    Visa sponsorship

    Deloitte

    San Francisco, CA
    4 days ago
  • $197.3k - $313.7k

     ...CategorySoftware EngineeringJob DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive customer success together....  ...where your engineering skills directly enable world-class research and products used by millions?At Salesforce, we are driving the... 
    Full time

    Salesforce

    San Francisco, CA
    7 days ago
  • $148.1k - $282.1k

    The Research and AI team builds foundational generative AI models and applications for Adobe products, enabling customers to ideate, develop, and scale content in new ways.We're looking for a Principal PM to define and drive our data roadmap, including sourcing and processing... 
    Full time
    Temporary work
    Local area
    Worldwide

    Adobe Systems

    San Francisco, CA
    3 days ago
  • About usSierra is the leading platform for customer-facing AI agents, working with many of the world's biggest brands — including The...  ...product and design teams for Google Workspace.Sierra’s Applied Research team advances the quality, speed, and cost efficiency of the... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    5 days ago
  •  ...About the Role Factory is seeking innovative Research Engineers to design and integrate advanced AI and ML capabilities that revolutionize productivity and accelerate innovation within software organizations.What you will do and achieveDesign, develop, and deploy AI-driven... 
    Work at office

    The San Francisco AI Factory

    San Francisco, CA
    3 days ago
  • $200k - $225k

     ...internal teams and customers to leverage advanced analyticsDocument research and implementation decisions for reproducibility and knowledge...  ...problems your wayCutting-edge technology: Work with advanced AI/ML algorithms, composite AI solutions, private NVIDIA DGX clusters... 

    Alembic

    San Francisco, CA
    3 days ago
  • $130k - $190k

     ...poster from Two Point Consulting Vice President of Recruitment at Two Point Consulting Responsibilities Legal/professional services AI tools Creating AI automated workflows Working with IT and knowledge management teams Vetting new AI tools Providing trainings and... 
    Full time

    Two Point Consulting

    San Francisco, CA
    3 days ago
  • $207k - $300k

     ...safety engineering to develop new capabilities based on adversarial research.Contribute to increasing Gemini’s capabilities for cyber defense...  ...in peer-reviewed conferences or journals.Passion for AI technology and all of its possibilities.At Google, research-focused... 

    Google

    San Francisco, CA
    3 days ago
  • $65 - $75 per unit

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  •  ...software vulnerabilities. We are training and scaling security AI agents to discover zero-days vulnerabilities across large customer...  .... About this role We’re seeking an experienced Research Engineer to join our effort in building and training AI agents for... 
    Full time
    Work at office

    depthfirst Inc.

    San Francisco, CA
    1 day ago
  •  ...Goliath is partnered with an innovative AI startup building the data layer and reinforcement learning environments for medical AI...  ...workflows. They are currently in need of an ambitious founding Research Engineer ~ Ability to construct healthcare specific RL environments... 

    Goliath Partners LP

    San Francisco, CA
    2 days ago
  • $140k - $250k

     ...Location: Remote (United States) Work Model: Remote Industry: AI training data infrastructure Compensation: $140K-$250K base,...  ...This is the company's top hiring priority and a genuinely hard research problem. Because data flows through a decentralized marketplace,... 
    Full time
    Work at office
    Remote work

    TalentPluto, Inc.

    San Francisco, CA
    16 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Director, Research - AI Evals. Be the first to apply!