Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Researcher, AI Evaluations

$169k - $250k
Full-time

Figma

Figma is growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma’s platform helps teams bring ideas to life—whether you're brainstorming, creating a prototype, translating designs into code, or iterating with AI. From idea to product, Figma empowers teams to streamline workflows, move faster, and work together in real time from anywhere in the world. If you're excited to shape the future of design and collaboration, join us!

We’re hiring a Researcher, AI Evaluations to join AI Insights within Figma’s Research team. This is a senior individual contributor role with no direct reports, reporting to the AI Insights lead and working directly with AI Research within AI Platform. You’ll strengthen the team’s model, agent system, and evaluation work through hands-on studies, clear rubrics, and reliable measures of quality that reflect user needs, and your findings will guide model and product improvements.

This is a full time role that can be held from one of our US hubs or remotely in the United States.

What you’ll do at Figma:

  • Own evaluation research for priority AI Research workstreams. Define quality criteria, design studies, analyze results, and recommend actions
  • Build rubrics, representative evaluation datasets, benchmarks, and quality thresholds that reflect user needs and expert design judgment
  • Work with AI Research and Modeling Platform engineers to integrate validated methods into repeatable evaluations and regression tests
  • Explain quality, failure patterns, and uncertainty. Recommend actions that guide model selection, model improvements, product priorities, and release decisions
  • Validate automated graders against human ratings, including LLM-as-judge methods that use a large language model to score outputs. Identify bias, gaps, and uncertainty
  • Connect user research with AI Research’s model development work. Share reusable methods, review work, and mentor peers

We’d love to hear from you if you have:

  • 7+ years of experience in research, applied research, or a related field, with a proven track record of independently leading complex evaluation projects and delivering measurable impact
  • Hands-on experience evaluating AI products, with demonstrated ability to translate evaluation findings into meaningful model and product improvements
  • Expertise in designing human evaluations, scoring rubrics, datasets, and benchmarks, including measuring inter-rater agreement and validating automated graders
  • Strong qualitative and quantitative research skills, with the ability to analyze data, interpret uncertainty, and draw actionable insights into model behavior and performance
  • Excellent problem-solving and stakeholder management skills, with the ability to navigate ambiguity, prioritize risks, balance speed with rigor, and deliver recommendations that influence cross-functional decisions

While it’s not required, it’s an added plus if you also have:

  • Experience integrating automated evaluations and regression testing into engineering workflows, with familiarity in tools such as Braintrust, LangSmith, DeepEval, or equivalent platforms
  • Proven ability to develop scalable, reusable evaluation frameworks and drive the adoption of consistent quality standards across teams
  • Background in product design, product management, data science, or engineering, with familiarity in Figma or similar design and developer tools
  • Strong statistical expertise in sampling, significance testing, and agreement metrics, with experience conducting experiments or analyzing performance metrics for AI-powered features
  • Advanced training in a quantitative or behavioral discipline, gained through formal education or equivalent hands-on experience

At Figma, one of our values is Grow as you go. We believe in hiring smart, curious people who are excited to learn and develop their skills. If you’re excited about this role but your past experience doesn’t align perfectly with the points outlined in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.

Pay Transparency Disclosure

Job level and actual compensation will be decided based on factors including, but not limited to, individual qualifications objectively assessed during the interview process (including skills and prior relevant experience, potential impact, and scope of role), market demands, and specific work location.

Figma offers equity to employees, as well as a competitive package of additional benefits, including health, dental, and vision coverage; retirement benefits with company contributions; parental leave and reproductive or family planning support; mental health and wellness benefits; and paid time off. Figma provides paid sick leave, holidays, and other leave benefits in compliance with applicable federal, state, and local laws, including the requirements of the Washington Minimum Wage Act and related regulations. Exempt employees are eligible for employer‑provided paid flexible PTO in addition to flexible paid sick leave. PTO is subject to manager approval. Additional benefits may include company recharge days, cell phone and home internet reimbursements, and a number of lifestyle spending accounts. Figma also offers sales incentive compensation for most sales roles and an annual bonus plan for eligible non-sales roles. All compensation and benefits are subject to applicable plan terms and may be modified by Figma at any time, consistent with applicable law.

Annual Base Salary Range:

$169,000—$250,000 USD

At Figma we celebrate and support our differences. We know employing a team rich in diverse thoughts, experiences, and opinions allows our employees, our product and our community to flourish. Figma is an equal opportunity workplace - we are dedicated to equal employment opportunities regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity/expression, veteran status , or any other characteristic protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.

We will work to ensure individuals with disabilities are provided reasonable accommodation to apply for a role, participate in the interview process, perform essential job functions, and receive other benefits and privileges of employment. If you require accommodation, please reach out to View email address on aiapply.co. These modifications enable an individual with a disability to have an equal opportunity not only to get a job, but successfully perform their job tasks to the same extent as people without disabilities.

Examples of accommodations include but are not limited to:

  • Holding interviews in an accessible location
  • Enabling closed captioning on video conferencing
  • Ensuring all written communication be compatible with screen readers
  • Changing the mode or format of interviews

To ensure the integrity of our hiring process and facilitate a more personal connection, we require all candidates keep their cameras on during video interviews. Additionally, if hired you will be required to attend in person onboarding.

By applying for this job, the candidate acknowledges and agrees that any personal data contained in their application or supporting materials will be processed in accordance with Figma's Candidate Privacy Notice.

Application limits: To help candidates focus on the opportunities most relevant to their skills and experience, Figma limits the number and frequency of applications. These limits apply to all roles:

  • Candidates may submit up to 5 applications during any rolling 30-day period.
  • Candidates may not reapply to the same role while an application is active or within 90 days of receiving a rejection.

Once the applicable days has passed, candidates are welcome to apply again.

Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Researcher, AI Evaluations in Remote vacancy
  • AuraOne is seeking a Domain Researcher, Legal to remotely evaluate AI outputs used in legal review workflows. You will grade citations, assess statutory reasoning, and document corrected analyses for training the modeling team. Responsibilities include verifying authorities... 
    Suggested
    Remote job
    Hourly pay
    For contractors

    Auraone

    New York, NY
    2 days ago
  • Crossing Hurdles is seeking a detail-focused evaluator to design domain-specific AI assessment tasks and surface reasoning gaps. This contract role is remote, with 10-40 hours per week, ideal for researchers and experts who can verify claims against primary sources. You... 
    Suggested
    Remote job
    Contract work
    10 hours per week

    Crossing Hurdles

    New York, NY
    3 days ago
  • Softechub is seeking a Research Evaluation Specialist (PhD or Masters / Researcher) for remote, worldwide engagement. The contractor role (~15...  ...per week) emphasizes rigorous test-material development for AI model evaluation. Candidates should show self-direction, reliability... 
    Suggested
    Remote job
    Part time
    For contractors
    Worldwide
    Flexible hours

    Softechub

    New York, NY
    4 days ago
  • Crossing Hurdles is seeking a senior researcher to design challenging microbiome and metagenomics problems with verifiable solutions. You will review AI model outputs for scientific accuracy, develop rubrics grounded in primary literature, and apply expertise in metagenomics... 
    Suggested
    Remote job

    Crossing Hurdles

    New York, NY
    3 days ago
  • $152.2k - $205.9k

     ...That's the question the Price Perception Research team answers at scale, for millions of customers...  ..., interviews, diary studies, concept evaluation, and survey design, choosing the right...  ...to directors and VPs.- Apply and evaluate AI-augmented research workflows, exercising... 
    Suggested
    Work experience placement
    Remote work
    Worldwide
    Flexible hours

    Amazon

    Seattle, WA
    1 day ago
  • $184k - $265k

     ...world’s most complex projects are built. Our AI-powered Visual Intelligence Platform uses...  ...is about ✨ The role: The Sr. AI Research Engineer will build agentic systems that...  ..., or query a BIM element. Build evaluations for tasks where ground truth is ambiguous... 
    Full time
    Local area
    Remote work
    Home office
    Flexible hours

    OpenSpace

    Remote
    9 days ago
  •  ...Midstream is an AI-native financial operating system for health systems, founded...  ...Opportunity We're looking for Applied Researchers who are excited to push the boundaries of...  ...and measurable outcomes Define clear evaluation frameworks and performance metrics from... 
    Full time
    Contract work

    Midstream Health

    Remote
    15 days ago
  •  ...Healthcare is seeking a Business Analyst (Computer Use Agent) - II to support a Computer-Using Agent evaluation project. You will review AI-generated responses, perform online research, and annotate findings to improve AI instruction understanding. Responsibilities include... 

    Executive Staff Recruiters / ESR Healthcare

    New York, NY
    2 days ago
  • Turing is seeking a Business Analyst to review AI responses and validate information through detailed analysis. You will work on a CU A evaluation project and perform online research, verify claims, and provide structured annotations and feedback. Ideal candidates have... 

    Turing

    New York, NY
    5 days ago
  • $135k - $170k

     ...Principal Researcher - AI/ML Market Intelligence Fractal Analytics is a strategic analytics partner to the most admired Fortune 500...  ...competitive intelligence, market research, strategic technology evaluation, or a closely related research discipline, with a focus on... 
    Hourly pay
    Full time
    Local area
    Remote work
    Shift work

    Fractal

    New York, NY
    26 days ago
  •  ...Role Title: Research Evaluation Specialist (PhD or Masters / Researcher) Role Type: Contractor (~15 hs a week) Location: Remote, worldwide...  ...expert insight in developing rigorous test material for advanced AI model evaluation as part of a customer's project. In this... 
    Temporary work
    For contractors
    Remote work
    Worldwide

    micro1

    United States
    9 days ago
  •  ...Researchers and academics write very hard original questions and verified answers to stress-test advanced AI models. Each item needs citations to primary sources, must resist shortcuts...  ...that prior AI training, labeling or evaluation experience is required. What you would... 
    Contract work
    Temporary work
    Remote work
    Worldwide

    micro1

    United Kingdom
    6 days ago
  • $30 - $60 per hour

    Research Evaluation Specialist (PhD or Masters / Researcher / Professor) $30 - $60/hour Remote, worldwide Posted 22 days ago Role Title: Research...  ...insight in developing rigorous test material for advanced AI model evaluation as part of a customer's project. Self-... 
    For contractors
    Remote work
    Worldwide

    Softechub

    New York, NY
    5 days ago
  • Truelancer AI is looking for PhD researchers and early-career scientists in Life Sciences to participate in a large-scale AI research, scientific data validation, and quality evaluation project . If you have a strong research background and enjoy working with scientific... 
    Contract work
    Remote work
    Flexible hours

    Truelancer AI

    New York, NY
    2 days ago
  • Serverless Ventures ( is hiring an AI Researcher Intern to explore the cutting edge of AI at the intersection of Edge AI , Vertical AI ,...  ...inference Run experiments and prototype novel AI approaches Evaluate model performance, benchmark outputs, and identify failure... 
    Internship
    Local area
    Remote work

    Binary

    Austin, TX
    2 days ago
  •  ...banking professional to design Excel-based financial models for AI evaluation projects. The role is fully remote with a flexible schedule...  ...emphasizes independence, data integrity, and timely delivery in a fast-moving AI research context. #J-18808-Ljbffr Jobgether
    Remote job
    Flexible hours

    Jobgether

    New York, NY
    1 day ago
  • $60 per hour

     ...nuanced, domain-specific tasks that probe AI model limits and surface reasoning flaws....  .... Apply detailed task guidelines and evaluation rubrics consistently to each submission....  ...or technology/industry history. Advanced research skills with a track record of verifying complex... 
    Remote job
    Contract work

    Crossing Hurdles

    New York, NY
    3 days ago
  •  ...compensation models and people analytics in Excel. You will translate real-world experiences into realistic Excel tasks, and you will evaluate AI performance against precise criteria. Candidates should have 3+ years in related roles, deep Excel expertise, and strong written... 
    Remote job

    Aptura

    New York, NY
    3 days ago
  • AuraOne, Inc. is seeking a Domain Researcher, Finance to remotely evaluate AI outputs across finance and risk workflows. The contractor reviews calculations, citations, and policy adherence, flags issues, and documents corrections to train the modeling team. Responsibilities... 
    Remote job
    For contractors

    AuraOne Human Data

    New York, NY
    14 hours ago
  • Truelancer AI is seeking PhD researchers and early-career scientists in Life Sciences to join a large-scale AI research, scientific data validation, and quality evaluation project. This remote, project-based role offers flexible hours (25-35 hrs/week) and collaboration... 
    Remote job
    Flexible hours

    Truelancer AI

    New York, NY
    1 day ago
  • AuraOne, Inc. is seeking a Domain Researcher, Legal to evaluate AI outputs in legal review workflows. You will assess citations, jurisdictional reasoning, and policy adherence, flag risks, and document corrected analyses for training. This remote, contractor role requires... 
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    2 days ago
  • $40 - $80 per hour

    Synthires is seeking a Generalist - Research & AI Evaluation for a part-time contract. This remote role offers ~10-20 hours weekly at $40-$80/hour. You will craft high-quality research questions, develop evaluation materials, and summarize findings to benchmark AI models... 
    Remote job
    Contract work
    Part time

    Synthires

    New York, NY
    4 days ago
  • Synthires is seeking a detail-oriented Generalist - Research & AI Evaluation for remote, part-time contract. You will develop research questions and evaluation materials to benchmark advanced AI models, and you will gather and summarize information across diverse subjects... 
    Remote job
    Contract work
    Part time
    Immediate start

    Synthires

    San Francisco, CA
    3 days ago
  •  ...Analyst US Educational Raters for a remote contractor assignment. You will analyze educational content, perform online research, validate facts, and evaluate AI outputs with detailed explanations. Strong writing and analytical skills are essential, along with a Gemini... 
    Remote job
    For contractors

    Crossing Hurdles

    New York, NY
    2 days ago
  •  ...to join its team. This Data Scientist / Researcher will collaborate closely with other HOF...  ...learning models that enable the scalable evaluation & monitoring of millions of tech companies...  ...HOF Capital's areas of interest (e.g., AI, enterprise software, deep tech, etc.).... 
    Full time
    Work experience placement
    Remote work
    Work from home
    Flexible hours

    HOF Capital

    Brooklyn, NY
    3 days ago
  • Weekday AI seeks detail-oriented Business Analysts - Educational Raters to support evaluation and improvement of AI systems. You will analyze educational conversations, assess...  ...usefulness. Candidates should have strong research, critical thinking, and written... 
    Full time
    Contract work
    Remote work
    Weekday work

    Weekday AI

    New York, NY
    2 days ago
  • $30 - $50 per hour

    A tech company is seeking an AI Researcher to support end-to-end research for modern AI systems. This remote role involves designing experiments, defining evaluation protocols, and improving evaluation rigor for large language models. Key responsibilities include developing... 
    Remote job
    Hourly pay

    Rex.zone

    New York, NY
    1 day ago
  • $60 - $90 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...sound scientific reasoning versus superficial reasoning. Evaluate models by reviewing attempts and identifying immediate mistakes... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    New York, NY
    12 days ago
  • A leading AI company is seeking Experts in Finance to improve AI models. This fully remote and flexible role is for 10-30 hours per week, focusing on evaluating financial models and collaborating with AI researchers. Candidates should have at least 2 years in Investment... 
    Remote job
    10 hours per week
    Flexible hours

    Turing

    New York, NY
    1 day ago
  •  ...Intelligence Community. The Group conducts research and development at the intersection of...  ...inference, experimental methods, and modern AI to extract actionable information from...  ...operational use. This includes developing and evaluating prototype capabilities; integrating them... 
    Internship
    Remote work
    Shift work

    Lincoln Laboratory

    Lexington, MA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Researcher, AI Evaluations. Be the first to apply!