Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Research Scientist (Measurement and Evaluation)

Abridge

About Abridge Abridge was founded in 2018 with the mission of powering deeper understanding in healthcare. Our AI-powered platform was purpose-built for medical conversations, improving clinical documentation efficiencies while enabling clinicians to focus on what matters most—their patients. Our enterprise-grade technology transforms patient-clinician conversations into structured clinical notes in real-time, with deep EMR integrations. Powered by Linked Evidence and our purpose-built, auditable AI, we are the only company that maps AI-generated summaries to ground truth, helping providers quickly trust and verify the output. As pioneers in generative AI for healthcare, we are setting the industry standards for the responsible deployment of AI across health systems. We are a growing team of practicing MDs, AI scientists, PhDs, creatives, technologists, and engineers working together to empower people and make care make more sense. We have offices located in the Mission District in San Francisco, the SoHo neighborhood of New York, and East Liberty in Pittsburgh. The Role Abridge is hiring Research Scientists to join our Strategic Research team to rigorously evaluate and advance the real-world impact of ambient AI on patient outcomes, care quality, and provider experience. In this role, you will design and lead empirical studies of Abridge models and products in partnership with health systems, leveraging large-scale clinical conversation data to generate new insights about care delivery, documentation quality, clinical decision-making, and downstream patient outcomes. You will operationalize complex constructs—such as quality of care, safety, cognitive burden, and return on investment—using principled measurement frameworks and rigorous experimental or quasi-experimental methods. Working closely with our science and product teams, you will also develop evaluation frameworks that inform model development and product strategy. Your work will contribute to broader scientific understanding of how AI systems affect patients and providers in the real world. This role sits at the intersection of methodological innovation and practical impact, applying serious measurement science to systems that directly shape patient care. About Strategic Research : The Strategic Research team at Abridge has two primary functions: (i) designing and conducting rigorous research studies investigating the impact of ambient AI as an intervention in partnership with collaborating health systems; and (ii) supporting external research efforts that leverage Abridge data. In addition to driving and supporting empirical studies of the impact of ambient AI-enabled technologies, the team works closely with our science and engineering teams on core model evaluation. The common thread to all our work is ensuring that every partner-facing research initiative meets the highest standards of rigour, credibility, and strategic value. What You’ll Do Design and conduct evaluations of Abridge models and products Engage with external researchers and other stakeholders on designing and conducting research on ambient AI and research that leverages Abridge data Develop a strong user-centric and patient-centric mindset, grounding the research in empathy for the real world experience of providers and patients Collaborate across our cross-functional product teams to ensure the research is deeply informed by current practices and our product roadmap Write technical reports and give presentations to internal and external stakeholders Actively contribute to the wider research community by publishing original research in leading peer-reviewed publication venues Mentor research interns What You’ll Bring PhD in statistics, biostatistics, computer science, economics, information systems, clinical informatics, or a related field. Expertise in rigorous quantitative or mixed-methods approaches for conducting evaluations using observational and experimental data. Strong research track record in evaluation and measurement, as evidenced by high-impact publications at peer-reviewed journals or conferences. A problem-before-method mindset. You do not change the question to make it amenable to simple analysis, but instead push the methodological frontier to solve the real world problems that matter to health systems, providers, and patients. A curious, adaptable, and proactive mindset, with a desire to learn and grow as a researcher in a fast-paced startup environment. Passion for and understanding of Abridge’s mission. Must be willing to work from our New York City office at least 3x per week. This position requires a commitment to a hybrid work model, with the expectation of coming into the office a minimum of (3) three times per week. Relocation assistance is available for candidates willing to move to New York City. We value people who want to learn new things, and we know that great team members might not perfectly match a job description. If you’re interested in the role but aren’t sure whether or not you’re a good fit, we’d still like to hear from you. Why Work at Abridge? At Abridge, we’re transforming healthcare delivery experiences with generative AI, enabling clinicians and patients to connect in deeper, more meaningful ways. Our mission is clear: to power deeper understanding in healthcare. We’re driving real, lasting change, with millions of medical conversations processed each month. Joining Abridge means stepping into a fast-paced, high-growth startup where your contributions truly make a difference. Our culture requires extreme ownership—every employee has the ability to (and is expected to) make an impact on our customers and our business. Beyond individual impact, you will have the opportunity to work alongside a team of curious, high-achieving people in a supportive environment where success is shared, growth is constant, and feedback fuels progress. At Abridge, it’s not just what we do—it’s how we do it. Every decision is rooted in empathy, always prioritizing the needs of clinicians and patients. We’re committed to supporting your growth, both professionally and personally. Whether it's flexible work hours, an inclusive culture, or ongoing learning opportunities, we are here to help you thrive and do the best work of your life. If you are ready to make a meaningful impact alongside passionate people who care deeply about what they do, Abridge is the place for you. How we take care of Abridgers: Generous Time Off : 14 paid holidays, flexible PTO for salaried employees, and accrued time off for hourly employees Comprehensive Health Plans : Medical, Dental, and Vision coverage for all full-time employees and their families. Generous HSA Contribution : If you choose a High Deductible Health Plan, Abridge makes monthly contributions to your HSA. Paid Parental Leave : Generous paid parental leave for all full-time employees. Family Forming Benefits: Resources and financial support to help you build your family. 401(k) Matching : Contribution matching to help invest in your future. Personal Device Allowance : Tax free funds for personal device usage. Pre-tax Benefits: Access to Flexible Spending Accounts (FSA) and Commuter Benefits. Lifestyle Wallet : Monthly contributions for fitness, professional development, coworking, and more. Mental Health Support : Dedicated access to therapy and coaching to help you reach your goals. Sabbatical Leave : Paid Sabbatical Leave after 5 years of employment. Compensation and Equity : Competitive compensation and equity grants for full time employees. ... and much more! Equal Opportunity Employer Abridge is an equal opportunity employer and considers all qualified applicants equally without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability. Staying safe - Protect yourself from recruitment fraud We are aware of individuals and entities fraudulently representing themselves as Abridge recruiters and/or hiring managers. Abridge will never ask for financial information or payment, or for personal information such as bank account number or social security number during the job application or interview process. Any emails from the Abridge recruiting team will come from an @abridge.com email address. You can learn more about how to protect yourself from these types of fraud by referring to this article. Please exercise caution and cease communications if something feels suspicious about your interactions. #J-18808-Ljbffr Abridge

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Research Scientist (Measurement and Evaluation) in New York, NY vacancy
  • A health tech company in New York City is hiring a Research Scientist to evaluate the impact of ambient AI on healthcare outcomes. The role emphasizes designing studies, engaging with health systems, and fostering collaboration across product teams. A PhD in a relevant... 
    Suggested
    Work at office

    Abridge

    New York, NY
    2 days ago
  • $216k - $270k

    Scale Labs, Research Scientist — Frontier Risk EvaluationsAs the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding...  ..., you will design and create evaluation measures, harnesses and datasets for measuring the risks... 
    Suggested
    Full time

    Scale AI

    New York, NY
    2 days ago
  •  ...customers. Cohere is a team of researchers, engineers, designers, and more,...  ...and Paris. Join us!Why this role?Evaluation is critical to making progress in...  ...methods and infrastructure to measure LLM progress.As a Senior Research Scientist, Model Evaluation, you will:Create... 
    Suggested
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    1 day ago
  • Scale Labs, Research Scientist — Frontier Risk Evaluations As the leading data and evaluation partner for frontier AI companies, Scale plays an integral...  ...labs to collectively scope and design evaluations to measure and mitigate risks posed by advanced AI systems. Publish... 
    Suggested

    Scale AI, Inc.

    New York, NY
    4 days ago
  • Rex.zone is seeking an AI Research Scientist to lead applied AI research projects for US-based customers, translating open-ended questions into measurable experiments in LLM evaluation and RLHF data design. You will evaluate prompts, design datasets, and work with cross... 
    Suggested
    Remote job
    Hourly pay
    Flexible hours

    AIToolboard

    New York, NY
    2 days ago
  • $172.4k - $223.4k

    The Ads Measurement Science team in the Measurement, Ad Tech, and Data...  ...Vision.As an Applied Scientist on the team, you will lead measurement...  ...problems to propose clear evaluation frameworks and success...  ...scientists across Applied, Research, Data Science and Economist... 
    Flexible hours

    Amazon

    New York, NY
    3 days ago
  • $157.3k - $212.8k

    The Ads Measurement Science team in the Measurement, Ad Tech, and Data...  ..., and partner with other scientists and engineers to carry solutions...  ...problems to propose clear evaluation frameworks and success...  ...consulting, government, or academic research experience- Experience in... 
    Flexible hours

    Amazon

    New York, NY
    2 days ago
  • OpenRouter in New York seeks a Research Scientist to advance how the world understands, evaluates, and routes large language models. You will design experiments, build evaluation frameworks, and publish findings that influence rankings and routing decisions. You will collaborate... 

    OpenRouter

    New York, NY
    3 days ago
  • Cohere is seeking a Senior Research Scientist, Model Evaluation, to create ambitious evaluation benchmarks and scale evaluation infrastructure for...  ...functional teams to translate model feedback into trustworthy measurements and to push the frontiers of LLM evaluation. The role... 
    Remote work

    cohere

    New York, NY
    23 hours ago
  • $157.3k - $212.8k

    We are seeking an Research Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world... 
    Flexible hours

    Amazon

    New York, NY
    2 days ago
  • $30 - $50 per hour

    A tech company is seeking an AI Researcher to support end-to-end research for modern AI systems. This remote role involves designing experiments, defining evaluation protocols, and improving evaluation rigor for large language models. Key responsibilities include developing... 
    Remote job
    Hourly pay

    Rex.zone

    New York, NY
    1 day ago
  • $175k - $250k

     ...Research Scientist About Millennium Millennium is a global, diversified alternative investment...  .... Conduct applied research to evaluate new AI and machine learning techniques...  ...and implement evaluation frameworks to measure model quality, robustness, reliability,... 
    Flexible hours

    Millennium Management Corp

    New York, NY
    3 days ago
  •  ...role We're looking for a toptier Research Scientist to join our tech team. Your core responsibilities...  ...for AI agents Prototype, train, and evaluate new models for factual search and...  ...maintain an evaluations framework to measure the progress of our search engine... 
    Flexible hours

    Linkup Inc

    New York, NY
    1 day ago
  • The Applied Research Scientist will drive rigorous research to deepen our understanding of complex...  ...expertise to uncover emerging risks, evaluate safety interventions, and translate insights...  ..., and modeling to identify trends, measure impact, and evaluate interventions.- Apply... 

    TikTok

    New York, NY
    3 days ago
  • $183.8k - $248.7k

    The Ads Measurement Science team in the Measurement, Ad Tech, and Data Science (MADS) team...  ...and Computer Vision.As a Senior Applied Scientist on the team, you will be at the...  ...are a team of scientists across Applied, Research, Data Science and Economist disciplines... 
    Flexible hours

    Amazon

    New York, NY
    2 days ago
  • $172.4k - $223.4k

     ...lead the Ads industry and redefine how we measure the effectiveness of Amazon Ads business...  ..., and connecting leading-edge science research to Amazon-scale implementation? If so, come...  ...processes and results for other scientists, both junior and senior• Work with leadership... 
    Flexible hours

    Amazon

    New York, NY
    1 day ago
  • $192k - $304.75k

     ...engineering, and industrial simulation. We are seeking an applied researcher who can connect machine learning with numerical...  ..., performance counters, and physics constraints.Define evaluation methods that measure numerical impact, including convergence rate, failure rate... 
    Full time
    Remote work

    Nvidia

    New York, NY
    1 day ago
  • $196k - $230k

     ...work.About the Role:We’re seeking an experienced UX Researcher to define and scale how we evaluate Notion’s AI-powered experiences—focusing on what “good...  ...insights into reusable rubrics, workflows, and measurement approaches that product, design, engineering, and data... 
    Local area
    Shift work

    Notion Labs

    New York, NY
    1 day ago
  • $285k - $380k

     ...growing group of committed researchers, engineers, policy experts,...  ...are seeking a People Research Scientist to join our People Data Solutions...  ...instruments and ensure measurement reliability Translate survey...  ...Build measurement frameworks to evaluate and improve manager... 
    Work at office
    Visa sponsorship
    Flexible hours

    Neura Market

    New York, NY
    3 days ago
  •  ...organizations operationalize LLMs across research, product, and production...  .... The Role As a Research Scientist, you will conduct deep,...  ...advances how the world understands, evaluates, and routes large language...  .... Success in this role is measured by the quality and impact of... 

    OpenRouter

    New York, NY
    3 days ago
  • $140k - $300k

     ...AI that ships, scales, and measurably improves how healthcare works...  ...toward the RS/RE blend - applied scientists who feel comfortable getting...  .... You’ll conduct applied research on real healthcare data,...  ...running large-scale training and evaluation on distributed... 
    Work at office
    Local area
    Flexible hours
    3 days per week

    R37 Lab, R1 RCM

    New York, NY
    1 day ago
  • Member of Technical Staff, Applied Research Scientist About Fleet We work with frontier labs, hyperscalers...  ...to build the environments, tasks, and evaluations that the next generation of agents are...  ...capability they're trying to teach or measure. Push back when the ask is wrong. Ship... 
    Local area
    Immediate start

    Fleet AI, Inc.

    New York, NY
    4 days ago
  • $290.4k - $363k

     ...the intersection of cutting-edge research, large-scale engineering, and real...  ...the foundational research, evaluation methodologies, and agent/RL infrastructure...  ...how next-generation AI is built, measured, and deployed.As a Research Scientist Manager, you will lead a world-... 
    Full time

    Scale AI

    New York, NY
    1 day ago
  • $40 per hour

     ...Applied Mathematician to train AI models in a remote role. Responsibilities include providing complex math problems to AI chatbots, measuring their performance, and ensuring output quality. Ideal candidates should have strong mathematical skills and experience in related... 
    Hourly pay
    Contract work
    Remote work
    Flexible hours

    DataAnnotation

    New York, NY
    3 days ago
  •  ...AI. About the Role AI research at WRITER isn't just about publishing...  ...world. As an AI research scientist, you'll be at the center of...  ...through model training, evaluation, and production deployment...  ...limitations, establishing rigorous measures for how well models perform... 
    Full time
    Work at office
    Local area
    Flexible hours

    Writer Corporation

    New York, NY
    8 hours ago
  • Cincinnatus LLC is seeking experienced machine learning practitioners to act as ground-truth experts for model evaluation and experimentation on a leading AI lab's GenAI team. You will author complex, multi-step ML tasks and verify where frontier models fall short. This... 
    Remote job
    Full time

    Mercor

    New York, NY
    3 days ago
  • Snap Inc. seeks a Senior Marketing Scientist to drive measurement and analytics across advertising products. You will lead experimental design, causal analytics, and insights generation for strategic partners and top advertisers. Collaborate with Product, Engineering,... 

    Snap Inc.

    New York, NY
    3 days ago
  • Anthropic is seeking a Research Scientist to measure and understand recursive-self-improvement in large models. You will design evaluations, build models, and interpret results to guide research direction. Senior roles may combine hands-on work with strategic planning.... 

    Anthropic

    New York, NY
    3 days ago
  • $196k - $230k

     ...work. About The Role We’re seeking an experienced UX Researcher to define and scale how we evaluate Notion’s AI‑powered experiences—focusing on what “...  ...those insights into reusable rubrics, workflows, and measurement approaches that product, design, engineering, and... 
    Local area
    Shift work

    Notion, LLC

    New York, NY
    2 days ago
  • $196k - $230k

     ...work. About the Role: We’re seeking an experienced UX Researcher to define and scale how we evaluate Notion’s AI-powered experiences—focusing on what “...  ...insights into reusable rubrics, workflows, and measurement approaches that product, design, engineering, and data... 
    Local area
    Shift work

    Apply

    New York, NY
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Research Scientist (Measurement and Evaluation). Be the first to apply!