Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Research Program Manager - Model Data & Evaluations Flywheel

Sierra Ventures

Our Mission

Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. We build open models that let anyone control their intelligence and help shape the future of AI. Our mission: make intelligence open and accessible to all.

About the Role

Research Program Managers at Reflection are high-leverage leaders and operators who embed directly with research and infrastructure teams to accelerate the pace of frontier model development. They are not project trackers. They are force multipliers who bring clarity to ambiguity, drive decisions when the path forward is unclear, and ensure that the work happening across multiple teams connects into a coherent whole.

This is a foundational role. Reflection is building model evals and safety from the ground up, and this RPM will be at the center of that effort. You won't be stepping into an established function with existing processes and tooling. You will be the person who figures out what this function needs to look like, stands it up, and makes it real. That means defining the evaluation frameworks, building the operational infrastructure for model safety, establishing the processes that connect evals to the model development lifecycle, and laying the groundwork for how Reflection interfaces with the broader safety ecosystem. This is 0-to-1 work in its purest form.

You bring a first-responder mentality. When things go sideways, you don't wait to be asked. You jump in, assess the situation, cut through noise, align the people who need to be aligned, and drive resolution.

What You'll Do
  • Build the foundational infrastructure for model evals and safety at Reflection. Define the evaluation frameworks, tooling requirements, and operational processes that will underpin how we assess model capabilities, risks, and readiness for release.

  • Stand up model safety operations as a function, including establishing the workflows, review cadences, and decision frameworks that connect safety evaluation to the model development and release lifecycle.

  • Partner with research and engineering leads across pre-training, mid-training, and post-training to embed safety and evaluation checkpoints into the development process in a way that is rigorous without being a bottleneck.

  • Drive the scoping and prioritization of eval science and eval infrastructure investments, working with technical leads to determine what to build in-house, what to adopt, and where to invest research effort.

  • Establish Reflection's engagement with the external safety ecosystem, including third-party assessments, academic partnerships, and industry safety frameworks. Represent the company's safety posture to external stakeholders with credibility and clarity.

  • Create visibility and reporting structures that give leadership a clear, honest picture of model safety status, evaluation coverage, and open risks, so they can make informed decisions at the pace the business requires.

  • Champion a culture of blameless post-mortems and continuous learning, turning every safety-relevant finding into a concrete improvement to our systems and processes.

About You
  • 7+ years of experience in technical program management, research operations, or ML engineering, with demonstrated experience standing up new functions, teams, or programs from scratch.

  • Familiar with the landscape of model evaluation and AI safety, including evaluation methodologies, red-teaming, alignment research, and the evolving regulatory and industry safety ecosystem. You don't need to be a safety researcher, but you need to understand the space well enough to make sound judgments about what matters and what to prioritize.

  • Deep enough technically to engage with researchers and engineers on topics like model behavior, evaluation design, data pipelines, and safety-critical system architecture. You follow the technical thread and you know when something doesn't add up.

  • Proven ability to build structures where none exists. You've taken ambiguous mandates and turned them into functioning programs with clear ownership, measurable outcomes, and durable processes.

  • Strong stakeholder management skills spanning deeply technical ICs, research leadership, and external partners. You build trust through competence and follow-through.

  • Excited to build from zero to one. We are a small, fast-moving team and this role will help define how model safety and evaluation works at Reflection.

  • Motivated by enabling researchers and engineers to build the world's most capable open-weight AI systems, responsibly.

What We Offer:

We believe that to make intelligence open and accessible to all, you need to start at the foundation. Joining Reflection means building from the ground up as part of a talent-dense team. You will help define our future as a company, and help define the future of open foundational models.

We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.

  • Top-tier compensation: Salary and equity structured to recognize and retain our talent globally.

  • Stock options: Everyone who joins and contributes to Reflection's success gets to share in the upside through stock options.

  • Health & wellness: Comprehensive medical, dental, vision, and life, with an annual wellness allowance.

  • Meals: Lunch and dinner are provided in the office daily.

  • Life & family: 22 weeks paid parental leave for all new birthing and non-birthing parents, including adoptive and surrogate journeys.

  • Vacation days: Unlimited paid time off in the U.S. and 30 days in the U.K.

  • Sponsorship support: We sponsor visas to help exceptional talent join our team and support long-term immigration pathways where applicable.

  • Team building: We have regular off-sites, happy hours, and team celebrations.

Export Control Notice: This position may require access to technology or source code subject to the U.S. Export Administration Regulations. Any offer of employment for this role may be conditioned on the Company's ability to provide the candidate with access to such technology or source code in compliance with applicable U.S. export control laws, which may require the Company to seek government authorization.

Candidate and Employee Privacy Notice: We collect and process personal data about applicants for the purposes described in our Candidate and Employee Privacy Notice

#J-18808-Ljbffr
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Research Program Manager - Model Data & Evaluations Flywheel in San Francisco, CA vacancy
  •  ...Research Program ManagerThe Human Data team at OpenAI is responsible for identifying and...  ...AI systems by designing evaluations, surfacing vulnerabilities...  ...researchers to strengthen model reliability and public trust...  ....As a Research Program Manager, you will lead... 
    Suggested
    Work at office
    Relocation package

    OpenAI

    San Francisco, CA
    1 day ago
  •  ...About the Team OpenAI's Research Data Team exists to accelerate the evaluation, safety and capabilities of our models and products. Made up of technical operators and software...  ...data. About the Role As a Research Program Manager (RPM), Data Acquisition, you will... 
    Suggested

    OpenAI

    San Francisco, CA
    1 day ago
  • $239k - $328k

    # Research Program Manager, Frontier AssuranceOpenAISan Francisco (Hybrid)Full-time$239,000 – $328,...  ...independent assessments of OpenAI’s frontier models and safeguards, coordinating external...  ...engages with frontier-model evaluations, safeguards, and AI safety research.... 
    Suggested
    Full time
    Work at office
    Remote work
    Relocation package

    We Love AI Jobs

    San Francisco, CA
    2 days ago
  • $365k

     ...growing group of committed researchers, engineers, policy experts,...  ...organization works across the full model development lifecycle, from...  ...development. As a Technical Program Manager for Research, you'll define...  ...-on exposure to training, evaluation, or large-scale distributed... 
    Suggested
    For contractors
    Work at office
    Relocation
    Visa sponsorship
    Work visa
    Flexible hours
    Shift work

    Anthropic Limited

    San Francisco, CA
    3 days ago
  •  ...AfterQuery AfterQuery is an applied research lab curating data solutions for foundation model development. We serve every...  ...is hiring a Research Program Manager to run the operations behind its...  ...with working Python and SQL. Evaluation literacy: you have taken part... 
    Suggested
    For contractors
    Local area

    AfterQuery

    San Francisco, CA
    3 days ago
  • $136.44k - $265.11k

     ...structured scientific data and AI are built into...  ...and run AI agents and models directly in their workflows...  ...build the datasets, evaluations, and systems that help...  ..., and external research partners. Desire...  ...comprehensive benefits program including equity, health... 
    Full time
    Work at office
    Local area
    Monday to Friday
    Shift work

    Benchling

    San Francisco, CA
    1 day ago
  •  ...growing group of committed researchers, engineers, policy...  ...to build the evaluations that tell us — and the...  ...leadership use to monitor model health during training...  ...qualifications Strong Python programming skills, including...  ...distributed systems, data pipelines, or other... 
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic Limited

    San Francisco, CA
    3 days ago
  •  ...About the Team OpenAI's Human Data Team creates custom data solutions driving groundbreaking research. Our work enhances and evaluates our flagship models and products like ChatGPT, GPT-5,...  .... About the Role As a Research Program Manager (RPM) in the Human Data team you... 
    Full time
    Interim role

    OpenAI

    San Francisco, CA
    2 days ago
  •  ...OpenAI, Inc. seeks a Data Program Manager in the Human Data team to drive complex data collection programs, coordinating with researchers, operations, and external vendors. You will ensure...  ...high-quality data for training and evaluation, acting as a key interface between... 

    OpenAI, Inc.

    San Francisco, CA
    3 days ago
  • $207k - $230k

     ...Data Program Manager, Human Data | OpenAI Careers Data Program Manager...  ...signals for training and evaluation. We design and run end-to-end...  ...high-stakes uses of our models. Our remit spans bespoke data...  ...partner closely across all research teams to translate these signals... 
    Work at office
    Local area
    Relocation package
    Flexible hours
    Shift work

    OpenAI, Inc.

    San Francisco, CA
    3 days ago
  •  ...Safety Measurement Research Program ManagerSafety Systems manages the complete lifecycle of safety...  ...for OpenAI's frontier models, ensuring our models are...  ...safeguards and model training to evaluation and red-teaming—all aimed...  ...with safety research, data science, and engineering... 
    Relocation package

    VU Venture Partners

    San Francisco, CA
    2 days ago
  •  ...Research Program ManagerSafety Systems manages the complete lifecycle of safety efforts for OpenAI's frontier models. We work across research, engineering, product, and policy to understand...  ...across launches.Coordinate evaluations and safety assessments. Organize capability... 
    Work at office
    Immediate start
    Relocation package

    Openai

    San Francisco, CA
    2 days ago
  • $129k - $207k

     ...Senior Technical Program Manager – AI & Data Analytics Location: Oakland / San Ramon, CA (Hybrid – 2–3 Days Onsite) Employment Type: Full-Time...  ...demand management , including enterprise intake, portfolio evaluation, and project prioritization. Experience using Project... 
    Permanent employment
    Full time

    Swoon

    San Francisco, CA
    3 days ago
  • $180.2k - $252.3k

    # Staff Program Manager, Human & LLM Data Annotation OperationsRemote - United States · Remote · Posted 12...  ...its central labeling engine to power models, taxonomies, and product experiments...  ...us. We will use this information to evaluate your application for employment or... 
    For contractors
    Work experience placement
    Work at office
    Remote work
    Flexible hours

    Localization Unlocked

    San Francisco, CA
    1 day ago
  •  ...cutting-edge foundation AI models and end-to-end products...  ...business problems.Evaluation is critical to making progress...  ...evaluations.Conduct research to advance the state-of...  ...; refining LLM-based data synthesis pipelines;...  ...office: a daily lunch program, plenty of snacks, and... 
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    San Francisco, CA
    15 hours ago
  •  ...Mercor connects elite creative and technical talent with leading AI research labs. This remote contract role, Business Domain Expert — AI Model Evaluation Pilot, asks you to judge model outputs and provide clear, evidence-grounded rationales. With 5+ years in Admin, Marketing... 
    Contract work
    Remote work

    Mercor Inc

    San Francisco, CA
    15 hours ago
  • $240k - $333k

     ...Senior Technical Program Manager II, Strategic Operations, Data Center Planning Share Senior Technical Program Manager II, Strategic Operations, Data...  ...Partner with cross-functional leadership to continuously evaluate and optimize business processes and operational... 

    Jobleads-US

    San Francisco, CA
    3 days ago
  • $257k

     ...Technical Program Manager, Storage & Data Infrastructure Technical Program Management...  ...experiences. As frontier models and agents evolve how they...  ...architectural inputs. Evaluate physical versus logical footprint...  ...OpenAI OpenAI is an AI research and deployment company... 
    Work at office
    Relocation package

    Jobleads-US

    San Francisco, CA
    2 days ago
  •  ...Data Program Manager, ScienceOpenAI's Research Data team exists to accelerate the evaluation, safety and capabilities of our models and products. Made up of technical operators and software engineers, we design the methods through which we acquire and create data.As a... 
    Local area

    Openai

    San Francisco, CA
    3 days ago
  • $112k - $209k

     ...lead to the same place.We are seeking a Data Program Manager to join K&L Gates. The Data Program...  ..., and the governed build-buy portfolio model.• Leading end-to-end delivery of data initiatives...  ...key performance indicators to evaluate program success and value, translating... 
    Full time
    Temporary work
    Work at office
    Local area
    Remote work

    K&L Gates

    San Francisco, CA
    4 days ago
  • $365k

     ...growing group of committed researchers, engineers, policy experts,...  ...organization works across the full model development lifecycle, from...  ...development. As a Technical Program Manager for Research, you'll define...  ...‑on exposure to training, evaluation, or large‑scale distributed... 
    Visa sponsorship
    Shift work

    Anthropic Limited

    San Francisco, CA
    4 days ago
  •  ...OpenAI seeks a Research Program Manager to lead initiatives that test the safety and robustness of its models through creative experimentation and structured evaluation. You will coordinate across research and engineering teams to transform ambiguous risks into concrete... 
    Work at office
    Relocation package

    Jobleads-US

    San Francisco, CA
    2 days ago
  •  ...Reflection is building model evals and safety from the ground up, and this RPM will be...  ...center of that effort. You will shape the evaluation frameworks, operational infrastructure...  ...broader safety ecosystem, coordinating across research and engineering to accelerate frontier... 

    Jobleads-US

    San Francisco, CA
    5 days ago
  • $125k - $160k

     ...Global Logistics Program Manager, Data Center and High-Tech Hardware Skypace is the ocean freight division of Maxton Shipping Inc., an FMC-licensed NVOCC (OTI license 027662) headquartered in California. Skypace runs ocean freight and 4PL programs for 200+ organizations... 
    Contract work
    Remote work
    Overseas

    Skypace

    San Francisco, CA
    1 day ago
  •  ...Cloud Engineer (Coding Agent Experience) to work remotely on frontier AI coding agents for infrastructure tasks. The role covers evaluating model-generated infrastructure, cloud platforms, Kubernetes, CI/CD, observability, and automation. Candidates should have 2+ years... 
    For contractors
    Remote work

    Mercor Inc

    San Francisco, CA
    4 days ago
  • $163k - $236k

     ...Technical Program ManagerA problem isn't truly solved until it's solved...  ...globe. As a Technical Program Manager at Google, you'll use your...  ...transmission and distribution, data center technology, electrical...  ...and transmission reliability modeling tools and concepts.Experience... 
    Temporary work

    CompliancePoint

    San Francisco, CA
    1 day ago
  •  ...Francisco, CA seeks a technically deep Technical Program Manager to lead cross-functional programs across data platforms, online databases and storage infrastructure...  ...assistance for new hires. You will translate model, product and data-consumer needs into architecture... 
    Work at office
    Relocation package

    OpenAI

    San Francisco, CA
    3 days ago
  • $207k - $285k

     ...for providing the training data to advance our models’ intelligence and expand...  ...capabilities. We build and manage some of the largest datasets...  ...you’ll work directly with researchers to create the datasets we...  ...Lead end-to-end programs to acquire and deliver high... 
    Work at office
    Relocation package

    OpenAI

    San Francisco, CA
    1 day ago
  •  ...DeepMind is seeking a Technical Program Manager to bridge frontier AI research with Google’s product ecosystems across...  ...sites. You will scale operating models for cross-product engagements,...  ...range roadmap alignment, structured evaluations, and coordinating across research... 

    Jobleads-US

    San Bruno, CA
    1 day ago
  • $85 per hour

     ...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud... 
    Contract work
    Summer work
    Remote work

    Mercor Inc

    San Francisco, CA
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Research Program Manager - Model Data & Evaluations Flywheel. Be the first to apply!