Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Member of Technical Staff - Evaluations

Reflection AI, Inc

Our Mission

Reflection's mission is to build open superintelligence and make it accessible to all .

We're developing open weight models for individuals, agents, enterprises, and even nation states. Our team of AI researchers and company builders come from DeepMind, OpenAI, Google Brain, Meta, Character.AI, Anthropic and beyond.
About the Role
  • Conduct critical comparative analysis to advance our understanding of model capabilities
  • Build and refine evaluation systems and processes that create tight feedback loops between data, evals, and model behavior
  • Develop generalizable evaluation frameworks that capture what matters for reasoning, alignment, and usefulness.
  • Collaborate closely with pre-training, post-training, and applied teams to translate insights into model improvements.
  • Push the boundaries of what's measurable, from synthetic evals to human feedback and real-world interaction data.
About You
  • Strong statistical analysis and experimental design skills to rigorously measure model improvements
  • Familiarity with LLM evaluation methodologies: static benchmarks, human preference evals, and/or agentic tasks.
  • High agency and thrive in a fast-paced startup environment; bias for impact over process.
  • Excited to work in a new frontier lab, defining how we measure and accelerate progress toward more capable models.
  • Collaborative, detail-oriented, and motivated by building the feedback loops that make models truly improve.
What We Offer:

We believe that to build superintelligence that is truly open, you need to start at the foundation. Joining Reflection means building from the ground up as part of a small talent-dense team. You will help define our future as a company, and help define the frontier of open foundational models.

We want you to do the most impactful work of your career with the confidence that you and the people you care about most are supported.
  • Top-tier compensation: Salary and equity structured to recognize and retain the best talent globally.
  • Health & wellness: Comprehensive medical, dental, vision, life, and disability insurance.
  • Life & family: Fully paid parental leave for all new parents, including adoptive and surrogate journeys. Financial support for family planning.
  • Benefits & balance: paid time off when you need it, relocation support, and more perks that optimize your time.
  • Opportunities to connect with teammates: lunch and dinner are provided daily. We have regular off-sites and team celebrations.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Member of Technical Staff - Evaluations in New York, NY vacancy
  •  ...provide the core infrastructure to tune, evaluate, and serve specialized models at scale...  ...more to be announced soon. Our Technical Staff develops the foundational technology that...  ...seems like a fit, please apply! As a Member of Technical Staff, you will contribute... 
    Suggested
    Live in
    Work at office
    Relocation
    Visa sponsorship

    Adaptive ML

    New York, NY
    2 days ago
  • $200k - $270k

     ...long-term success for both clients and candidates. Member of Technical Staff Location: New York City Company Stage of Funding:...  ...applications Build and improve LLM-powered systems, including evaluations, monitoring, and reliability tooling Analyze production... 
    Suggested
    Work at office
    Visa sponsorship

    Recruiting from Scratch

    New York, NY
    4 hours ago
  •  ...a typical "Applied Scientist" or "ML Engineer" role. As a Member of Technical Staff, Applied ML, you will: # Work directly with enterprise...  ...CPT, post-training, retrieval + agent integrations, model evaluations, and SOTA modeling techniques. # Influence the... 
    Suggested
    Full time
    Work at office
    Remote work
    Flexible hours

    Cohere

    New York, NY
    3 hours ago
  •  ...Member of Technical Staff — Internal AI Harness Stuut is transforming accounts receivable for B2B companies—making collections smarter and...  ...including HubSpot, Slack, Fathom, and Linear. Implement evaluation frameworks, logging, and feedback loops to continuously... 
    Suggested
    Full time
    Flexible hours
    Shift work

    Stuut

    New York, NY
    4 days ago
  • $139.9k - $274.8k

     ...create trustworthy AI that scale. We are looking for a Member of Technical Staff who is truly AI‑native—someone who experiments constantly,...  ...models, multimodal models), including prompt engineering, evaluation, or fine‑tuning. Hands‑on experience with AI-assisted coding... 
    Suggested
    Ongoing contract
    Local area

    Microsoft Corporation

    New York, NY
    4 days ago
  •  ...help create something truly transformative. The Role As a Member of Technical Staff, you'll be a core technical contributor building high‑impact...  ...precise prompting systems, fine‑tune models, and develop evaluation frameworks to deliver consistent, reliable results from... 
    Local area

    Atomic

    New York, NY
    1 day ago
  •  ...Activant, 1984 Ventures and Page One. The Role We’re hiring a Member of Technical Staff - AI/ML to design, build, and deploy AI-powered systems...  ..., or similar) — feature engineering, model selection, evaluation, calibration Have strong opinions on AI/ML evals — golden... 
    Full time
    Flexible hours

    Stuut

    New York, NY
    4 days ago
  • $175k - $220k

     ...Member Of Technical Staff, Cloud Infrastructure New York, NY; San Mateo, CA At Fireworks, we're building the future of generative AI infrastructure...  ...into robust infrastructure solutions. Continuously evaluate and integrate cloud-native and open-source technologies (e... 

    Fireworks AI

    New York, NY
    4 days ago
  •  ...provide the core infrastructure to tune, evaluate, and serve specialized models at scale...  ...more to be announced soon. Our Technical Staff develops the foundational technology that...  ...and help document findings Nearly all members of our Technical Staff work across both... 
    Internship
    Live in
    Work at office

    Adaptive ML

    New York, NY
    4 days ago
  •  ...and outside of the Asset Data role: Technical Skills: Drive adoption of real-time asset...  ...at Anchorage Digital by 2x Member of Technical Staff (Infrastructure) Business Analyst - Workforce...  ...Technical Staff, Data Analysis and Evaluation We’re unlocking community knowledge in... 
    Full time
    Work at office
    Remote work

    Anchorage Digital

    New York, NY
    1 day ago
  •  ...US-based 501(c)(3). Job Description: We are looking for a Member of Technical Staff, Research to investigate, design, test and develop state of...  ...logging results for later review. Develop a framework to evaluate the models’ learning using visual and statistical tools to... 
    Remote work

    Firstprinciples

    New York, NY
    1 day ago
  • $180k - $238.1k

     ...with engineers to tackle all aspects of evaluating database performance under a wide spectrum...  ...first 30 days, you will become an integrated member of our performance engineering team. You’...  ...experience level ranges from mid to staff level. At a minimum, this role requires:... 
    Local area
    Remote work
    Worldwide
    Flexible hours

    Cockroach Labs

    New York, NY
    1 day ago
  •  ...let's talk anyway. - if you find something here that resonates, mention it in your application. About the Role Members of Technical Staff at Anterior own problems end-to-end - from system design through to production. You'll build and scale the core platform... 
    Apprenticeship
    Flexible hours

    Anterior

    New York, NY
    2 days ago
  •  ...context. You think critically and consider both the business and technical tradeoffs of your solutions. Not ideological about...  ...design product experiences that help automate common workflows ~ Evaluating LLMs across a diversity of life science specific tasks ~ Government... 

    Valkai

    New York, NY
    3 days ago
  •  ...Pace Technical Staff Role Pace is an AI-native business process outsourcer for insurers. We combine the speed of AI agents with expert...  ...of the largest companies in the world. We're looking for a Member of Technical Staff who will partner with our team on product.... 

    Pace

    New York, NY
    3 days ago
  • $160k - $320k

     ...be able to concisely and accurately share knowledge with their teammates. About the Role We're seeking a remarkable Member of Technical Staff to join our team in managing and enhancing reliability, automating processes, and conjuring excellent experiences for platforms... 
    Work at office

    Liquid

    New York, NY
    2 days ago
  • $119.8k - $234.7k

     ...proven expertise, demonstrated through impactful publications or technical leadership on high-scale projects. Possess strong...  ...architectures, conduct experiments, champion measurement and evaluation, innovate datasets and data pipelines. Improve training and... 
    Ongoing contract
    Work at office
    Local area

    Microsoft Corporation

    New York, NY
    2 days ago
  •  ...Member Of Technical Staff, Product Listen is building the human layer of AI. We're Sequoia-backed, raised $100M, and our customers include...  ...what McKinsey does for $1M per engagement. The bottleneck is evaluating those qualitative outputs. Once you have the eval, you can... 
    Flexible hours
    Shift work

    Listen Labs

    New York, NY
    1 hour ago
  •  ...Member of Technical Staff - IT Engineer Reflection AI is looking for a Member of Technical Staff - IT Engineer. In this role, you'll be expected to manage a broad range of operational, strategic, and project-based responsibilities with maximum autonomy and minimal oversight... 
    Work at office
    Relocation package

    Reflection AI

    New York, NY
    5 hours ago
  • $119.8k - $234.7k

     ...Overview As a Member of Technical Staff - Software Engineer & Machine Learning, you will work building AI Insights, a Copilot analytics product...  ...-on with observability (metrics, tracing, logs) and model evaluation frameworks. Qualifications Required Qualifications:... 
    Ongoing contract
    Work at office
    Local area

    Microsoft Corporation

    New York, NY
    7 days ago
  •  ...directly shape how users interact with ATG, shaping the future of mobile AI experiences. As a key member of our team, you'll push the boundaries of iOS development, combining technical excellence with design finesse. From pixel-perfect UI to smooth animations and thoughtful... 
    Shift work

    ATG intelligence

    New York, NY
    3 hours ago
  •  ...data generation and reinforcement learning pipelines at scale. Build high-performance inference platforms capable of serving and evaluating models across thousands of GPUs. Optimize throughput, latency, and GPU utilization for large language model inference and... 
    Relocation package

    Reflection AI

    New York, NY
    1 hour ago
  • $189.59k

     ...Portugal; Singapore; and Sioux Falls, South Dakota. Learn more at anchorage.com, on X @Anchorage, and on LinkedIn. Job title: Member of Technical Staff, Banking SolutionsCompany name: Anchor LabsJob site address: New York, New YorkJob Requirements: Position requires a... 
    Bank staff
    Remote work

    Crypto Pro Network

    New York, NY
    3 days ago
  • $160k - $320k

     ...be able to concisely and accurately share knowledge with their teammates. About the Role We're seeking a remarkable Member of Technical Staff to join our team to design a central intelligent trading unit that can trade autonomously for tens of thousands of users.... 
    Work at office

    Liquid

    New York, NY
    2 days ago
  •  ...backed by top-tier investors including a16z, Khosla, Activant, 1984 Ventures and Page One. The Role We're hiring a Member of Technical Staff - Applied AI, Fullstack to design, build, and scale end-to-end systems that power Stuut's platform for B2B financial... 
    Full time
    Flexible hours

    Stuut

    New York, NY
    1 day ago
  • $185k - $200k

     ...collaborative culture and exchange knowledge with a highly experienced technical organization. Ensure that CockroachDB remains scalable,...  ...In your first 30 days, you will become an integrated member of our engineering team. You'll spend time learning about the Storage... 
    Local area
    Remote work
    Flexible hours

    Cockroach Labs

    New York, NY
    5 hours ago
  •  ...About the Role Own the red-teaming and adversarial evaluation pipeline for Reflection's models, continuously probing for failure...  ..., or equivalent practical experience in AI Safety. Deep technical understanding of LLM safety, including adversarial attacks, red... 
    Relocation package

    Reflection AI, Inc

    New York, NY
    4 days ago
  •  ...sponsor quarterly in‑person collaboration days to work together and further deepen our Village. A successful member of this team would help bridge business and technical uses of data, creating an easy to use platform for Data and Data Tooling for Engineering, Reporting,... 
    Work at office
    Remote work

    Crypto Pro Network

    New York, NY
    1 day ago
  •  ...datasets, metadata, provenance, and versions so experiments are reproducible and it’s clear what data went into which training and evaluation runs Own CI/CD and development tooling for the data stack (GitHub, Python, PyTorch), and automate repetitive workflows to reduce... 

    Reka

    New York, NY
    1 day ago
  • $125k - $185k

     ...About the job Founding AI Engineer / Member of Technical Staff YC - Startup Role: Founding AI Engineer LocationPrimary: New York...  ...Contribute across the stack when needed (APIs, internal tools, evaluation dashboards) to keep the overall AI surface area robust,... 
    Temporary work
    Work at office

    Butterfly Recruitment

    New York, NY
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Member of Technical Staff - Evaluations. Be the first to apply!