Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Code Quality Specialist / Research Engineer

$75 - $105 per hour

Braintrust

About the Opportunity

Our client, a leading AI research organization, is looking for engineers and computational researchers to help evaluate how AI systems solve coding tasks.

You’ll investigate whether an unsuccessful result comes from the AI’s code, an issue in the tests, or unclear task requirements. This work combines code reading, debugging and technical judgment to help make AI evaluations more accurate and reliable.

The role is suited to people who enjoy understanding unfamiliar codebases, examining test behavior and explaining technical findings clearly.

What You’ll Do

  • Review code, tests, execution logs and the steps an AI coding agent took while attempting a task.
  • Investigate reported problems in coding benchmarks, including undocumented expectations, missing definitions and overly restrictive test assertions.
  • Determine whether a reported issue is a genuine flaw, a gap that still permits reasonable solutions to pass, or an error in the AI’s implementation.
  • Assess whether an identified issue affected the task’s pass/fail result.
  • Write concise technical explanations supported by relevant excerpts from task instructions, tests and run logs.
  • Diagnose test-harness and environment issues using Docker configuration, dependencies and execution logs.
  • Participate in daily calibration discussions with research scientists to compare findings and align evaluation decisions.

Individual claim investigations are expected to take approximately 30–60 minutes. You’ll use review tools and supporting technical resources to examine the evidence and document a clear conclusion.

What We’re Looking For

  • Either 2+ years of professional experience writing, reviewing and debugging software , or a completed STEM master’s degree or PhD with intensive programming and computational research experience.
  • Strong Python code-reading, debugging and comprehension skills, including familiarity with complex frameworks and test assertions.
  • Reading familiarity with at least one additional language: JavaScript/TypeScript, Go, C++ or Rust.
  • Practical experience with Git and Docker.
  • Familiarity with unit-testing frameworks, such as pytest, unittest, Jest or Mocha, and software build environments.
  • Careful attention to edge cases, ambiguous requirements and discrepancies in test logic.
  • Fluent professional English, with the ability to communicate technical findings clearly in writing and conversation.

Nice to Have

  • Experience with LLM benchmarking, AI evaluation, prompt engineering or coding-agent workflows.
  • Familiarity with benchmarks such as SWE-bench or CodeContests.
  • Experience in AI red teaming, safety or reliability auditing.
  • Previous collaboration with AI or machine-learning research scientists.

Engagement Details

  • Location: United States only.
  • Expected commitment: 35 hours per week.
  • Start: Immediate.

Compensation

$75–$105 USD per hour, depending on relevant skills and experience.

Interview Process

Shortlisted candidates will be invited to a 15-minute AI interview. The hiring team will then review your profile and interview responses to determine next steps, with no additional interview rounds required.

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Code Quality Specialist / Research Engineer in Boston, MA vacancy
  • $30.6 - $44.51 per hour

     ...Mass General Brigham Coding Quality Assurance Specialist Mass General Brigham relies on a wide range of professionals, including doctors, nurses, business people, tech experts, researchers, and systems analysts to advance our mission. As a not-for-profit, we support... 
    Suggested
    Remote job
    Hourly pay

    MGH Institute of Health Professions

    Somerville, MA
    1 day ago
  • Mass General Brigham is seeking an Inpatient Coding Education & Improvement Specialist to support inpatient coding quality, education, and continuous improvement. You will review findings, identify trends, and develop targeted education in collaboration with coding operations... 
    Suggested
    Remote job

    Mass General Brigham (Enterprise Services)

    Somerville, MA
    1 day ago
  •  ...Galois From building digital engineering tools that make space...  ...Are We are a community of researchers, engineers, and operations people...  ...Galois, and developing library code to be shared among projects...  ...to self-manage deadlines and quality goals Preference will be given... 
    Suggested
    Local area
    Immediate start

    Galois

    Boston, MA
    4 days ago
  •  ...that. Tutor is a robotics research and deployment company, building...  ...We’re a team of researchers, engineers, technicians, operators, and...  ...that catch degraded sensor quality. Build reproducible datasets...  ...Python, with maintainable code, tests, and reproducible experiments... 
    Suggested
    Hourly pay
    Full time
    Work at office

    Tutor Intelligence

    Watertown, MA
    11 days ago
  • $100k - $300k

     ...Watertown, MA R&D - Research / Full-time / On-site The Company We...  ...state-of-the-art hardware, every line of code written at Tutor has a direct impact on the...  ...extraordinary things. As a research engineer, your work lies at the center of this challenge... 
    Suggested
    Full time
    Work at office

    Tutor Intelligence

    Watertown, MA
    2 days ago
  • $82.3k - $220k

     ...is an independent, nonprofit research and development company headquartered...  ...exploration to biomedical engineering, lives often depend on the...  ...existing best practices and Quality Management System processes.•...  ..., computer language coding, descriptive modeling (e.g. SysML... 
    Full time
    Local area

    Draper Laboratory

    Somerville, MA
    4 days ago
  •  ...develops, maintains, and researches the relationship...  ...are a diverse group of engineers and research scientists...  ...who can conduct high-quality self-directed research...  ...implementing AI-assisted coding and familiarity with...  ...engineers, statisticians, and specialists in the financial... 
    Worldwide

    Verisk Analytics

    Boston, MA
    1 day ago
  •  ...About Code Metal Code Metal is the leader in automated software engineering you can trust. As AI writes more of the world's code, the bottleneck in software has shifted...  .... About the Role Code Metal is seeking a Research Engineer specializing in Formal Methods to join... 
    Full time
    Remote work
    Relocation package
    Flexible hours
    Shift work

    code-metal

    Boston, MA
    5 days ago
  • $180k - $315k

    Watertown, MAR&D - Research /Full-time /On-siteAbout TutorWe believe robots are the biggest...  ...United States. We’re a team of researchers, engineers, technicians, operators, and sellers who...  ...cloud infrastructure, infrastructure as code, or ML deployment.Operator tools and... 
    Hourly pay
    Full time
    Work at office

    Tutor Intelligence

    Watertown, MA
    4 days ago
  • $82.5k

     ...Job Location Boston, MA, United States Tracking Code Posted Date 9/25/2026 Salary Grade Grade 0...  ...Position Type Full-Time/Regular The Electrical Engineer / Research Scientist will lead the design, development, characterization... 
    Full time
    For contractors
    Work experience placement
    Work at office
    Flexible hours

    Boston University

    Boston, MA
    3 days ago
  •  ...Research Software EngineerResearch Software Engineer, McGovern Institute for Brain Research, to work on the Connectome Annotation Versioning Engine (CAVE) infrastructure...  ...operational documentation; and debugging and testing code. The Dorkenwald Lab works on synapse-resolution... 

    MIT

    Cambridge, MA
    4 days ago
  • $200k - $250k

     ...investment firm is seeking a Machine Learning Research Engineer to join a highly technical team focused...  ...clean, organized, and maintainable code that supports ongoing research efforts...  ...maintaining high standards for code quality and validation Continuously experiment,... 
    Full time
    Remote work

    Motion Recruitment

    Boston, MA
    10 hours ago
  • $145.92k - $191.05k

     ...Senior Research Software EngineerLocation: Boston, Massachusetts, USA (Hybrid, a few days...  ..., regression resolution, and long-term code quality improvements.Improve software...  ...D software tools for scientists.Mentor engineers and contribute technical leadership across... 
    Contract work
    Work at office

    Quantum Jobs USA

    Boston, MA
    2 days ago
  • $90k - $112.5k

     ...workplace. About the role: As a Principal Quality Specialist, you will become a member of the...  ...Responsibilities: Ensuring contract and code requirements are maintained throughout...  ...Assurance effort Coordinate with project engineering to determine/communicate customer and regulatory... 
    Contract work
    Relocation
    Flexible hours

    Westinghouse

    Boston, MA
    5 days ago
  •  ...of Massachusetts Job Description - Quality Control Specialist (26000737) Job Description Quality...  ...or belligerent, in conjunction with research activities that are significantly different...  ...work accurately with names, numbers, codes and/or symbols and assemble items of... 
    Work at office
    Local area

    Mass Inc

    Boston, MA
    5 days ago
  • $147k - $210k

     ...product or system development code.Participate in, or lead...  ...network, or service operations and quality.Minimum qualifications:...  ...machine learning or software engineering.2 years of experience with software...  ...Learning and Neuroscience research scientists. Your responsibilities... 

    Google

    Somerville, MA
    4 days ago
  • $106k - $176.6k

     ...systems you build will support researchers tackling some of the toughest...  ...alongside experts in AI, engineering, computational biology, and drug...  ...Deliver production-quality software and AI solutions...  ...grow through collaboration, code reviews, and technical coaching... 
    Permanent employment
    H1b
    Local area
    Visa sponsorship
    Work visa
    Relocation package
    2 days per week

    Pfizer

    Cambridge, MA
    4 days ago
  •  ...Research EngineerPercepta's mission is to transform critical institutions with applied AI. We care that industries that power the world...  .... We bring together:Forward-deployed expertise in engineering, product, and researchMosaic, our in-house toolkit for rapidly... 
    Work at office
    Flexible hours

    Percepta

    Boston, MA
    1 day ago
  •  ...SPHERE Research EngineerThe Security and Privacy Heterogeneous Environment for Reproducible Experimentation (SPHERE) is a newly funded...  ...aligned with cybersecurity and privacy research needs.The Research Engineer will help build this innovative cyberinfrastructure. The... 
    Work experience placement
    Remote work
    Worldwide

    Northeastern University

    Boston, MA
    3 days ago
  •  ...methods or transformer-only architectures, combining cutting-edge research with real-world deployment. Our solutions are already deployed...  ...driven by real field use. We are seeking a Research Engineer to join our manipulation efforts. In this role, you will work... 

    FieldAI

    Boston, MA
    17 days ago
  • $67.86k - $95.85k

     ...About the Opportunity Northeastern University is seeking a motivated Research Engineer to support computational modeling and imaging efforts investigating the effects of non-ionizing electromagnetic fields on the human nervous system. Working under the direction of... 
    Contract work
    Work experience placement
    Worldwide

    NU USA Northeastern University

    Boston, MA
    1 day ago
  • $105.3k - $168.5k

     ...We are seeking experienced and highly motivated Research Engineers to join our R&D team in Londonderry, NH. Position opens for Junior to Senior level commensurate with experience. Area: Londonderry - NH In Outokumpu’s EvoMaterials Team, we aim to leverage the best available... 
    Flexible hours

    Outokumpu

    Boston, MA
    1 day ago
  • $57.2k - $108.8k

     ...RTX BBN Technologies, a subsidiary of RTX Corporation, is the research company behind many revolutionary advances that we take for granted...  ...first routers were implemented. Today, BBN's scientists and engineers are working to advance the state-of-the-art on topics ranging... 
    Full time
    Temporary work
    Work experience placement
    Internship
    Work at office
    Remote work
    Relocation
    Flexible hours

    RTX

    Cambridge, MA
    10 hours ago
  • $189k - $260k

     ...Biomedicines is seeking a Director/ Senior Director of Wet Lab Engineering to lead our multidisciplinary team in building the...  ...degree in a physical engineering discipline,; equivalent depth of research and industry experience will also be considered. An inventive... 
    Shift work

    Generate Biomedicines

    Somerville, MA
    4 days ago
  • $140k - $160k

     ..., we shape the future by inventing high-quality technologies and services that spark enthusiasm...  ...: At Bosch, you change lives.The Bosch Research and Technology Center North America with...  ...Research is seeking a Research Engineer to conduct applied research in computational... 
    Work experience placement
    Worldwide

    Robert Bosch

    Watertown, MA
    10 hours ago
  • $20 - $60 per hour

     ...role, you will be a key contributor to the Research team, focusing on the integration and...  ...DoCollaborate with a multidisciplinary team of engineers and scientists to design, develop, and...  ...ensuring they meet the operational and quality standards required for clinical and... 
    Permanent employment
    Full time
    Summer work
    Internship
    Work at office

    Moderna Therapeutics

    Somerville, MA
    4 days ago
  • $177k - $225k

    As a SLAM & Spatial AI Research Engineeron the Atlas VLA Research team, you will...  ..., robotics, ML & system software specialists and rapidly test your work on...  ...and how to combine them.Write high-quality, maintainable C++ and Python code that fits into a large production... 
    Full time
    Work at office

    Boston Dynamics

    Waltham, MA
    10 hours ago
  •  ...superhuman capabilities. We are defining the frontier of robotics research and deployment. Our team consists of pioneers in robotics...  ...robotics. Responsibilities Evaluate and extend physics engines (e.g. MuJoCo, Newton) Scale and optimize GPU-based physics simulators... 
    Full time
    Immediate start

    ekarobotics

    Boston, MA
    5 days ago
  •  ...methods or transformer-only architectures, combining cutting-edge research with real-world deployment. Our solutions are already deployed...  ...driven by real field use. We are seeking a Research Engineer to join our manipulation efforts. In this role, you will work... 
    Full time

    field-ai

    Boston, MA
    5 days ago
  • $100.3k

     ...innovative and inclusive community. We're looking for a Research Computing Systems Engineer to help operate and grow the high-performance computing (...  ...working on systems end to end — from writing infrastructure as code, to troubleshooting hardware, to planning physical rack... 
    For contractors
    Work experience placement
    Work at office
    Remote work

    Boston University

    Boston, MA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Code Quality Specialist / Research Engineer. Be the first to apply!