Code Quality Specialist / Research Engineer
$75 - $105 per hourBraintrust
About the Opportunity
Our client, a leading AI research organization, is looking for engineers and computational researchers to help evaluate how AI systems solve coding tasks.
You’ll investigate whether an unsuccessful result comes from the AI’s code, an issue in the tests, or unclear task requirements. This work combines code reading, debugging and technical judgment to help make AI evaluations more accurate and reliable.
The role is suited to people who enjoy understanding unfamiliar codebases, examining test behavior and explaining technical findings clearly.
What You’ll Do
- Review code, tests, execution logs and the steps an AI coding agent took while attempting a task.
- Investigate reported problems in coding benchmarks, including undocumented expectations, missing definitions and overly restrictive test assertions.
- Determine whether a reported issue is a genuine flaw, a gap that still permits reasonable solutions to pass, or an error in the AI’s implementation.
- Assess whether an identified issue affected the task’s pass/fail result.
- Write concise technical explanations supported by relevant excerpts from task instructions, tests and run logs.
- Diagnose test-harness and environment issues using Docker configuration, dependencies and execution logs.
- Participate in daily calibration discussions with research scientists to compare findings and align evaluation decisions.
Individual claim investigations are expected to take approximately 30–60 minutes. You’ll use review tools and supporting technical resources to examine the evidence and document a clear conclusion.
What We’re Looking For
- Either 2+ years of professional experience writing, reviewing and debugging software , or a completed STEM master’s degree or PhD with intensive programming and computational research experience.
- Strong Python code-reading, debugging and comprehension skills, including familiarity with complex frameworks and test assertions.
- Reading familiarity with at least one additional language: JavaScript/TypeScript, Go, C++ or Rust.
- Practical experience with Git and Docker.
- Familiarity with unit-testing frameworks, such as pytest, unittest, Jest or Mocha, and software build environments.
- Careful attention to edge cases, ambiguous requirements and discrepancies in test logic.
- Fluent professional English, with the ability to communicate technical findings clearly in writing and conversation.
Nice to Have
- Experience with LLM benchmarking, AI evaluation, prompt engineering or coding-agent workflows.
- Familiarity with benchmarks such as SWE-bench or CodeContests.
- Experience in AI red teaming, safety or reliability auditing.
- Previous collaboration with AI or machine-learning research scientists.
Engagement Details
- Location: United States only.
- Expected commitment: 35 hours per week.
- Start: Immediate.
Compensation
$75–$105 USD per hour, depending on relevant skills and experience.
Interview Process
Shortlisted candidates will be invited to a 15-minute AI interview. The hiring team will then review your profile and interview responses to determine next steps, with no additional interview rounds required.
- ...Research Engineer, Code RL (Reinforcement Learning) About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable... ...engineering excellence, with a deep commitment to building high-quality, scalable systems that push the boundaries of what AI can...SuggestedFull timeWork at officeVisa sponsorshipFlexible hours
- ...company based in San Francisco, California. The Role: As a Research Engineer - Brain Computer Interface Models , you will be a core... ...communication and collaboration skills, with clean and concise coding practices Strong implementation and prototyping skills, with...SuggestedWork at officeRelocation package
$122k - $215k
...the world in a positive way. To learn more visit: As a Research Engineer, you will be at the forefront of advancing and deploying artificial... ..., and a commitment to rigorous experimental validation and code quality as we strive to bring research ideas into production and...SuggestedFull timeWork at officeWork from homeFlexible hours- ...Description Bridge the gap between cutting-edge research and production systems. Build and... ...understanding Optimize research code for production deployment Design and... ...Bridge communication between research and engineering teams What We're Looking For ~3+ years...Suggested
$125k - $400k
...idler is a frontier data research lab. We build the evals and environments... ...the last year developing coding evals for top coding models... ...the role As a Research Engineer at idler, you’ll build the systems... ...to validate environment quality and fairness Work directly...SuggestedWork at officeRelocation package$200k - $375k
...understands complex enterprise systems, the tooling experience developers use every day, and the analysis and planning engine that transforms legacy code. You will work directly with customers on high-leverage projects and help define our culture, our engineering bar,...Visa sponsorship- ...starting with data centers. About the Role At Watney, research engineers decide what our models learn to do next, and a live fleet of... ...you are one of the people deciding it. Write production ML code (Python, PyTorch, etc.). We’re committed to building a diverse...
$134k - $235k
...positive way. To learn more visit: As a Research Engineer in Neural Rendering, you will create... ...in reading and developing production quality software, versus only creating prototypes... ...You write efficient and maintainable code in Python and PyTorch, as well as in at...Full timeWork at officeWork from homeFlexible hours- ...months, we’ve gone from our first line of code to deploying real humanoid capabilities. The role As our VLA/WAM Research Engineer, you'll develop the foundation models... ...learning, RL, reward/value-guided learning, and specialist-to-generalist distillation. Continual...Work at officeRelocation
- ...Research EngineerWe believe that software is the foundation of modern civilization - yet... ...Infrastructure.We're seeking an experienced Research Engineer to join our effort in building and... ...learning environments to train security coding agentsWork on a Product that Solves a...Full timeWork at office
$9.7k - $19k
...Research Engineer InternThe Center for AI Safety (CAIS) is a leading research and advocacy organization focused on mitigating societal-scale... ...defend their impact. You will plan and run experiments, conduct code reviews, and work in a small team to create a publication...Full timeInternshipLocal area- ...Research EngineerNew York - Hybrid; San Francisco Bay Area - HybridAbout... ...clients. As a Research Engineer, you'll work on how frontier... ...design the approach and write the code that proves it out.Depending... ...We NeedProduction-quality code written daily; this is a...Full timeWork at officeLocal areaRemote work
- ...Greptile is an AI code reviewer that catches bugs and anti-patterns in pull requests with complete context of the codebase. Hundreds... ...prototypes into production systems Stay on top of the latest research across LLMs, information retrieval, and developer tooling Qualifications...Work at office
- ...Research EngineerSesame believes in a future where computers are lifelike... ...product goals. As a Research engineer at Sesame, you are a research... ...and live evals — surface quality regressions before users or... ...comfortable writing production code across the stack.Hands-on...Full timeContract workShift work
- ...Outset invented a better way to do research: AI-powered user interviews. Global brands... ...sectional levels of analysis. The Research Engineer reports to the Head Scientist and builds... ...from stored inputs, and that research code stays in step with the shipped product. The...
- ...fleet of affordable, durable robots. We're looking for a research engineer to help us train increasingly capable models across... ...system is doing Are comfortable moving between research code and production-quality systems Like running experiments, getting surprising...
$250k - $400k
...frontier AI labs to generate high-quality datasets, reinforcement... ...learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work,... ...for Software Engineering / coding agents # UI-Environments...$200k - $275k
...positive way. To learn more visit: As a Research Engineer focused on Generative AI Deployment,... ...research to production. Turn research code and checkpoints into versioned,... ...experience reading and developing production-quality software at scale, not only prototypes....Full timeWork at officeWork from homeFlexible hours- ...We are seeking a highly technical Forward Deployed AI Engineer – Agents & Research Systems to build and deploy advanced agent systems for large... ...solutions. Ability to personally write and own production code. Strong understanding of evaluation, agent behavior, and...
$315k
...Research Engineer, Interpretability San Francisco, CA About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable... ...candidates may also have experience with: Designing a code base so that anyone can quickly code experiments, launch them...Full timeTemporary workWork at officeRemote workVisa sponsorshipFlexible hours- ...Job Description Job Description Research Engineer, Post-Training Data Location: San Fran... ...opportunity to join a team building high-quality post-training data, reinforcement... ...Debug ML pipelines, understand unfamiliar code quickly, and solve open-ended technical...Full time
- ...and we're seeking the team of researchers that will build them.... ...We're looking for a Research Engineer to sit between our data and our... ...from, and write the research code (libraries, dataloaders, evaluation... ...Comprehensive, high-quality health, dental, and vision insurance...Full timeWork at officeRelocation package
- ...to the work of semiconductor engineering. Our goal is to help... ...design cycles. This work brings research, model training, and hardware... ...reproducible. We value strong coding fundamentals, careful experimental... ...the team’s hardware specialists. In this role, you will:...Full time
- ...about as well. If you want to have your research come into contact with reality, Ando is the... ...are not Ando's moat; execution, product quality, and customer experience are. Research here... ...over credentials. Work samples or code that demonstrate the skills: eval frameworks...Full timeWork from home
- ...is changing that. Tutor is a robotics research and deployment company, building generally... ...States. We’re a team of researchers, engineers, technicians, operators, and sellers who... ...Strong Python, and maintainable, tested code that runs on real hardware. Hands-on experience...Hourly payFull timeWork at office
- ...way amounts to creating AGI. The economic engine we’re creating on the e-commerce side,... ...to Have significant software, ML, or research engineering experience Command a deep,... ...efforts. Quickly and iteratively ship code that is correct, robust, scalable, and maintainable...Full timeWork at officeLocal areaRemote workRelocationShift work3 days per week
- ...for long-horizon tasks like autonomous research, and the full-stack platform our own research... ...learning. We’re hiring research engineers and scientists across RL algorithms, scaling... ...Develop and test RL methods for reasoning, coding, and long-horizon tasks, including...Full timeWork at officeRemote workVisa sponsorshipRelocation packageFlexible hours
- ...Devin, the first AI software engineer. Our team is extremely talent... ...join us. Role Mission Research moves at the speed of the... ...hundreds of thousands of concurrent coding agent rollouts in VM... ...and evaluation. Ensure data quality, reproducibility, and efficiency...Full timeLocal area
$175k - $260k
...the role Were looking for a Senior Research Infrastructure Engineer to accelerate research on training... ...in experimentation and turn research code into reliable reusable systems. Youll... ...behavior agent failure modes and data quality issues Make experimentation faster...Full timeWork at officeRemote workRelocationVisa sponsorship- ...build. We look for world-class engineers who have both a vision for... ...down and produce exceptional code. Responsibilities Develop... ...Stay current with relevant research and apply it pragmatically... ...waiting for at the speed and quality the market demands (or if you...Full timeFor contractors
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Code Quality Specialist / Research Engineer. Be the first to apply!
- quality review specialist San Francisco, CA
- quality assurance auditor San Francisco, CA
- quality control associate San Francisco, CA
- quality specialist San Francisco, CA
- warehouse quality control associate San Francisco, CA
- quality management specialist San Francisco, CA
- quality system auditor San Francisco, CA
- quality systems specialist San Francisco, CA
- quality associate San Francisco, CA
- quality improvement specialist San Francisco, CA





