Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Code Quality Specialist / Research Engineer

$75 - $105 per hour

Braintrust

About the Opportunity

Our client, a leading AI research organization, is looking for engineers and computational researchers to help evaluate how AI systems solve coding tasks.

You’ll investigate whether an unsuccessful result comes from the AI’s code, an issue in the tests, or unclear task requirements. This work combines code reading, debugging and technical judgment to help make AI evaluations more accurate and reliable.

The role is suited to people who enjoy understanding unfamiliar codebases, examining test behavior and explaining technical findings clearly.

What You’ll Do

  • Review code, tests, execution logs and the steps an AI coding agent took while attempting a task.
  • Investigate reported problems in coding benchmarks, including undocumented expectations, missing definitions and overly restrictive test assertions.
  • Determine whether a reported issue is a genuine flaw, a gap that still permits reasonable solutions to pass, or an error in the AI’s implementation.
  • Assess whether an identified issue affected the task’s pass/fail result.
  • Write concise technical explanations supported by relevant excerpts from task instructions, tests and run logs.
  • Diagnose test-harness and environment issues using Docker configuration, dependencies and execution logs.
  • Participate in daily calibration discussions with research scientists to compare findings and align evaluation decisions.

Individual claim investigations are expected to take approximately 30–60 minutes. You’ll use review tools and supporting technical resources to examine the evidence and document a clear conclusion.

What We’re Looking For

  • Either 2+ years of professional experience writing, reviewing and debugging software , or a completed STEM master’s degree or PhD with intensive programming and computational research experience.
  • Strong Python code-reading, debugging and comprehension skills, including familiarity with complex frameworks and test assertions.
  • Reading familiarity with at least one additional language: JavaScript/TypeScript, Go, C++ or Rust.
  • Practical experience with Git and Docker.
  • Familiarity with unit-testing frameworks, such as pytest, unittest, Jest or Mocha, and software build environments.
  • Careful attention to edge cases, ambiguous requirements and discrepancies in test logic.
  • Fluent professional English, with the ability to communicate technical findings clearly in writing and conversation.

Nice to Have

  • Experience with LLM benchmarking, AI evaluation, prompt engineering or coding-agent workflows.
  • Familiarity with benchmarks such as SWE-bench or CodeContests.
  • Experience in AI red teaming, safety or reliability auditing.
  • Previous collaboration with AI or machine-learning research scientists.

Engagement Details

  • Location: United States only.
  • Expected commitment: 35 hours per week.
  • Start: Immediate.

Compensation

$75–$105 USD per hour, depending on relevant skills and experience.

Interview Process

Shortlisted candidates will be invited to a 15-minute AI interview. The hiring team will then review your profile and interview responses to determine next steps, with no additional interview rounds required.

Vacancy posted 10 hours ago
Similar jobs that could be interesting for youBased on the Code Quality Specialist / Research Engineer in Bay County, FL vacancy
  •  ...Invisible is an Applied AI research company closing the gap between...  ...enterprise clients. As a Research Engineer, you'll work on how frontier...  ...the approach and write the code that proves it out....  ...What We Need Production-quality code written daily; this is a... 
    Suggested
    Full time
    Work at office
    Local area
    Remote work

    Invisible Technologies

    Bay County, FL
    1 day ago
  •  ...Merge Labs is a frontier research lab with the mission of bridging biological and artificial...  ...a cross-functional data and software engineering group which supports the intersection of...  ...scientists and establishing best practices for code hygiene, testing, and reproducibility.... 
    Suggested
    Full time

    Merge%20Labs

    Bay County, FL
    4 days ago
  • $150k - $250k

     ...real client portfolios at scale, and the quality of our investment outcomes depends on a...  .... We're looking for a senior quant research engineer to help design the models that determine...  ...production-quality systems. Use AI coding tools as part of your day-to-day workflow... 
    Suggested
    Remote job
    Full time
    Work at office
    3 days per week

    artafinance

    Bay County, FL
    4 days ago
  • $130k - $190k

     ...Senior Research Engineer, Advanced Technology Group About the Company In the coming decades...  ...safety, construction, and operations specialists. By combining industry knowledge and...  ...instrumentation that produces rich, high-quality datasets, feeding the machine learning... 
    Suggested
    Full time
    Contract work
    For contractors
    Work at office
    Remote work

    KoBold Metals

    Bay County, FL
    1 day ago
  •  ...Quality Assurance Specialist/Loan Processor We are seeking a strong-minded individual to join our team as a Quality Assurance Specialist/Loan Processor. This role is positioned at the end of our sales process and is essential in assisting with closing documents as customers... 
    Suggested
    Contract work

    Hilton Grand Vacations

    Panama City Beach, FL
    12 hours ago
  •  ...Siemens Energy is seeking a Logistics Engineering Specialist in Tampa, FL. This specialist role is...  ...Collaborate with manufacturing and quality teams to ensure logistics capacity aligns...  ...sustainably. The technologies created in our research departments and factories drive the... 
    Full time
    Work at office
    Visa sponsorship

    Siemens Energy

    Bay County, FL
    4 days ago
  • $60 - $70 per hour

     ...PCB Design Engineer $60 - $70 per hour (W2) 3 – 6 Month Contract + fully remote...  ...in a senior-level position, supporting a specialist manufacturer designing and delivering solutions...  ..., known for its values-led approach and quality-first engineering, and they are... 
    Hourly pay
    Full time
    Contract work
    Remote work
    Monday to Friday

    Rise Technical

    Bay County, FL
    12 hours ago
  •  ...ML/AI Research Engineer — Agentic AI Lab (Founding Team) Location: San Francisco Bay Area Type: Full-Time Compensation: Competitive salary + meaningful equity (founding tier) Backed by 8VC, we're building a world-class team to tackle one of the industry... 
    Full time

    fabrion

    Bay County, FL
    4 days ago
  •  .... Our small, elite team of founders and engineers have previously launched and exited successful...  ...stays reliable. We ship quickly with coding agents. You'll enhance the automated QA,...  ...safeguards, and PR review that keep quality up as we go faster. This is a hybrid... 
    Remote job
    Full time
    Visa sponsorship
    Relocation package
    Flexible hours
    3 days per week

    develop-health

    Bay County, FL
    4 days ago
  •  ...Twenty Labs is an applied research lab working on augmenting knowledge...  ...leading researchers and engineers from Google DeepMind/Brain and...  ...directly with the founders who all code. No empire building – we...  ...off as you need it ~ Top-quality computers and ergonomic... 
    Full time
    Work at office
    Local area

    compound%20jobs

    Bay County, FL
    4 days ago
  • $185k

     ...generation of products for the AI-native era — and we're hiring product engineers to build them. Some ship in the open to hundreds of thousands...  ...-stack, product-minded engineer who owns products, not just code — no PMs, no designers, you make the product and UX calls... 
    Full time
    Live in
    Local area
    Relocation
    Visa sponsorship
    Work visa
    Flexible hours

    ycombinator

    Bay County, FL
    4 days ago
  • $180k - $245k

     ...computing. We are looking for a Senior Design Engineer to design, integrate, and sign off RTL...  ...& Sign-off Own block-level front-end quality: lint-clean RTL, CDC and RDC analysis...  ...scan insertion, ATPG, and DFT-friendly RTL coding. Scripting skills (Python, Tcl, Perl)... 
    Night shift

    Efficient Computer

    Bay County, FL
    1 day ago
  •  ...About Attention Engineering The gap between how top engineers around the world use AI systems today - elaborate coding agent configs, massive context files, and engineered workflows...  ...We’re a small team of builders/researchers. We’ve worked at companies like Citadel... 
    Full time

    attention

    Bay County, FL
    4 days ago
  • $250k

     ...Y Combinator is hiring a full-stack product engineer to work at the center of how YC operates — from the application pipeline to events...  ...stack, product-minded engineer who enjoys owning products, not just code Love using AI in your work -- your dev stack is fully AI-... 
    Full time
    Work at office
    Local area
    Relocation
    Visa sponsorship
    Work visa
    Flexible hours

    ycombinator

    Bay County, FL
    4 days ago
  •  ...previously co-founded a unicorn), and AI researchers from top institutions who have created some of the most popular open-source code generation projects. About the role:...  ...products - as a product manager, designer, engineer, or founder. A traditional engineering background... 
    Full time
    Work at office
    Local area
    Flexible hours
    Day shift

    Coframe

    Bay County, FL
    4 days ago
  • $223k - $359k

     ...re looking for. The Role Product Engineers at Hadrian are software engineers who own...  .... You'll spend most of your time writing code, primarily in React, Next.js, TypeScript,...  ...behind the request. ~ Ship fast, keep quality high, and feed the platform. Get working... 
    Permanent employment
    Full time
    Work at office
    Relocation package
    Flexible hours

    hadrian-automation

    Bay County, FL
    4 days ago
  • $100k - $200k

     ...creating AI films. Founded by Ray Wang (engineer who built Instagram Stories) and Zoey Zhang...  ...ambiguous ideas into intuitive, high-quality user experiences • Continuously refine...  ...experience • Open-source contributions and coding outside of regular work Compensation... 
    Remote job
    Full time
    Visa sponsorship

    embedding-vc

    Bay County, FL
    4 days ago
  •  ...delays and inefficiencies. Our small, elite team of founders and engineers have previously launched and exited successful healthcare...  ...that do real work in healthcare, and still write a lot of the code. You'll lead projects that span systems, from the experience prescribers... 
    Remote job
    Full time
    Visa sponsorship
    Relocation package
    Flexible hours
    3 days per week

    develop-health

    Bay County, FL
    4 days ago
  • $160k - $200k

     ...The Role We are looking for a Senior Product Software Engineer to join our Engineering team and lead key initiatives that...  ...emphasis on building scalable systems and delivering high-quality, maintainable code. Our Product Tyba has two main products for customers... 
    Remote job
    Full time
    Flexible hours

    tyba

    Bay County, FL
    4 days ago
  • $250k

     ...re looking for a Software Product Design Engineer — someone who lives at the intersection...  ...translate high-fidelity mocks into production code, and proactively elevate existing...  ...compelling, clear UI/UX and produce high-quality mocks that demonstrate strong product taste... 
    Full time
    Live in
    Local area
    Relocation
    Visa sponsorship
    Work visa
    Relocation package
    Flexible hours

    ycombinator

    Bay County, FL
    4 days ago
  •  ...the real world. Founded by researchers from UC Berkeley's SkyLab, we...  ...in real workflows: agentic coding, creative generation, professional...  ...take on specific customer engineering asks and ship solutions that...  ...run evals, and deliver high-quality results Drive consistent... 
    Full time
    Work at office

    arena

    Bay County, FL
    4 days ago
  • $200k - $275k

     ...generation of computing. We are looking for a Lead RTL Design Engineer to own microarchitecture definition and RTL implementation...  ...engineers; review RTL, flag microarchitecture risks, and enforce coding style and lint-clean standards across the team. Driving PPA... 
    Night shift

    Efficient Computer

    Bay County, FL
    1 day ago
  •  ...York. About the role As a Software Engineer working on AI systems, you will play a foundational role in research, experimentation and rapid improvement of AI...  ...For Proven track record of shipping high-quality code in challenging projects Bachelor's or Master... 
    Full time
    Work at office
    Work from home
    Worldwide
    Monday to Friday

    luminai

    Bay County, FL
    4 days ago
  •  ...the real world. Founded by researchers from UC Berkeley's SkyLab, we...  ...in real workflows: agentic coding, creative generation, professional...  ...take on specific customer engineering asks and ship solutions that...  ...run evals, and deliver high-quality results Drive consistent... 
    Full time
    Work at office

    arena

    Bay County, FL
    4 days ago
  • $269.18k - $376.84k

     ...Principal Modem PHY IP Design Verification Engineer to lead and execute verification of...  ...verification environments, and ensuring high-quality modem PHY IP delivery for next-generation...  ...models. Define and track functional, code, assertion, and scenario coverage to ensure... 
    Permanent employment
    Full time
    Temporary work
    Local area
    Worldwide

    Blue Origin

    Bay County, FL
    4 days ago
  • $250k

     ...Combinator is looking for a full-stack product engineer who also wants to apply their skills to...  ...much of your time writing production code — building internal and founder-facing tools...  ...KPIs, and conducting market or customer research Build and maintain analytical tools,... 
    Full time
    Local area
    Relocation
    Visa sponsorship
    Work visa
    Flexible hours
    Shift work

    ycombinator

    Bay County, FL
    4 days ago
  •  ...We are hiring a Applied AI Engineer for a leading fintech company...  ...lead two primary streams: AI Coding tooling and developer...  ...personally deliver 5–10 reference-quality Skills as exemplars; raise team...  ...Track record of translating research output (papers, blog posts) into... 
    Full time

    embedding-vc

    Bay County, FL
    4 days ago
  •  ...publishing video on the internet. We are looking for exceptional research engineers and applied researchers to help push the frontier of...  ...embodied AI systems. Use AI-native engineering tools (Claude Code, Codex, or similar) to ship faster. What We're Looking For... 
    Full time

    embedding-vc

    Bay County, FL
    4 days ago
  • $180k - $325k

     ...The Role As an AI Product Engineer at Brain Co., you will play a...  ...back-end systems, you will turn research breakthroughs into practical solutions...  ...and AI. Participate in code reviews, share knowledge, and set an example with high-quality engineering practices. Mentor... 
    Remote job
    Full time
    Local area
    Worldwide

    brainco

    Bay County, FL
    4 days ago
  • $250k

     ...feedback from both other YC alumni and the broader public.  About the team Our whole software team is only 15 full-stack product engineers, and we enjoy working in a small team with high impact and knowing each other by name. We have a broad range of experiences from... 
    Full time
    Live in
    Work at office
    Local area
    Relocation
    Visa sponsorship
    Work visa
    Relocation package
    Flexible hours

    ycombinator

    Bay County, FL
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Code Quality Specialist / Research Engineer. Be the first to apply!