Senior Software Engineer for AI Model Evaluation

Help advance frontier AI models by applying senior-level engineering judgment to real software and engineering work. You will work closely with an AI lab’s research and program management teams to improve how models reason about engineering tasks, define correct solutions, and demonstrate measurable progress.

This role is designed for a practicing specialist with deep experience in a technical domain, not a generalist. You will work within the client’s tools using client-issued accounts and equipment alongside research teams.

• Review engineering knowledge-work tasks and model outputs, identifying missing behaviors, weak reasoning, subtle code defects, unhandled edge cases, and responses that would not meet real code-review standards.
• Create detailed instruction specifications, golden solutions, and realistic engineering tasks that reflect professional practice.
• Design challenging engineering evaluations and benchmarks, and help develop engineering-specific skills and tools with the research team.
• Collaborate with researchers and adjacent-domain specialists to calibrate standards and turn tacit engineering judgment into clear, teachable evaluation criteria.

• Bachelor’s degree or higher in computer science, software engineering, or a closely related engineering discipline. An MS or PhD is strongly preferred.
• At least 4 years of substantive professional engineering experience building and shipping production software or engineered systems at a reputable organization. Internships and academic coursework alone do not qualify.
• Specialization in at least one area, such as distributed systems and backend infrastructure, security and cryptography, embedded systems and firmware, machine learning systems, developer tooling and platform engineering, or quality and test engineering.
• Progression to a senior-level role, such as Senior, Staff, Principal, Architect, or Engineering Lead, with ownership of systems and design decisions.
• Hands-on professional use of large language models and the ability to distinguish sound reasoning from plausible but incorrect answers.
• Excellent written communication and the ability to provide precise, well-structured feedback.
• A record of engineering output, such as shipped systems, granted patents, open-source contributions, or published technical work, is preferred.

• Full-time W-2 employment, with a reliable commitment of 40 hours per week for an initial 6-month engagement.
• Hybrid role based in the Bay Area, California. On-site work with the client team is required multiple days per week as needed; this is not a remote role.
• You must live in the Bay Area before the engagement begins, or relocate there at your own expense. Relocation assistance is not available.
• This is a structured, role-based position integrated with the client’s internal team and enterprise workflows, not a freelance engagement.

$65 to $105 per hour.

Applications may be submitted through the opportunity platform. If selected, employment, onboarding, payroll, benefits, and compliance are administered by the employer of record.

Qualified applicants are considered without regard to legally protected characteristics. Reasonable accommodations are available for qualified individuals with disabilities and disabled veterans throughout the application process.