Senior Software Engineer for AI Model Evaluation

Apply deep engineering judgment to help frontier AI models reason more accurately about real-world software development and engineered systems. You will work closely with an AI research and program management team to define high-quality engineering work, evaluate model performance, and turn expert practice into clear standards and benchmarks. Key Responsibilities

• Review engineering knowledge-work tasks and model outputs, identifying missing behaviors, weak reasoning, subtly incorrect code, unaddressed edge cases, and responses that would not hold up in a real code review.
• Create detailed instruction specifications, golden solutions, and new engineering tasks grounded in practical engineering work.
• Design demanding engineering benchmarks and evaluation sets that measure meaningful model improvement.
• Help develop engineering-specific skills and tools with the research team.
• Collaborate with researchers and specialists in adjacent disciplines to calibrate standards and translate tacit engineering judgment into explicit, teachable criteria.

• Bachelor''s degree or higher in computer science, software engineering, or a closely related engineering discipline. An MS or PhD is strongly preferred.
• At least 4 years of substantive professional engineering experience building and shipping production software or engineered systems at a reputable organization. Internships and academic coursework alone do not qualify.
• Specialized expertise in at least one area, such as distributed systems and backend infrastructure, security and cryptography, embedded and firmware, machine learning systems, developer tooling and platform engineering, or quality and test engineering.
• Demonstrated progression to a senior role, such as Senior, Staff, Principal, Architect, or Engineering Lead, with ownership of systems and design decisions.
• Preferred evidence of engineering impact, including shipped systems, granted patents, open-source contributions, or published technical work.
• Hands-on professional use of large language models and the ability to distinguish sound reasoning from plausible but incorrect answers.
• Excellent written communication skills and the ability to provide precise, well-structured feedback.

• Full-time W-2 employment, with placement on an AI lab''s extended workforce and collaboration within its internal tools and research environment.
• Client-issued accounts and equipment will be provided.
• Commitment of 40 hours per week for an initial 6-month engagement.
• Hybrid role based in the Bay Area, California, with on-site work alongside the client team multiple days per week as required. This is not a remote position.
• Applicants must live in the Bay Area or relocate there at their own expense before the engagement begins. Relocation assistance is not available.

$65 to $105 per hour.

This is a structured, role-based W-2 position, not a freelance or project-based engagement. Candidates may discover the opportunity through an online job platform; employment, onboarding, payroll, benefits, and compliance are administered by the employer of record. Equal employment opportunity is provided without discrimination based on legally protected characteristics, and reasonable accommodations are available for qualified applicants with disabilities and disabled veterans during the application process.