Technical Staff Member, Coding Agent Evaluation

To advance the evaluation and development of frontier coding agents, the full-time remote Member of Technical Staff, Coding Agent Evaluation will design benchmarks, lead research initiatives, and develop datasets to measure and improve coding model performance. Key responsibilities Design and manage evaluation frameworks for coding agents, including benchmarks and scoring methodologies Lead research initiatives to assess and enhance coding model performance across various software engineering tasks Analyze model behavior and failure modes to identify weaknesses and implement actionable improvements Required qualifications Strong software engineering background with expertise in Python, C++, or similar programming languages 3+ years of experience in software engineering, machine learning, AI research, or related technical fields Experience in designing or validating technical assessments and evaluation methodologies Familiarity with large language models, coding agents, and model evaluation systems Proven ability to build tooling and automate workflows for large-scale experimentation