Machine Learning Engineer for Model Evaluation

Author and run rigorous, multi-step machine learning evaluation tasks for a leading generative AI research team. You will take high-level research ideas, implement changes to training or evaluation procedures, execute experiments, and analyze results to determine where frontier models succeed or fail. Typical tasks require one to two days of continuous, focused work and combine implementation, experiment execution, and careful analysis. You will collaborate closely with the lab''s researchers in a fast feedback loop.

• Design tasks that translate real ML research ideas into well-defined, multi-step experiments, for example modifying how an RL reward is computed and specifying success criteria.
• Implement changes to code and training pipelines required by each task.
• Set up, run, and monitor training experiments end-to-end, ensuring reproducibility and correctness.
• Analyze experiment outputs to demonstrate what a correct solution looks like and to identify model failure modes.
• Build tasks that explore reinforcement learning concepts such as reward functions and training behavior.
• Evaluate how frontier models handle your tasks, documenting where and why they fall short.
• Share findings and align task design, rigour, and fairness with researchers and fellow experts.

• MSc or PhD in machine learning, computer science, or another STEM field, or equivalent practical experience in a research-heavy role.
• At least 1 year of experience in a research or research-engineering position.
• Hands-on experience training and evaluating ML models and running experiments end-to-end, including setup, execution, and analysis.
• Strong familiarity with large language models, including their capabilities, limitations, and common evaluation techniques.
• Working proficiency in Python and Git, comfortable in both scripting and notebook environments.
• Basic understanding of reinforcement learning concepts such as reward functions and policy training is preferred.
• Prior experience in AI training, model evaluation, or benchmark or task authoring is preferred.
• High attention to detail, creative task design, strong written communication, and the ability to work independently on ambiguous, open-ended problems.
• Availability to engage reliably for approximately 35 hours per week.

• Employment type: W-2 employment through Cincinnatus LLC, placed to work as part of a leading AI lab''s extended workforce.
• Work location: fully remote within the United States.
• Time commitment: approximately 35 hours per week, full-time role-based assignment integrated into the client team''s workflows.
• Role structure: this is a structured, role-based position rather than a project-based or freelance engagement; it involves close collaboration with the client team.

• Hourly W-2 pay range: $60.00 to $90.00 per hour.

• Cincinnatus LLC serves as the employer of record and administers employment, onboarding, payroll, and benefits for this role.
• To apply, submit your application through the job listing or application portal; applications and subsequent employment administration will be handled by Cincinnatus LLC.
• Equal Employment Opportunity: Cincinnatus is an equal opportunity employer and provides reasonable accommodations for qualified applicants with disabilities.