Machine Learning and NLP Practitioner for AI Model Evaluation

Join a cutting-edge GenAI team at a leading AI lab, where your expertise will be pivotal in developing advanced AI models. This role focuses on designing and evaluating machine learning and natural language processing tasks that will help identify and address capability gaps in frontier AI models. Key Responsibilities

• Task design and development: Create challenging, real-world ML and NLP problems from your area of expertise, targeting specific capability gaps in a frontier AI model.
• Spec and golden-solution generation: Prepare all necessary components for the problems in an agentic development environment using Python.
• Evaluation and analysis: Assess the target model''s performance on your tasks.
• Headroom identification: Identify and classify tasks where the target model fails.
• Collaborate with other experts: Work with fellow subject-matter experts to ensure consistent and accurate evaluations.

• Deep, hands-on experience in machine learning and/or natural language processing, gained through applied industry work, research, or a graduate/PhD background.
• Working proficiency in Python, applied in research, industry, or open-source projects.
• Strong command of modern ML/NLP methods, including model training and evaluation, transformers, large language models, and standard tooling.
• Availability to engage for approximately 20 hours per week.
• Preferred experience in AI training, model evaluation, or data annotation.
• Strong written communication skills and the ability to work independently while managing your own time.

This is a part-time W-2 employment position with Cincinnatus LLC, offering the opportunity to work remotely within the United States.

Hourly compensation ranges from $80 to $110.

This role is open to candidates located in the United States.