Retail Subject Matter Expert for AI Model Evaluation

Provide deep retail subject-matter expertise to improve and validate large language models, by creating realistic retail tasks, producing model-grounded solutions, and judging model outputs against structured rubrics. You will work closely with research and engineering teams to close domain knowledge gaps in merchandising, category management, and retail operations, while being employed by Cincinnatus LLC and placed with a leading AI lab.

• Guide research and engineering teams to close knowledge gaps in retail merchandising, category management, and operations reasoning.
• Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
• Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
• Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
• Collaborate with other subject matter experts to ensure consistency and accuracy in training data.

• Minimum 8 years of professional experience in retail functions such as merchandising, category management, retail operations, or buying and planning, at a recognized, top-tier organization (examples: Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
• Prior hands-on experience evaluating LLM or other AI model outputs against rubrics or structured scoring criteria, this is mandatory. Please describe this experience in your application.
• Demonstrable career progression, for example Category Manager to Senior Manager to Director of Merchandising.
• Strong verbal and written communication skills, solid problem-solving ability, and effective interpersonal skills.

• Employment type: W-2 employee of Cincinnatus LLC, placed to work with a leading AI lab as part of their extended workforce.
• Location: United States.
• Schedule commitment: Able to engage reliably for at least 35 hours per week, during weekdays.

Hourly pay range: 60 - 80 hourly.

• This is a W-2 employment position with Cincinnatus LLC, with placement on client teams at a leading AI lab.
• Cincinnatus LLC is an Equal Employment Opportunity employer and does not discriminate on the basis of any legally protected characteristic.

Submit your resume and include a clear description of your hands-on experience evaluating LLM or AI model outputs against rubrics, and examples that demonstrate your retail domain progression. Applications that do not describe prior LLM/AI evaluation experience may not be considered.