part-time Senior LLM Evaluation / Agent Reliability 2-5hrAdvisor

Position Summary Shopint is looking for a part‑time Senior LLM Evaluation / Agent Reliability Advisor. We are building a focused evaluation product for commerce‑related AI agents, with early case evidence and feedback from principal‑level AI evaluation, reliability, and safety leaders. We are looking for someone to pressure‑test and refine our evaluation approach as we prepare our first pilot‑ready custom diagnostic package. Ideal Background Senior Applied Scientist, LLM Eval Lead, Agent Evaluation Lead, AI Quality Lead, or similar Experience with LLM evaluation, agent reliability, RAG/tool‑use evals, rubric design, human evaluation, or regression testing Comfortable turning messy early‑stage cases into serious eval / regression assets Key Responsibilities Review rubric quality and case structure Pressure‑test logic Review pilot‑ready case cards and diagnostic outputs Other Details Time: 2–5 hours/week to start. Compensation: flexible — paid advisory, equity, part‑time contributor role, or future core/founding role depending on fit and commitment. #J-18808-Ljbffr Shopint