UI/UX Designer for AI Evaluation

Help define the standard for excellent digital product design by evaluating completed UI/UX work samples and creating rigorous criteria for assessing AI performance on real-world product design tasks. This role focuses on clear, defensible evaluation and written design critique rather than producing design deliverables yourself. Key Responsibilities

• Create precise, task-specific grading criteria for digital product and UX deliverables, including design systems and component libraries, responsive interfaces and dashboards, onboarding, checkout and booking flows, logged-in customer portals, UX writing, product content, and design-to-engineering handoff.
• Evaluate AI-generated and human work samples against established criteria, providing detailed written justification for every score.
• Apply consistent, evidence-based judgment so evaluations are reproducible and defensible.
• Incorporate structured feedback from senior reviewers and revise work quickly.

• At least 5 years of professional experience in product design, UI/UX, or design systems for real digital products, rather than brand or marketing websites.
• Experience with leading digital product or UX studios, in-house product design teams at major product-led companies, or e-commerce and marketplace businesses with sophisticated customer portals and dashboards.
• Experience in one or more relevant roles, such as Product Designer, UI Designer, Design Systems Designer, UX Lead, UX Writer, Content Designer, digital Product Analyst, or Business Analyst.
• Strong command of interaction and visual design, design systems, responsive layouts, end-to-end product flows, Figma prototyping and production handoff, and design-review-quality critique.
• Exceptional written communication and the ability to explain specifically why work does or does not meet the needs of a real product and its users.
• Detail-oriented, consistent, and comfortable having evaluation judgment reviewed and calibrated with peers.
• Experience with AI training, evaluation, or human-data projects is a strong plus.

• Remote, hourly engagement.

• $80 to $150 per hour.

• Submit a resume, portfolio, or relevant professional background for consideration.
• Qualified applicants may be asked to complete a brief assessment or provide additional information.