Product Designer - AI Creative Evaluation (Contra Labs)
Contra Labs
About Contra Labs
Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our research is grounded in how top creatives actually work, studying the patterns behind what gets created, what gets chosen, and what resonates. We believe human input will always be at the center of net new creation, and our mission is to help shape technology so human creativity can be amplified to create what today is unimaginable.
By joining the Contra Labs network you gain access to paid projects with some of the most innovative companies building the next generation of creative and AI-powered tools.
How It Works
This is an application to join the network and be eligible for roles as they arise. Projects are staffed on an as-needed basis, typically 5 to 10 hours at a time, though some may extend into longer engagements. In unique cases, we may have full-time roles available as well. When a project matches your skill set, we'll reach out with details and timeline.
To Join the Network
- Portfolio review - We'll assess your Contra profile and portfolio for strength in product design, specifically across desktop and mobile applications.
- Record a 30 second intro video - Share a quick 30 second introduction so we can get to know you.
- Complete profile - Once you’ve applied, we recommend completing your profile for better job matches and setting up your wallet so you can get paid when a job comes through.
What You May Work On
- Evaluate AI-generated product design outputs across desktop and mobile app interfaces, applying professional design standards and best practices
- Provide clear, structured feedback on UI/UX quality including layout, hierarchy, interaction patterns, accessibility, and visual polish
- Help define creative benchmarks and taste standards for what great product design looks like in the next generation of AI-powered tools
- Contribute to research on how convergence (shared best practices) and divergence (individual taste and creative direction) shape design quality
What We're Looking For
- Strong portfolio showcasing high-quality product design work across desktop and/or mobile applications, shared via your Contra profile
- Deep understanding of what makes app interfaces intuitive, polished, and well-crafted, from information architecture to micro-interactions
- Ability to articulate design and UX feedback clearly and constructively
- Comfort working in fast-paced, experimental environments (experience with AI or design tools is a plus)
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and technology. We partner with leading product companies to improve the creative tools... ...styles. # Skills evaluation - You’ll complete a... ...videography submission designed to evaluate attention...CreativeFull timeContract work
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our... ...What You May Work On Evaluate AI-generated graphic design...CreativeContract workImmediate start
- ...About Contra Labs Contra Labs sits at the intersection of human creativity and AI. We partner with leading AI labs and product companies to improve their tools across design, video, development, and beyond. Our... ...What You May Work On Evaluate AI-generated advertisements...CreativeFull timeContract work
$230k - $270k
Who We AreHP IQ is HP’s new AI innovation lab. Combining startup agility with HP’s global... ..., world-class team—engineers, designers, researchers, and product minds—focused on creating an intelligent... ..., adaptive solutions that spark creativity, boost productivity, and make...CreativeFull timeTemporary workLocal areaFlexible hours- Mercor connects elite creative and technical talent with leading AI research labs, and seeks a Music & Lyrics Expert to evaluate AI-generated content remotely. You will assess music quality, compare lyrics to published songs, and rate for creativity, prompt adherence,...CreativeRemote jobContract work
- Mercor connects elite creative and technical talent with leading AI research labs. The remote contract role evaluates AI-generated artifacts against domain-specific quality rubrics and provides clear, structured feedback to ensure accuracy and high presentation standards...CreativeRemote jobContract work
- Mercor connects elite creative and technical talent with leading AI research labs. We are hiring a Music Audio Expert - Punjabi for a remote contract up to 6 months. Responsibilities include evaluating AI model outputs for lyrics and voice, scoring training data quality...CreativeRemote jobContract work
$80 - $120 per hour
Mercor connects elite creative and technical talent with leading AI research labs. This contract role evaluates AI-generated artifacts in humanities/arts/culture, delivering clear feedback to improve model outputs. Remote work, English fluency required, 5+ years of domain...CreativeRemote jobContract workWork at office$70 per hour
About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark... ...: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy and depth...CreativeContract workSummer workRemote work- Mercor connects elite creative and technical talent with leading AI research labs. A remote contract role focuses on evaluating AI-generated artifacts against quality rubrics, identifying errors, and delivering structured feedback to support AI model improvements. Applicants...CreativeRemote jobContract workWork at office
- Mercor connects elite creative and technical talent with leading AI research labs. A remote contract role as Customer success / support operations evaluator seeks a highly experienced professional to assess AI outputs against quality rubrics. The candidate will identify...CreativeRemote jobContract work
- YO AI Labs is seeking a Creative & Design Specialist to evaluate AI-generated outputs, provide high-quality human feedback, and improve creative workflows. You will work remotely, collaborating on graphic design, UI/UX, motion graphics, and multimedia tasks across a range...CreativeRemote jobFlexible hours
$60 - $100 per hour
...Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Jack Dorsey . Position: CAD / PCB / 3D Design Engineer — AI Evaluation Type: Contract Compensation: $60–...CreativeContract workSummer workImmediate startRemote work$105 - $140 per unit
...the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Football (Soccer) Fan - Live Match AI Evaluator (UK & Spain) Type: Contract... .... Run questions on two AI products side by side and capture share...CreativeContract workSummer workRemote workFlexible hours$60 - $90 per hour
...the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Learning Engineer — Model Evaluation & Experimentation Type: Contract... ...Role Responsibilities Design tasks by transforming real ML research...CreativeHourly payWeekly payContract workFor contractorsSummer workRemote work$50 - $90 per hour
...the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...Domain Expert — AI Training & Evaluation Type: Contract Compensation... ...reflecting practical work. Design challenging finance tasks and...CreativeContract workSummer work$60 - $70 per hour
...Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy...CreativeContract workSummer workRemote work$147k - $215k
...ourTotal Rewards philosophy.AI is a fundamental part... ...for a AI-native Design Systems Designer to join... ...serves LLMs and AI-assisted products as much as it serves... ...is made up of over 70 creative, collaborative people who... ...directing AI assistance, evaluating its output, and knowing...CreativeFull timeWork at officeLocal area2 days per week3 days per week$189k - $274k
...the AIUX repertoire.Design targeted... ...concepts based on your creative ideas.Help drive breakthrough... ...for hero products and high-impact launches... ...Contribute to the lab's creation of... ...and designing human-AI collaboration... ...leaders.Experience in evaluating complex spaces to...Creative$164k - $294k
...growing our team of passionate creatives and builders on a mission to make design accessible to all. Figma’s... ...into code, or iterating with AI. From idea to product, Figma empowers teams to streamline... ...capabilities and oversee evaluation quality. This is an exciting opportunity...CreativeFull timeFor contractorsRemote workWork from home$176k - $193k
Build a Safer World. TRM Labs provides AI-powered intelligence... ...and they do it with our products. As Senior Product Designer, you'll design the tools... ...adaptability, collaboration, and creative problem-solving. At TRM,... ...leverage You will be evaluated on applied AI fluency...CreativeWorldwideDay shift1 day per week- Mercor connects elite creative and technical talent with AI research labs and offers a remote contract role for a Product Design/UX Expert. You will develop product mockups in Figma/Sketch/Adobe, evaluate design decisions, and provide UX feedback to improve usability while...CreativeRemote jobContract work
$50 - $125 per hour
...the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Summers, and Jack Dorsey. Position: Product Design/UX Expert Type: Contract... ...hours/week Role Responsibilities Evaluate product design decisions using...CreativeContract workSummer workRemote work$75 - $150 per hour
AI labs and AI product companies have been approaching Contra to test out their latest capabilities with our best creative experts. This is important because these products... ...become an expert at evaluating the latest AI tools... ...looking for a Brand Designer to join our network...CreativeHourly payStart working today$150 per hour
...the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco,... ...model response grading. Evaluate clinical dialogues for accuracy... ...medical education Grading-criteria design or clinical guideline...CreativeContract workSummer workRemote work$170 - $190 per hour
...job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...- Clinical Documentation & AI Evaluation Type: Contract Compensation... ...feedback for engineering and product teams. Contribute expert...CreativeContract workWork experience placementSummer workRemote work$20 - $160 per hour
...Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/...CreativeContract workSummer workRemote work$210k - $240k
...every weekday working on products that are not really... ...my job search, I was evaluating 3 offers and all were... ...Safer World. TRM Labs provides AI-powered intelligence... ...customers. You’ll also design backend APIs with a... ..., collaboration, and creative problem-solving. At...CreativeFull timeWork at officeWorldwide3 days per weekWeekday work- ...Koah Product Designer Koah is building AI–powered ad formats that work across any surface. In a chat app, a Koah ad feels like part of the conversation... ...to own the advertiser and publisher experiences – the creative tools, dashboards, and integration flows that make Koah...CreativeLive inImmediate start
$85 per hour
...job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...coding agents to complete and evaluate complex infrastructure engineering... ...~ Experience supporting production-scale systems. Compensation...CreativeContract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Product Designer - AI Creative Evaluation (Contra Labs). Be the first to apply!
- product designer ux ui San Francisco, CA
- product designer senior San Francisco, CA
- staff product designer San Francisco, CA
- product designer San Francisco, CA
- product designer full time San Francisco, CA
- product designer remote San Francisco, CA
- full creative work from home San Francisco, CA
- creative operations San Francisco, CA
- creative circle San Francisco, CA
- creative services San Francisco, CA






