Architecture AI Model Evaluator & Task Designer
$50 per hourMercor Inc
Mercor is seeking an Architecture Expert to evaluate AI models in visual document understanding within the Architecture domain. The role involves authoring grounded tasks with clear rubrics and collaborating with SMEs to improve dataset quality. The contractor role is remote, with a commitment of 15–20 hours per week, at a rate of $50/hour. Flexible scheduling and asynchronous work are expected to meet deadlines while advancing model performance. #J-18808-Ljbffr
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Architecture AI Model Evaluator & Task Designer in New York, NY vacancy
- ...Weekday AI is an AI-native recruiting platform offering a remote, contract... ...legal professionals. You will design complex legal tasks, judge AI-driven responses, and provide precise rationales to guide model training and evaluation. This senior, task-based position is...SuggestedContract workRemote workFlexible hoursWeekday work
Weekday AI
New York, NY5 days ago - ...Prolific is seeking Product Designers and UX Specialists to join its... ...Expert Network, designing and evaluating AI training materials with real... ...participant to train and evaluate AI models from home. Candidates bring a... ...knowledge. Pay varies by task and is per hour. #J-18808-...SuggestedHourly payRemote work
Prolific
New York, NY3 days ago $30 per hour
...not just another player in the AI space – we are building the... ...Professional Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple... ...and now tackle complex design tasks such as layout, typography,...SuggestedRemote workWork from homeFlexible hoursProlific
New York, NY1 day ago$18 per hour
...Outlier helps the world's most innovative companies improve their AI models by providing human feedback. Are you a strong communicator with fluency in English who would like to lend your expertise to train AI models?About the opportunity:Outlier is looking for talented...SuggestedHourly payFreelanceRemote workFlexible hoursScale AI
New York, NY7 hours ago- SaidGig seeks a life sciences task designer to help train future AI systems by creating expert-level evaluation tasks from authentic research artifacts. You will craft problems that require multi-step reasoning and rigorous rubrics, testing AI agents on real data and protocols...SuggestedRemote jobContract work
- ...Rise Data Labs seeks a Media & Information AI Training Specialist to craft realistic media tasks and rubrics for AI training in journalism, editing, and video... ..., or video production experience and comfort evaluating professional-grade work to benchmark AI performance...
BAM VENTURES LLC
New York, NY4 days ago - Mercor connects elite creative and technical talent with leading AI research labs. The company is headquartered in San Francisco,... ...for a contract role with remote work options. The role focuses on evaluating AI-generated responses for safety, accuracy, and policy compliance...Remote jobContract workSummer work
- HumanSignal is a remote-friendly platform pairing graphic design professionals with AI training projects. We’re seeking designers to rate and edit design samples, helping AI models learn what makes visuals effective. You’ll rate typography, color, layout, and hierarchy...Remote jobFreelance
- Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical reasoning and care workflows, and provide structured feedback to ensure medical accuracy and safety. The role requires...Remote jobPart timeFor contractors
- Prolific is seeking Senior Python Developers to join Expert Network to help train and evaluate AI models. You’ll review AI-generated Python code for accuracy, validate logic, and run tests in appropriate environments to ensure high-quality outputs. We value deep Python...Flexible hours
$150 per hour
Mercor is seeking a General Physician for a remote, short-term contract. You will evaluate AI model outputs for clinical validity, provide structured feedback, and flag safety issues while collaborating with AI researchers. The assignment spans 6 weeks at ~20+ hours per...Remote jobHourly payTemporary work- ...Anyone AI is seeking Civil Engineering Experts to design and validate realistic tasks for AI agents. You will interpret tender documents, specifications, drawings, and... ...review AI-generated outputs and help establish evaluation criteria. We require a bachelor’s degree or higher...
Anyone AI Inc.
New York, NY3 days ago - ...Partner Company, a remote-first AI research initiative, seeks a senior animator-evaluator to assess AI-generated performances and establish rigorous evaluation standards for realism. You will define benchmarks and rubrics, then apply them to character animation, facial...Remote work
Partner Engineering and Science
New York, NY4 days ago $85 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms...Contract workSummer workRemote work- ...Volga Partners is seeking an experienced Senior Investment Research & Capital Markets SME to support the development, evaluation, and improvement of AI models. This is not a traditional Investment Banking position; you will assess AI-generated financial analyses, validate...Bank staffRemote work
Volga Partners
New York, NY3 days ago - MetLife seeks a Lead, Operating Model Design within Global Technology to act as an internal consultant and operating model designer for an AI-First framework. You will author team archetypes, role definitions, accountability frameworks, and transition artifacts, then support...
$164k - $294k
A leading design collaboration company is seeking an AI Model Designer to bridge AI research and design principles. This full-time role involves collaborating with AI researchers, developing design heuristics, and enhancing AI-generated outputs. Ideal candidates have strong...Remote jobFull time- ...is seeking a remote VFX Artist / 3D Animator reviewer to evaluate AI outputs for 3D design workflows. You will assess craft, constraints, and... ...readiness issues, and document corrections to train the modeling team. This is a contractor role with eligibility to work...For contractorsRemote work10 hours per week
AuraOne
New York, NY4 days ago - Table Image Reasoning Model Evaluator is a remote evaluation track for reviewing table image reasoning... ...use to retrain. Why this role matters AI data reviewers help turn table image... ...at least 10 hours per week. Example tasks Compare two table image reasoning model...Hourly payFor contractorsRemote work10 hours per week
- ...Labs Rise Data Labs is a leading AI training and data annotation... ...to improve machine learning models through expert human annotation... ..., economically valuable media tasks and rubrics based on your own... ...experience with AI model training, evaluation, or benchmark development is a...Flexible hours
- ...Uniti AI is seeking a Senior Product Designer to own end-to-end design across client products and AI-powered platforms. You will craft user journeys, information architecture, and polished interfaces for both operational tooling and customer-facing experiences. Join...
Motion Recruitment
New York, NY5 days ago - ...ELEKS Product Design Office seeks a Senior Customer Experience Expert to design end-to-end CX and leverage agentic AI to streamline research synthesis, persona generation, journey analysis, and reporting. The role focuses on building reliable AI workflows that reduce manual...Work at office
Eleks
New York, NY2 days ago $15 per hour
...Outlier is an AI training platform owned and operated by Scale AI. Outlier Experts... ..., from generating training data to evaluating the performance of models. About the opportunity: We are... ...to what you saw or heard Common tasks: Image evaluations: Compare two...Hourly payContract workFor contractorsWork experience placementFreelanceInternshipRemote workNight shiftScale AI
New York, NY2 days ago- ...Science Specialists to join our Expert Network to help train and evaluate AI models. After a brief 10–15 minute test, successful candidates may join Prolific as participants and earn money for paid tasks. In this role you will perform AI evaluation, ranking, and scientific...Remote work
Prolific
New York, NY3 days ago $50 per hour
...Mercor is seeking a Spanish (Mexico) Audio Generalist Evaluator for a contract role. The position is remote and requires 10–20 hours... ...will transcribe, evaluate, and provide feedback to improve AI language tasks in Spanish and English. The role focuses on developing...Contract workRemote work10 hours per weekMercor Inc
New York, NY3 days ago$100 - $160 per hour
...60 per hour equivalent, paid per accepted task • 100% remote • Part-time • Fully flexible... ...Work on the cutting edge of legal AI, on your own schedule, and get paid for the... ...for factual and legal accuracy • Put the model through challenging litigation edge cases...Hourly payPart timeFor contractorsRemote workFlexible hoursOpenTrain AI
New York, NY2 days ago- ...Mindrift is seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for a medical AI evaluation program. Projects are part-time and non-permanent, with compensation paid per accepted task. Contributors will review vignette content across multiple medical...Permanent employmentPart time
Mindrift (Toloka)
New York, NY7 hours ago - ...Own dynamic model routing: the right model for each task, under a hard cost and policy ceiling. Dipp AI's strategy is decoupled orchestration — no single frontier model, but... ...task classification, capability profiles, evaluation harnesses, and the cost governor that enforces...
Dipp AI Technologies, Inc.
New York, NY3 days ago - ...project involving real-world litigation drafting for an advanced AI evaluation initiative. In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world...Remote job
$124k - $174.8k
...close on: 10/24/2026Meet the TeamContent designers at Splunk are integral to the product... ...consistent UI text, establish information architecture, and develop content-led design patterns... ...design coverage, providing frameworks and AI-assisted tools to maintain quality at...Full timeTemporary workLocal areaRemote workFlexible hoursCisco
New York, NY2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Architecture AI Model Evaluator & Task Designer. Be the first to apply!
Related searches