Human AI Response Evaluation Generalist
OpenTrain AI, Inc.
The Work You will evaluate language-model responses in social scenarios and provide structured feedback that can help improve AI system behavior. The work involves careful review, evidence-based decisions, and consistent use of project guidelines. Review source materials and language-model responses against project guidelines. Apply evaluation rubrics and document evidence for your judgments. Explain decisions clearly in written feedback. Maintain consistent quality across repeated review tasks. What It Pays and Takes This is remote contractor work. The project is expected to run for 2 to 3 weeks, with individual tasks taking about 60 minutes. Pay rate: Not provided in the listing. Time requirement: 20 or more hours per week. Work arrangement: Remote and open worldwide. Language: Fluency in English. Experience level: Entry level; prior AI training experience is not required. Relevant professional, academic, or language expertise in language-model response evaluation. Strong written communication and careful attention to detail. Ability to follow detailed instructions and apply rubrics consistently. Comfort working independently on remote contractor tasks. About AI Training Work AI training is the human work behind systems that learn from examples, including reviewing model responses and rating their quality. People with strong subject knowledge are paid for careful judgments and clear feedback that help improve how AI systems respond. #J-18808-Ljbffr OpenTrain AI, Inc.
- ...countries, and we believe that scaling AI responsibly is what makes it durable. Technology... .... We are not looking for generalists who advise on AI risk in the abstract... ...and data governance, privacy, safety evaluations, and human oversight, and build these controls...SuggestedFull timeWork experience placementLive inWork at officeLocal area
- ...About the role Swahili Evaluation AI Evaluator is a remote evaluation track for reviewing swahili generalist evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare... ..., and regression cases for AuraOne Human Data. Review frontier model outputs....SuggestedHourly payFor contractorsRemote work10 hours per week
- ...Bilingual Portuguese AI Evaluation Specialist is a remote evaluation track for reviewing portuguese generalist evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare... ..., and regression cases for AuraOne Human Data. Review frontier model outputs...SuggestedHourly payFor contractorsRemote work10 hours per week
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...hour Location: Remote Role Responsibilities Design precise, task-specific... ..., and technical reports. Evaluate AI-generated or human-created work against established criteria...SuggestedContract workSummer workRemote work- OpenTrain AI, Inc. is seeking a remote contractor to evaluate language-model responses in social scenarios and provide structured feedback that helps improve AI behavior. The work involves careful review, evidence-based decisions, and consistent use of project guidelines...SuggestedRemote jobFor contractors
$197.3k - $225.1k
AI Engineer 4 (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital... ...applications of AI & ML are bringing humanity and simplicity to banking. We are...Full timePart timeLocal area$229.9k - $262.4k
Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years... ...of AI & ML are bringing humanity and simplicity to banking. We are committed...Full timePart timeLocal area$20 - $160 per hour
...technical talent with leading AI research labs.... ..., and Jack DorseyPosition: Generalist Annotator — Health AI Conversation... ...relevance and clarity of AI responses. Ensure the AI answers the... ...PreferredExperience in annotation, human-feedback, evaluation, or content review....Summer work$20 - $160 per hour
...technical talent with leading AI research labs.... ...Dorsey . Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract... ...Location: Remote Role Responsibilities Evaluate the... ...Experience in annotation, human-feedback, evaluation,...Contract workSummer workRemote work- Get AI-powered advice on this job and more exclusive features. Job Title: Human Data Manager Job Type: Full-time Location: Remote... ..., autonomously sources and evaluates researchers and data... ...impact from day one. Key Responsibilities Design, build, and manage...Full timeInternshipRemote work
$120 - $170 per hour
...technical talent with leading AI research labs. Headquartered... ...Location: Remote Role Responsibilities Design precise, task-specific... ...Score AI-generated and human work samples against criteria... ...experience with AI training , evaluation, or human-data projects....Contract workSummer workRemote work- .../ US The role You will own Anyone AI's data initiatives and proposals to... ...this is the operational center of the Human Data Division. Responsibilities Proposals & requests. Study public... ...engagement through delivery. Deep evaluation and quality expertise: LLM benchmarking...Remote job
$270k - $365k
...Data Operations Manager, Human DataSan Francisco, CA | New York... ...research teams working on frontier AI capabilities. You'll partner... ..., we'd love to hear from you.Responsibilities:Own and execute data strategy... ...-quality data pipelinesBuild evaluation frameworks and quality...Work at officeVisa sponsorshipFlexible hoursShift work- ...About the role AI Response Adjudicator is a remote review track for evaluating AI outputs across ai response adjudicator specialist operations workflows. Reviewers... ...details. 02 Attach this role deliberately. 03 Receive a human review decision or follow-up. Placement timing...Hourly payFor contractorsWork experience placementRemote work10 hours per week
$77k - $202k
...LevelSenior AssociateJob Description & SummaryThe OpportunityAs a Responsible AI Senior Associate you will advise on privacy laws and data... ...assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted...Full timeInternshipH1b$180k - $220k
Aircall is a unicorn, AI-powered customer communications platform... ...Engineer, Detection and Response you will develop, scale, and evolve... ...model, guardrails, evaluation harness and tracing.Conduct threat... ...recruitment team but do not replace human judgment. Final hiring decisions...Full timeWorldwide- ...Reference Number: 26-00001Title: AI EvangelistLocation: New York,... ...advocacy to facilitate the responsible adoption of AI in finance.Key... ...guidelines for both human and AI-driven contributionsCollaborate... ...retrievalObservability & Evaluation: Setting up logging, debugging...Full timeBank staff
$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Larry Summers , and Jack Dorsey. Position: Generalist Expert Type: Contract Compensation: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to ensure...Contract workSummer workRemote work$270k - $320k
...days per week Role Summary As AI adoption expands rapidly... ...rather than to say "no." Key Responsibilities AI Governance Enforcement Through... ..., agentic systems, MLOps, evaluation and monitoring tooling, and... ...in our core values of humanity and integrity, ensuring that...Local area3 days per week- ...Category: Frontier Model Evaluation Pay: Hourly rate confirmed after the... ...Remote US-eligible Contractor AI Trainer & Evaluator is a... ...trainer evaluation prompts and responses against AuraOne's quality rubric... ...regression cases for AuraOne Human Data. Responsibilities Evaluate...Hourly payFor contractorsRemote work10 hours per week
- ...Job Title AI Evaluation Engineer Location Hybrid / Remote Employment Type Full-time Job Summary... ...AI system reliability and user experience. Key Responsibilities Design and implement evaluation frameworks for AI...Full timeRemote work
- Category: Frontier Model Evaluation · Pay: Hourly rate confirmed after... ...Contractor Gmail & Google Calendar AI Assistant Evaluator is a... ...assistant evaluation prompts and responses against AuraOne's quality... ...regression cases for AuraOne Human Data. Responsibilities Evaluate...Hourly payFor contractorsRemote work10 hours per week
$38 per hour
...You'll work closely with both people and AI systems — auditing outputs, supporting day... ...recurring issues, and edge cases in both human and model outputs Participate in... ...AI/ML data workflows or human-in-the-loop evaluation environments ~ Prior experience auditing...Full timeContract workRemote workVisa sponsorship- ...AI Architect – GenAI / ML Role Overview Photon is looking... .... This role will be responsible for defining and implementing... ...routing, model versioning, and evaluation . Design architectures for... ..., observability, and human-in-the-loop controls . Lead...Full time
- AI Trainer Jobs seeks a Generalist (Macbook User) to remotely evaluate prompts and responses against AuraOne's rubric. You will compare paired outputs, label edge cases, and draft structured feedback for retraining. Role involves reviewing frontier model outputs, judging...Remote job
- AuraOne is seeking a remote Indonesian generalist evaluation reviewer to assess prompts and model outputs against our quality rubric. You... ..., and help produce auditable feedback for retraining. Responsibilities include comparing paired responses, labeling inconsistencies...Remote jobFor contractors
- AI Trainer Jobs is seeking a remote contractor to evaluate rust programming AI evaluation model outputs, compare responses, and provide structured feedback. You will label issues using a versioned... ...prior experience in evaluation or human-rater roles related to rust...Remote jobFor contractors10 hours per week
- ...assets of the practice as they continue to develop and evaluate a new model of care. Responsibilities This position will be in the lab of Dr. Lili Chan... ...in the Department of Artificial Intelligence and Human Health at the Icahn School of Medicine at Mount Sinai...Full timeTraineeshipWork experience placementLocal area
$205k - $275k
EXL is seeking a visionary VP, Head of Responsible AI to architect trust across our enterprise AI initiatives. Reporting jointly to the CIO... ...gone through a formal interview process with members of EXL’s Human Resources team, as well as our hiring managers.EXL is the indispensable...$189.3k - $320.7k
...redefining mobility. Through a human-centered design process, we... ...autonomous driving? Join the Embodied AI team at General Motors. Our... .... You will help define and evaluate machine learning and... ...Congestion and we embrace the responsibility to lead the change that will...Full timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Human AI Response Evaluation Generalist. Be the first to apply!





