Remote Data Annotation Specialist - Frontier Model Evaluation
AuraOne
New York, NY
- Remote job
AuraOne is seeking a remote Generalist for Data Annotation to evaluate prompts and responses against a quality rubric. You’ll help turn outputs into auditable labels, rationales, and regression cases for Human Data, reviewing frontier model outputs and calibrating evaluators. Responsibilities include selecting stronger answers with rationale, tagging issues, and routing ambiguous prompts for rubric updates. Strong attention to detail and async availability are essential. #J-18808-Ljbffr AuraOne
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote Data Annotation Specialist - Frontier Model Evaluation in New York, NY vacancy
- AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses... ...evolving quality rubric. You will compare model outputs, label edge cases, and... ...stronger answers with rationale, and annotating issues like hallucinations or unsafe...Remote jobFor contractors
$20 per hour
...Compensation $15-$20/hour Location Remote Role Responsibilities... ...Generate high-quality human evaluation data by identifying response strengths... ...of responses. Ensure model responses align with expected... ..., model evaluation, or data annotation work Experience writing or editing...Remote jobContract workPart timeSummer work$20 - $50 per hour
Remote | United States | Project-Based Independent... ...the development and evaluation of advanced AI models. We’re seeking... ...Submission Writing Specialists experienced in CTD/eCTD... ...laboratory operations, data review, and... ...development, or data annotation. Previous involvement...Remote workHourly payExtra incomeFor contractorsFreelance- ...Frontier Model Misuse Risk Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios... ...model Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants...Remote jobHourly payFor contractors10 hours per week
$295k
...cutting-edge foundation AI models and end-to-end... ...training and deploying frontier models for enterprises... ...Join us!Role Overview:Evaluation is critical to making... ...judges; refining LLM-based data synthesis pipelines;... ...This role can be based remotely or from one of our...Remote workFull timeWork at officeLocal areaHome officeCohere
New York, NY1 day ago- Agentic Task Completion Model Evaluation Specialist is a remote evaluation track for reviewing... ...Why this role matters AI data reviewers help turn... ...Qualifications Prior evaluation, annotation, or human-rater... ...Model Evaluation evaluation Frontier evaluation Rubric calibration...Remote workHourly payFor contractors10 hours per week
- ...We’re training and deploying frontier models for developers and... ...the future! Why this role?Evaluation is critical to making progress... ...judges; refining LLM-based data synthesis pipelines; and improving... ...workspace improvement ~? Remote-flexible, offices in Toronto...Remote workFull timeWork at officeFlexible hours
cohere
New York, NY8 days ago $85 per hour
...Type: Contract Compensation: $85/hour Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms ,...Remote workContract workSummer workMercor
New York, NYa month ago$60 per hour
Prolific is seeking Biology Experts and Life Science Professionals to evaluate AI-generated science and ensure compliance with scientific standards. Responsibilities include reviewing biological inquiries, validating technical claims from public databases, and critiquing...Remote workHourly payWork from homeFlexible hoursProlific
San Jose, CA3 days ago- ...training and deploying frontier models for developers and... ...safely accessible to all. Data quality is... ...Own Device (laptop). Remote work within Canada. 1... ...included! As a Data Annotation Specialist, You Will Label and... ...Preference-Based Tasks: Evaluate and complete tasks, assessing...Remote workHourly pay16 hoursContract workTemporary workPart timeFor contractors
$15 - $35 per hour
Remote | United States | Project-Based Independent Contractor... ...information, evaluates medical scenarios, and... ...Artificial Intelligence (AI) models. We're seeking... ...specific scoring rubrics, annotation instructions, and quality... ...privacy, ethical, and data security requirements....Remote workHourly payFor contractorsTraineeshipFreelanceInternshipWeekday work- Computer Vision Data Annotation Specialist is a remote evaluation track for reviewing computer vision data annotation evaluation prompts and responses against... ..., and write the kind of structured feedback the modeling team can use to retrain. Category: Video, Vision & Spatial...Remote workHourly payFor contractors10 hours per week
- Computer Vision Data Annotation Specialist is a remote evaluation track for reviewing computer vision data annotation evaluation prompts and responses against... ..., and write the kind of structured feedback the modeling team can use to retrain. Why this role matters AI data...Remote workHourly payFor contractors10 hours per week
- ...speakers to review AI-generated media. You will roleplay live conversations with AI voice models, evaluate image/video prompts, and explain in writing what looked or sounded wrong, all remotely and on your own schedule. Responsibilities include ranking naturalness, comparing...Remote job
- ...seeking a detail-oriented contractor to assist in evaluating business scenarios used to train AI. You will interact with the model, pose analytical questions, and assess the... ...clear, metacognitive feedback. This is a remote freelance role with flexible hours. #J-18808...Remote jobFor contractorsFreelanceFlexible hours
$72k - $76.44k
...may take place by phone. For remote roles, and at our... ...have. We’re looking for a Data Annotation Specialist who can work full-time with... ...and how annotation supports model training Physical Requirements... ...(Brighthire) to assist in evaluating candidates. Collectors Universe...Remote workFull timeLocal areaWorldwideFlexible hoursShift work- AuraOne seeks a Spanish Language Expert (Spain) to remotely evaluate generalist prompts and responses, applying a versioned rubric and producing structured feedback for retraining. You will review pairings, label edge cases, and document your rationale for program improvement...Remote jobPart timeFor contractors
- ...Contract) to contribute to next-generation AI systems by applying deep CS expertise. You will analyze, evaluate, and create high-quality technical content to improve AI model reasoning, accuracy, and performance. Prior AI experience is not required; CS expertise is the...Remote jobContract workWeekday work
$8 per hour
We are looking for AI Data Specialists to support the improvement of AI-generated... ...Type: Freelance Location: Remote Work Schedule: Part-time - 1... ...and safety of today’s AI models? What You’ll Do Data collection, evaluation, and annotation. Pairwise comparisons. Counting...Remote workHourly payExtra incomePart timeFreelanceImmediate startWork from home10 hours per weekFlexible hours$30 - $50 per hour
...in AI solutions is searching for a remote, full-time candidate to support AI/... ...role, you will perform high-accuracy data labeling, execute RLHF evaluation, and ensure training data quality.... ...experience in evaluation or data annotation, familiarity with NLP concepts, and...Remote workHourly payFull timeRex USA
New York, NY1 day ago$45 - $70 per hour
Gridnaut Recruiting is hiring a remote Behavioral Health Expert, AI Safety and Model Evaluation contractor (pay $45-$70/hr). Contribute to frontier AI research and evaluation work. Ideal... ...psychologists, case managers, peer support specialists) helping evaluate how AI models...Remote workContract workPart timeFor contractors$8 - $10 per hour
...Required Skills video annotation audio annotation data labeling... ...guideline-based evaluation independent work... ...confidentiality remote collaboration About... ...lab for training frontier models and evaluating AI... ...Video Data Annotation Specialist Role Type:...Remote workPart timeFor contractorsmicro1
San Francisco, CAa month ago$200k - $320k
...language, robotics and agent models meant to match frontier models on targeted... ...assembling training data through fine-tuning, evaluation and production deployment... ...s own data generation, annotation, QA and grading workflows. It is a full-time, remote employee position. What...Remote workFull time- ...Frontier Model Misuse Red Team Specialist is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios... ...the gap. Why this role matters Adversarial evaluation is how AuraOne hardens AI models before they ship to...Remote jobHourly payFor contractors10 hours per week
- ...Causal Reasoning Model Evaluation Specialist is a remote review track for evaluating AI outputs across causal reasoning model evaluation research review... ...Causal Reasoning Model Evaluation research review Frontier evaluation Rubric calibration Failure analysis...Remote jobHourly payFor contractors10 hours per week
- DataAnnotation is seeking an experienced Electrical Engineer to guide how frontier AI models reason about real electrical and electronics engineering work... ...and critique the results with detailed, evidenced reasoning. Remote collaboration and cross-domain insights are essential for...Remote job
- A leading AI solutions provider is seeking AI Data Specialists to improve AI-generated content in English. This freelance, part-time... ...per week. Responsibilities include data collection, evaluation, annotation, and labeling across various content types. Candidates should...Remote jobExtra incomePart timeFreelance10 hours per weekFlexible hours
$200k - $320k
...of Technical Staff, Model Training (applied research... ...time Location : Remote The Role In... ...model adaptation, evaluation, and production... ...exceed the quality of frontier models on targeted workloads... ..., improving data generation, annotation, quality assurance,...Remote workFull timeLocal area$25 per hour
...You’ll be at the forefront of evaluating how LLMs interact with real... ...Software Engineers to support a frontier AI training and evaluation... ...Turing Work in a fully remote environment. Opportunity... ...date is next week] Payment model: $25 per approved task...Remote workContract workFor contractorsFreelance$30 - $35 per hour
...and scalability. We are looking for a Model Evaluation Operator- AI Robotics (Contractor) to help... ...also collect high‑quality operational data and performance feedback that directly... ...Maintain accurate evaluation logs, notes, annotations, and performance records for...Hourly payFor contractorsMonday to FridayShift workAfternoon shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Data Annotation Specialist - Frontier Model Evaluation. Be the first to apply!
Related searches
- data specialist New York, NY
- data operations specialist New York, NY
- data management specialist New York, NY
- data conversion specialist New York, NY
- data visualization specialist New York, NY
- oncology data specialist New York, NY
- data migration specialist New York, NY
- data entry associate New York, NY
- data management associate New York, NY
- data science associate New York, NY