Remote AI Data Annotation Jobs in Phoenix

Remote AI Data Annotation Jobs in Phoenix (Remote) Rex.zone is hiring Phoenix-based candidates for full-time remote roles focused on producing labeled training data and evaluation signals used in modern LLM training pipelines. You will label and review datasets across NLP, content safety, and (as needed) computer vision, while supporting model performance improvement through consistent, high-quality evaluation. Key Responsibilities Execute data labeling for NLP tasks (text classification, summarization, and named entity recognition) Perform RLHF preference work including pairwise ranking, preference labeling, and rationale capture Conduct prompt evaluation and response grading for instruction-following models Complete QA evaluation (self-checks, peer review, audits) to maintain training data quality Follow annotation guidelines compliance, document edge cases, and elevate ambiguous items Meet throughput and accuracy targets while maintaining consistent rubric interpretation Required Qualifications Mid-to-senior experience in AI data annotation, data labeling, or QA evaluation workflows Familiarity with RLHF and large language model evaluation concepts (or strong QA skills with ability to ramp) Strong attention to detail for policy-based content safety labeling Comfort with annotation tools, guideline documents, calibration sessions, and audit workflows Workflow & Quality Metrics Quality measured via guideline adherence, audit scores, defect-rate tracking, and consensus alignment for RLHF rankings Structured queues with clear acceptance criteria, spot checks, and calibration to improve consistency Location & Remote Notes This is a Remote role for candidates based in Phoenix, US , collaborating asynchronously with distributed teams and following secure data handling practices. Compensation Hourly base pay range: $30-$50/hr (USD), based on scope, complexity, and demonstrated evaluation accuracy. #J-18808-Ljbffr Rex.zone