Media / journalism / communications Evaluator [Remote]
$80 - $120 per hourAuraOne Human Data
Remote
- Remote job
Media / journalism / communications Evaluator is a remote evaluation track for reviewing media / journalism / communications evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can use to retrain.
Why this role matters
AI data reviewers help turn media / journalism / communications evaluation outputs into auditable labels, rationales, and regression cases for AuraOne Human Data.
Responsibilities
- Evaluate media / journalism / communications evaluation model outputs against a versioned rubric and assign severity tags for Media / journalism / communications Evaluator assignments.
- Compare paired responses and pick the stronger answer with a written rationale.
- Label hallucinations, instruction-following failures, and unsafe content with structured tags.
- Capture ambiguous prompts and route them back to the program team for rubric updates.
- Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
- Document recurring failure modes so the modeling team can target them in the next training run.
Qualifications
- Prior evaluation, annotation, or human-rater experience on media / journalism / communications evaluation or adjacent content for Media / journalism / communications Evaluator work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the issue and the rubric clause being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Compare two media / journalism / communications evaluation model responses to the same prompt and pick the stronger one with rationale.
- Tag an unsafe response with the correct policy category and severity.
- Audit a 50-row batch for rubric consistency and report drift to the program lead.
- Propose a rubric clarification after spotting a recurring failure mode.
Nice to have
- Background in linguistics, content moderation, or trust & safety review.
- Experience with inter-rater agreement metrics and calibration cycles.
- Domain expertise that lets you spot subject-matter errors automated checks miss.
Skills
- Model output evaluation
- Rubric-based annotation
- Severity tagging
- Inter-rater calibration
- Media / journalism / communications evaluation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
$80–$120 / hr
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Media / journalism / communications Evaluator [Remote] in Remote vacancy
- Mercor offers a remote contract opportunity for a Media/Journalism/Communications Evaluator. You will assess AI-generated artifacts, identify errors, and provide structured feedback to improve model outputs. The role requires 5+ years in media or related fields, native...SuggestedRemote jobContract work
$120 per hour
...Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Media / journalism / communications Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities...SuggestedRemote jobContract workSummer workWork at officeMercor
San Francisco, CA10 days ago- Obsidian is looking for expert Evaluators in Media, journalism, and communications to review and assess AI-generated work products, ensuring accuracy and rigor. This role is remote, offering flexibility, requiring 5+ years of relevant experience and proficiency in Microsoft...SuggestedRemote jobWork at office
- Effizotech is seeking a Social Media Evaluator to enhance the integrity of information shared online. This entry-level, part-time position is completely remote and requires a commitment of 20 hours per week. As a Social Media Evaluator, you'll analyze social media posts...SuggestedRemote jobPart time
- Alignerr is seeking a Social Media Content Evaluator (AI Training) to review AI-generated posts, captions, and comments. You will help ensure content feels natural, engaging, and appropriate for real audiences. This is a fully remote, hourly contract role with 10-40 hours...SuggestedRemote jobHourly payContract work10 hours per week
- Work from Home | Internet Analyst | Social Media Evaluator At Appen, we work with 8 out of the top 10 global technology companies in the world to improve and optimize their products for users worldwide, including social media platforms, search engines, voice recognition...Part timeWork from homeWorldwideFlexible hours
- Obsidian is hiring expert Evaluators in Public health communications for a remote position. The role involves reviewing and assessing AI-generated work products for accuracy and domain quality. Candidates must have 5+ years of experience, native or professional fluency...Remote jobHourly payWork at office
$47k - $49k
...strengths to achieve shared objectives.Emotionally Intelligent Communication: Engages in a skillful exchange of information, thoughts, and... ...Jesuit Catholic values.Job Description SummaryThe Admissions Evaluator - International Credential Evaluations carefully evaluates...Temporary workWork at officeFlexible hours- ...operate on a hybrid schedule with four in-office days per week (Monday–Thursday). This approach enhances creativity, innovation, communication, and relationship-building, fostering a dynamic and collaborative work environment. Chamberlain is accredited by the...Part timeWork at officeWorldwideFlexible hoursWeekday work
- ...1 is seeking a Journalist/Writer contractor to participate in evaluating advanced AI-generated writing from a remote environment. Your... ...concise rationales. No prior AI experience is required; a strong journalism or editorial background and superior English fluency are...Remote jobFor contractors
- DataAnnotation is seeking a Designer to evaluate how models translate into UX and visual design, identifying flows that hinder usability and proposing stronger critique. You’ll review AI design work across user flows, layout, and typography, catch polish failures, and...Remote job
$37.5 per hour
...In this role you will be responsible for evaluating advertising copy generated by a large... ...candidate should have:-A creative advertising, media, marketing, or copywriting-Experience... ...editing, quality assurance, creativity, journalism, editorialJob Type & LocationThis is a...Contract workTemporary workRemote workTEKsystems
New York, NY1 day ago- ...assessments, analyze data, and develop value estimates following government, company, and client guidelines. Strong writing and communication skills are essential. Minimum 0-1 years in mortgage/appraisal settings; active license or certification required. The role offers...Remote jobFlexible hours
- ...growth opportunities. You will conduct unbiased appraisals, document on-site observations, analyze data, and prepare value estimates in line with government, company, and client guidelines. Strong writing and communication are essential. #J-18808-Ljbffr First AmericanRemote jobFlexible hours
- ...hiring for a Personalized Internet Ads Evaluator role! This is a freelance, independent contractor... .... Basic Requirements Excellent communication skills with full professional... ...familiarity with current and historical business, media, sport, news, social media, and cultural...Part timeFor contractorsFreelanceCurrently hiringImmediate startWork from homeFlexible hours
$60 per hour
...quality assessment and brand alignment. This work directly supports our efforts to standardize how visual content is evaluated across different media formats. Establishing a strong baseline for photographic excellence helps us maintain high visual standards moving forward...Hourly payRemote work- ...provider is seeking a Personalized Ads Evaluator for a part-time position. This entry-level... .... Candidates should have excellent communication skills and full professional proficiency... ...English, alongside familiarity with various media and cultural communities. The position allows...Remote jobPart time
- RWS Group is seeking a self-motivated evaluator to help refine search quality for global AI systems. You will assess everyday queries, provide precise feedback on various media, and help improve AI data used by major search engines and platforms. The role offers flexible...Remote jobPart timeWork from homeFlexible hours
$14 per hour
...A Day in the Life of an Internet Safety Evaluator: In this role, you will be evaluating and... ...Working as a freelancer with excellent communication skills in Hebrew and English. Being a... ...familiarity with current and historical business, media, sport, news, social media, and cultural...Hourly payPart timeFreelanceImmediate startWork from home$35.81 - $42 per hour
...Health is looking for a Part-time PASRR Evaluator to join our growing team. Job Summary: Join... ...interpersonal, verbal, and written communication skills, with the ability to communicate... ...through spoken, written, and electronic media Attention to detail and the ability to effectively...Contract workPart timeWork at officeLocal areaRemote workWork from home$176.8k
...platform. During the session, you will review several image pairs and evaluate how well each matches a set of provided brand guidelines. You... ...content moderators, and general evaluators with an interest in media curation. You should be comfortable reading a brief set of...Hourly payContract workRemote work$41.6k - $47.84k
...$23.00 Position Summary The Records Evaluator I analyzes and assesses student academic... ...Excellent interpersonal, oral, and written communication skills with the ability to work within a... ...or other technology-based communication media. Ability to maintain confidentiality....Hourly payFull timePart timeWork at officeLocal areaRemote work- ...Description Mystery Evaluators (Gig / project-based work) - Freelance work - Multiple Opportunities Available in Tamil Nadu-Madurai... ...voucher by participating in the quiz, kindly follow our social media page link given below Website: LinkedIn: LinkedIn...Extra incomeFreelance
$80 - $120 per hour
...Public health communications Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical...Remote jobFor contractors10 hours per week- Overview At PAM Health, we care for chronically and critically ill patients who require extended hospital care. PAM Health has over 80 hospital locations and employs over 11,000 people across the country. Our teams work together to deliver the highest level of compassionate...Full timeLocal area
PAM Health
Humble, TX5 days ago - ...TechnicalPer DiemAll ShiftsvariedJob SummaryThe Mental Health Evaluator (MHE) is a critical role dedicated to delivering... ...documentation of assessments, interventions, discharge planning and communications with internal and external stakeholders.Work collaboratively,...
HealthAlliance of the Hudson Valley
Poughkeepsie, NY5 days ago $80 - $120 per hour
...This role is for one of our clients Compensation: $80 - $120 per hour We are hiring expert Evaluators in Special education / IEP to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality...Hourly payContract workFor contractorsWork at officeRemote workWeekday AI
United States1 day ago$80 - $120 per hour
...Special Education Iep Evaluator This role is for one of our clients. Compensation: $80 - $120 per hour. We are hiring expert Evaluators in special education / IEP to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy...Hourly payRemote workWeekday
United States1 day ago$9 - $30 per hour
ALTA Language Services, Inc. is looking for a remote Testing Evaluator in Atlanta, GA. This part-time position involves assisting with... ...according to set criteria. Candidates should have strong communication skills, professionalism, and ideally a college degree. The pay...Part timeRemote workALTA Language Services
United States3 days ago- ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...Contract workTemporary workImmediate startRemote work
MERIT Beauty
United States5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Media / journalism / communications Evaluator [Remote]. Be the first to apply!
