Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Human Evaluator, Studio (Contract)

$23.2 per hour

Roblox

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences – all created by our global community of developers and creators. At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device. We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there. A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone. Role Overview We are searching for a self‑motivated individual who is passionate about Roblox and thrives in detail. You will be responsible for classifying content based on a specific set of guidelines to help us improve Roblox systems. As a Human Evaluator you will have the opportunity to provide us direct feedback on the performance of various products across the company, but primarily for features within the studio and creator store. You will: Classify content based on a set of instructions; Content classification includes the review and classification of text, image, video, scripts, and audio Track and document insights and trends related to annotation projects Test out new features within studio and provide detailed feedback Become an expert in a variety of topics to enable more accurate evaluations Dedicate between 25-29 hours per week with a work schedule from Monday to Friday You have: At least 1+ years of experience developing on Roblox or other creation platforms. Solid knowledge of the technical aspects of Roblox Studio. Proficiency with Lua Strong verbal and written communication skills Demonstrated patience for repetitive tasks and attention to detail Effective time management skills In-depth understanding of Roblox and the Roblox community. Fluency in a second language, such as: Japanese, Korean, Chinese, Portuguese, Spanish, French, German, Italian Prior experience with content moderation or data annotation. A deep passion for gaming trends and the Roblox community ecosystem. Please note Benefits below only apply to full time employees in the US. This role is fully remote For roles that are based at our headquarters in San Mateo, CA: The starting base pay for this position is as shown below. The actual base pay is dependent upon a variety of job‑related factors such as professional background, training, work experience, location, business needs and market demand. Therefore, in some circumstances, the actual hourly rate could fall outside of this expected range. This pay range is subject to change and may be modified in the future. Please note that not all benefits on this page are applicable to this hourly position. Hourly Pay Range

$23.20 — $23.20 USD

Roles that are based in an office are onsite Tuesday, Wednesday, and Thursday, with optional presence on Monday and Friday (unless otherwise noted). Roblox provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. Roblox also provides reasonable accommodations to candidates with qualifying disabilities or religious beliefs during the recruiting process. #J-18808-Ljbffr Roblox

Vacancy posted 11 hours ago
Similar jobs that could be interesting for youBased on the Human Evaluator, Studio (Contract) in New York, NY vacancy
  • $20 per hour

     ..., and Jack Dorsey. Position Generalist - English & Urdu Type Contract Compensation $15-$20/hour Location Remote Role Responsibilities...  ...public sources and external tools. Generate high-quality human evaluation data by identifying response strengths, areas for improvement... 
    Contract work
    Remote job
    Part time
    Summer work

    Mercor

    New York, NY
    3 days ago
  •  ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers... 
    Contract work
    Temporary work
    Immediate start
    Remote work

    MERIT Beauty

    New York, NY
    3 days ago
  •  ...level quantitative finance experts, the full-time Remote AI Research Evaluator will assess AI-generated financial content, craft relevant questions, and evaluate model responses in a flexible contract role. Key responsibilities Assessing the factuality and relevance of... 
    Contract work
    Full time
    Remote work
    Flexible hours

    Freelance Labs Inc. d/b/a Braintrust

    New York, NY
    2 days ago
  •  ...people find information online? We're looking for Search Quality Evaluators to assess AI-generated answers, search results, and...  ...relevant, and genuinely useful. This is a fully remote, flexible contract role open to anyone with strong critical thinking and a keen eye... 
    Contract work
    Hourly pay
    Ongoing contract
    Freelance
    Remote work
    Flexible hours

    Alignerr

    New York, NY
    2 days ago
  •  ...seeking a Benchmark Dataset Data Specialist to remotely review and evaluate prompts and model outputs against our quality rubric. You'll...  ..., and provide structured feedback to retrain our models. This contract-based, remote role requires strong attention to detail and consistent... 
    Contract work
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne

    New York, NY
    3 days ago
  • A leading research accelerator is seeking a contractor to evaluate North American teen humor. The role involves reviewing short-form content...  ...to require about 30 hours a week for 1 month, with pay and the potential for future contract extensions. #J-18808-Ljbffr Turing
    Contract work
    Remote job
    For contractors
    Freelance
    Flexible hours

    Turing

    New York, NY
    1 day ago
  • A talent marketplace is seeking soccer experts to evaluate live soccer games. The role involves assessing AI-generated commentary, scoring...  ...skills, and be comfortable with online tools. This contract position aims to refine AI's soccer insights through expert evaluations... 
    Contract work

    Mercor

    New York, NY
    12 hours ago
  • $80 - $120 per hour

     ...looking for a Brand / creative direction / marketing collateral Evaluator to assess AI-generated work for quality. The ideal candidate...  ...be proficient in Microsoft Office and Google Workspace. This contract position allows for remote work, offering compensation between... 
    Contract work
    Remote job
    Work at office

    Mercor

    New York, NY
    2 days ago
  • Mercor is seeking a Public-sector procurement / RFI response Evaluator to evaluate AI-generated artifacts and provide structured feedback. Candidates should have 5+ years of experience in public-sector procurement and be highly proficient with Microsoft Office and Google... 
    Contract work
    Remote job
    Work at office
    Flexible hours

    Mercor

    New York, NY
    3 days ago
  • $30 per hour

    Type: Hourly contract Compensation: $20-$30/hour Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems using defined rubrics and quality standards. Review multi-step agent workflows,... 
    Contract work
    Remote job
    Hourly pay

    Crossing Hurdles

    New York, NY
    22 hours ago
  • $39 per hour

    Position: Music Audio Expert - English (UK/Australia) Type: Hourly contract Compensation: $39 per hour Location: Remote Commitment: Part-time, 10+ hours per week Role Responsibilities Evaluate AI model output lyrics, voice generation, and other standards Score quality... 
    Contract work
    Remote job
    Hourly pay
    Part time
    10 hours per week

    Crossing Hurdles

    New York, NY
    4 days ago
  •  ...investments and an entrepreneurial drive fueled by our partnership with EQT. Website: Linkedin Job Title: Evaluator - Political Science Location: Remote (USA) Job Type: Contract Key Responsibilities Review and Validate AI Responses: Evaluator will work via our online platform... 
    Contract work
    Remote work
    Worldwide

    Straive

    New York, NY
    22 hours ago
  • About the Role We’re looking for contract, remote Turkish-speaking annotators to evaluate AI-generated content with a sharp eye for detail and cultural nuance. You’ll evaluate whether outputs are coherent, consistent, and aligned with real-world expectations in Turkish... 
    Contract work
    Temporary work
    Freelance
    Immediate start
    Remote work

    BAM Ventures

    New York, NY
    3 days ago
  • About the Role We’re looking for contract, remote French-speaking annotators to evaluate AI-generated content with a sharp eye for detail and cultural nuance. You’ll be making judgment calls on whether outputs actually make sense, stay consistent, and reflect real-world... 
    Contract work
    Temporary work
    Freelance
    Immediate start
    Remote work

    BAM Ventures

    New York, NY
    3 days ago
  • $20.3 per hour

    Contract type: Freelance Hourly rate: $20.30 Language: English (US) Estimated volume: 8 - 10 hours Start date: The project runs on...  ...looking for qualified raters to support a project focused on evaluating personalized search and location recommendations. This role involves... 
    Contract work
    Hourly pay
    Freelance
    Immediate start

    Dorado

    New York, NY
    2 days ago
  • $50 - $190 per hour

     ...Dorsey . Position: MCP & Plug In Connectors Expert Type: Contract Compensation: $50–$190/hour Location: Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal health... 
    Contract work
    Hourly pay
    For contractors
    Summer work
    Remote work
    Trial period

    Mercor

    New York, NY
    4 days ago
  • AuraOne is seeking a remote Spoken Instruction Conversation Evaluator to review prompts and responses against its quality rubric. You...  ...judgments across long batches with attention to detail and clear reasoning. Remote US-eligible, hourly contract. #J-18808-Ljbffr AuraOne
    Contract work
    Remote job
    Hourly pay
    For contractors

    AuraOne

    New York, NY
    12 hours ago
  •  ...company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least 3 years... 
    Contract work
    Remote job
    Flexible hours

    Handshake

    New York, NY
    4 days ago
  • $95k - $120k

     ...precision, and long-term partnership. Overview The Sr. Human Resources Generalist for TDS is responsible for supporting the...  ...records in HRIS systems. Track assignment timelines and upcoming contract completions. Generate HR reports related to hiring activity,... 
    Contract work
    Full time
    Temporary work
    For contractors
    H1b
    Relocation

    Total Deployment Solutions

    New York, NY
    1 day ago
  •  ...SolidWorks and related CAD software, and professional proficiency in Mandarin Chinese, to support AI research through flexible, hourly contract work. This ongoing, project-based opportunity is fully remote and asynchronous, with no minimum hour requirement; many... 
    Contract work
    Remote job
    Hourly pay
    Flexible hours

    Handshake AI

    New York, NY
    1 day ago
  • Welo Data is seeking a detail-oriented Data Labeling Associate to evaluate AI outputs and improve system behaviors in real-world scenarios. The ideal candidate will have native French proficiency and a university degree. This full-time role, based in New York City, involves... 
    Full time

    University of Delaware

    New York, NY
    11 hours ago
  • $75 per hour

    Position: Sociology Teachers, Postsecondary Type: Hourly contract Compensation: $45-$75/hour Location: Remote Commitment: 10-40 hours/week...  ...through AI training initiatives for postsecondary education. Evaluate and review AI-generated coursework, assignments, and simulated... 
    Contract work
    Remote job
    Hourly pay

    Crossing Hurdles

    New York, NY
    22 hours ago
  • $150 per hour

     ...Investment Banking Expert Type: Contract Compensation: $150–$220/hour Location...  ...analyses. Score AI-generated and human work samples against criteria with...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    3 days ago
  • $100 - $150 per hour

     ...Position: B2B Sales Expert Type: Contract Compensation: $100–$150/hour Location...  ...emails. Score AI-generated and human work samples against criteria with...  ...~ Prior experience with AI training , evaluation, or human-data projects. Application... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    5 days ago
  • $50 per hour

     ...Dorsey . Position: Spanish (Mexico) Audio Generalist Evaluator Expert Type: Contract Compensation: $50/hour Location: Remote...  ...students or recent graduates. Background in linguistics, humanities, social sciences, journalism, translation/localization... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $50 per hour

     ...Jack Dorsey . Position: Hindi Audio Generalist Evaluator Expert (San Francisco Bay Area) Type: Contract Compensation: $50/hour Location: San...  ...recent graduates. Background in linguistics, humanities, social sciences, journalism, or technical disciplines... 
    Contract work
    Summer work
    10 hours per week

    Mercor

    New York, NY
    6 days ago
  • FAWE seeks to recruit a Multimedia Communications Assistant on a one-year fixed-term contract to support the establishment and day-to-day operation of FAWE Studio. Working within the Communications Unit, the successful candidate will support podcast production, multimedia... 
    Contract work
    Fixed term contract

    FAWE

    New York, NY
    2 days ago
  • A remote AI research organization is looking for skilled LMMS users to evaluate AI-generated content and provide feedback. This flexible, hourly contract role allows you to utilize your experience in music production and composition, making it ideal for music producers... 
    Contract work
    Remote job
    Hourly pay
    Flexible hours

    Handshake

    New York, NY
    22 hours ago
  • YO IT Consulting is seeking an AI Trainer & Evaluator for a remote contract role. You will apply your domain expertise to train next-generation AI systems, shaping how models learn, reason, and perform through high-quality real-world input. Responsibilities include scoring... 
    Contract work
    Remote job

    YO IT Consulting

    New York, NY
    4 days ago
  •  ...Licensed Psychologists (PhD or PsyD) - Pre-Hire Psychological Evaluators About National Testing Network (NTN): National Testing Network...  ...Psychologists (PhD or PsyD) to join our growing national network of contracted pre-employment psychological evaluators. This fully remote... 
    Contract work
    Remote work
    Flexible hours

    National Testing Network

    New York, NY
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Human Evaluator, Studio (Contract). Be the first to apply!