Korean Language Reviewer for AI Session Evaluation

Role Overview

Evaluate recorded interactions between users and a voice-and-camera AI assistant, using your Korean language judgment to identify inaccurate visual claims and assess response quality. Your evidence-based written feedback will help prioritize improvements to the assistant.

Key Responsibilities

• Review recorded sessions where a user speaks with an AI assistant while streaming their surroundings through a camera.
• Compare the assistant''s responses with the video feed and identify hallucinated or fabricated visual details.
• Score sessions against a detailed rubric covering visual interpretation, accuracy, relevance, response timing, and use of spatial or directional guidance, such as instructions to move left or raise a finger.
• Assess whether the assistant gave timely, proportionate warnings when an answer could affect health, physical safety, or financial security.
• Write detailed comments that support each score with specific evidence from the session.
• Document the conversation turn by turn in the intake form.

Qualifications

• Native or native-level Korean fluency, spoken and written, with clear and precise written Korean.
• Professional English reading comprehension, as the scoring guide is in English.
• An Android phone with a functioning camera and microphone, which is required for this project.
• Strong attention to detail and the ability to apply a rubric consistently without replacing it with personal criteria.

Preferred Qualifications

• Experience in data annotation, linguistic quality assurance, content moderation, or AI model evaluation.
• Training in linguistics, translation, or interpreting.
• Ability to recognize Korean that sounds unnatural or non-native across regional registers.

Work Terms

• Remote, hourly engagement.
• Immediate start.
• High work volume is expected from August 26 through August 31.
• Some sessions involve safety-sensitive scenarios, including identifying medication, checking whether food has spoiled, crossing a street, or reading bank card details. The work contains no graphic or violent content, but includes evaluating whether risk warnings were needed and appropriately delivered.

Compensation

$24 to $30 per hour.

Application Process

• Submit a resume or relevant background information to apply.
• Qualified applicants may be asked to complete a short calibration exercise using sample sessions. It evaluates scoring consistency and the usefulness of written feedback, not speed.