Japanese-Speaking AI Evaluation Specialist

Role Description

We are looking for a Japanese-Speaking AI Evaluation Specialist to join us as we build cutting-edge technology solutions! This is your opportunity to be part of a team that is committed to delivering best in class service to our customers.

In this role, you’ll help assess and improve the quality of AI-generated responses. You will analyze how AI systems interpret and respond to real user queries in Japanese, ensuring outputs are accurate, relevant, and culturally appropriate. This is not a translation role—it’s focused on evaluating how AI understands and communicates in Japanese.

Responsibilities:

• Evaluate and compare AI-generated responses in Japanese
• Perform side-by-side analysis of outputs from different AI systems
• Assess responses for accuracy, relevance, clarity, and instruction-following
• Identify nuances in meaning, tone, and cultural context
• Apply structured annotation guidelines consistently
• Work with datasets of real user queries

Qualifications

• Native or professional fluency in Japanese
• Strong English reading comprehension
• Analytical thinking and attention to detail
• Experience with structured evaluation or guidelines

Requirements

• Background in linguistics, translation, or localization (preferred)
• Experience with data annotation, AI evaluation, or search relevance (preferred)

Benefits

• Medical, dental, and vision coverage
• Flexible Spending Account
• 401k program
• Competitive PTO offerings
• Parental Leave
• Opportunities for professional growth and development

Salary Range

At Blueprint, we strive to offer competitive pay that reflects the value of our team members. Compensation for this role is influenced by a variety of factors, including skills, education, responsibilities, experience, and geographic market.

For candidates based in Washington State, the anticipated salary range is $30.00 to $35.00 USD/hour, with a midpoint of $32.50 USD/hour. Please note that we typically do not hire new employees at the top of the posted range. Actual starting pay will be determined based on experience, skills, and internal equity. The final salary and job title may vary depending on the selected candidate’s qualifications and could fall outside the stated range.

Equal Opportunity Employer

Blueprint Technologies, LLC is an equal employment opportunity employer. Qualified applicants are considered without regard to race, color, age, disability, sex, gender identity or expression, orientation, veteran/military status, religion, national origin, ancestry, marital, or familial status, genetic information, citizenship, or any other status protected by law.

If you need assistance or a reasonable accommodation to complete the application process, please reach out to: View email address on remotive.com

Location

Remote - USA