AI Coding Evaluator
OpenKrill
Help shape the future of AI by evaluating the technical accuracy, language quality, and usefulness of AI-generated responses.
- Location: Remote
- Type: Independent contractor / Part-time
- Project duration: up to 3 months
ABOUT THE ROLE
At Anyone AI, we create high-quality training and evaluation data for frontier AI models. We’re looking for multilingual experts with coding knowledge and a solid understanding of JSON to evaluate AI-generated technical content and customer service responses. In this role, you’ll assess whether outputs are clear, helpful, natural, technically accurate, and aligned with project guidelines. Your evaluations will help improve both the customer experience and the technical reliability of AI systems.WHAT YOU’LL DO
- Review AI-generated customer service responses and technical content.
- Evaluate responses for clarity, helpfulness, natural language, and overall quality.
- Assess technical correctness using your coding knowledge.
- Read, validate, and troubleshoot JSON, identifying structural and formatting issues.
- Check outputs for formatting accuracy and adherence to project guidelines.
- Provide clear, well-reasoned feedback explaining your evaluations.
WHAT WE’RE LOOKING FOR
- Fluency in one or more of the supported language varieties listed below.
- A solid understanding of JSON, including syntax, structure, validation, and troubleshooting.
- Coding knowledge sufficient to assess technical content and identify errors.
- Strong judgment when evaluating customer service responses.
- Excellent attention to detail and strong written communication skills.
- Ability to follow detailed evaluation guidelines consistently.
SUPPORTED LANGUAGES/LOCALES
- Dutch — Netherlands
- Welsh — United Kingdom
- French — Canada
- German — Germany
- Italian — Italy
- Portuguese — Brazil
- Greek — Greece
NICE TO HAVE
- Previous experience in AI evaluation, quality assurance, or technical support.
- Experience working with APIs and structured data.
- A background in customer service, software development, or technical writing.
VOICE RESPONSE INSTRUCTIONS
When completing the voice response, speak naturally and spontaneously, as if responding in real time. Do not pre-plan, write, or script your answer—we are evaluating natural voice cadence. Responses may be up to one minute long. A few sentences are sufficient if that feels natural. #J-18808-Ljbffr OpenKrillVacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the AI Coding Evaluator in New York, NY vacancy
- Anyone AI is seeking multilingual evaluators to assess AI-generated responses for clarity, usefulness, and JSON accuracy. This independent contractor,... ...feedback to improve AI systems. Strong language skills and coding knowledge are essential. Experience with APIs and...SuggestedRemote jobPart timeFor contractorsFlexible hours
- Anyone AI Inc. is seeking multilingual coding experts to evaluate AI-generated responses on a part-time remote contract lasting up to 3 months. This opportunity supports frontier AI model improvement by providing precise feedback and evaluations. You will review AI-generated...SuggestedRemote jobContract workPart time
- Anyone AI is seeking multilingual coding experts to evaluate AI-generated responses. This part-time remote contract lasts up to 3 months and supports frontier AI model improvement. What you'll do: Review AI-generated customer service and technical content for clarity, correctness...SuggestedRemote jobContract workPart timeFor contractors
- AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually... ...communication. Generalists welcome, domain experts preferred. No coding or AI background needed. An active personal AI account (Free, Go,...SuggestedWork at office
- ...Job Description Job Description Role Title: AI Agent Evaluator Role Type: Contractor Location: United States We are seeking detail... ...recording. Preferred: Relevant domain expertise. No coding or AI development experience is required. Selection Process...SuggestedRemote jobFor contractorsWork at office
YO AI Labs
New York, NY11 days ago $90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate the quality and correctness of AI-... ...training and evaluation. Assess end-to-end coding sessions produced with AI-assisted...Remote jobContract workSummer work$400 per month
Obsidian is seeking contributors for a project with a leading AI research lab. The role involves using frontier AI coding agents to finalize and assess complex engineering tasks. Contributors will review code, identify bugs, and compare outputs from different models. Strong...- ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...Contract workTemporary workImmediate startRemote work
MERIT Beauty
New York, NY1 day ago - ...and quantitative analytical skills; ability to process large volumes of market information Proficiency in Excel with Visual Basic coding and/or practical Python experience Excellent communication skills for client and internal interactions Preferred qualifications, capabilities...Worldwide
$20 - $26 per hour
Prolific is seeking a fluent Punjabi speaker to act as an AI trainer/evaluator. You will listen to audio and compare it with text, rating naturalness and authenticity. The role requires strong linguistic intuition and the ability to provide precise feedback to help models...Remote jobHourly payWork from homeFlexible hours- ...educator to analyze assessments, provide constructive feedback, and guide students through AI/ML curricula. The role emphasizes collaboration and staying current with evaluation processes in a remote, flexible environment. The position requires advanced degrees in AI/ML...Remote jobPart timeFlexible hoursAfternoon shift
- ...averaging around 15 hours. Compensation is up to $5 per hour, with payments via PayPal or bank transfer. No prior experience is required, making it ideal for freelancers, students, or flexible workers seeking to contribute to AI model improvements. #J-18808-Ljbffr RWS TrainAIRemote jobHourly payPart timeFreelanceWork from homeFlexible hours
- Obsidian is seeking expert Evaluators for a remote, hourly role focused on reviewing AI-generated work products for quality and accuracy. The successful candidates will leverage their subject matter expertise to assess various documents, spreadsheets, and slide decks....Remote jobHourly payWork at office
$13 per hour
Rate: $13/hr Hours: 20 hours / week Experience: 1 - 3 years Location: Argentina | Belize | Bolivia + 15 more Help evaluate how an AI assistant uses information from connected Google apps to respond to personalized prompts. This remote project is for people based in...Hourly payFor contractorsRemote work$50 per hour
Prolific seeks Product Designers and UX Specialists to join our Expert Network for training and evaluating AI models with real design expertise. After a quick test, you may join Prolific as a participant and get paid for AI training tasks. Researchers pay up to $50 per...Remote jobHourly payFlexible hours- RWS Group is seeking a remote evaluator to help improve AI search results by providing detailed feedback on text, audio, images, and videos. You’ll work from home, simulating everyday searches to influence how major search engines respond to real users. Ideal candidates...Remote jobWork from home
$14.5 per hour
Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant, diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data leverages...Hourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hours- Feedinkoo is seeking a Graphic Designer to help train and improve AI models focused on UI/UX design, visuals, and user experiences. You will critique AI outputs, provide feedback, and guide model improvements to better reflect aesthetics and usability for designers. The...Remote work
- A leading AI research accelerator is hiring a position focused on contributing to projects that evaluate and enhance AI systems. You will design community service scenarios, write structured explanations, and evaluate AI accuracy. The ideal candidate will have 4+ years...Remote jobFull timeFor contractors
$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...Remote jobHourly payPart timeFlexible hours- Blueprint is seeking an English-language AI Response Labeler / Annotator to evaluate the quality of AI-generated responses. The role involves comparing outputs from different AI models, judging based on accuracy, relevance, completeness, reasoning, and safety, and applying...
- Volga Partners is seeking an AI Language Quality Evaluator fluent in Greek and English to assess localization accuracy across UI texts and app screens. This freelance, task-based role offers flexible remote engagement with on-demand tasks and no guaranteed hours. As part...Remote jobFreelanceFlexible hours
$70 per hour
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. We’re seeking US-based physicians (MD/DO) to evaluate clinical EHR vignettes for our medical AI evaluation program, primarily...Hourly payPermanent employmentPart time- Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across genres and rate it against detailed quality standards, labeling songs by genre, structure...Immediate startFlexible hours
- Remotebridge is seeking Cantonese Language Evaluators to join a remote contractor team. You will evaluate Cantonese text and speech to train AI systems, providing feedback and annotations to improve model outputs. No AI background required beyond domain expertise. You...Remote jobFor contractors
- Mercor seeks experienced brand designers to define grading criteria for real-world brand deliverables and to score AI-generated versus human work with detailed written justifications. You will apply evidence-based judgment, ensure reproducibility, and iteratively incorporate...
$15 - $35 per hour
Volga Partners is seeking Medical AI Evaluators who are currently pursuing medical education or completing supervised clinical training in General Practice, Family Medicine, Internal Medicine, NP, or PA pathways. You will evaluate AI interpretations of medical scenarios...Remote jobHourly payFor contractorsTraineeshipFlexible hours$24 per hour
Prolific is seeking an AI Trainer with advanced Tamil fluency to evaluate AI models' understanding of the Tamil language's emotional and cultural nuances. Responsibilities include assessing audio clips, reviewing tones for cultural relevancy, and ensuring quality control...Remote jobFlexible hours$13 per hour
RWS Group is seeking a flexible, part-time Search Quality Evaluator to help improve AI-driven search results. You will test everyday queries and provide feedback on text, audio, images, and videos from users across major search engines and platforms. This remote role is...Remote jobExtra incomePart timeFreelanceFlexible hours- Mercor is seeking experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards, working in Hindi and English. Responsibilities...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Coding Evaluator. Be the first to apply!
Related searches