AI Evaluation Specialist
Crossing Hurdles
A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities include ensuring clarity and empathy in evaluations, identifying weaknesses in AI reasoning, and contributing to a safe AI environment. Immediate availability post-onboarding is required for this role. #J-18808-Ljbffr
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Evaluation Specialist in New York, NY vacancy
- ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Dermatologist (MD) — Clinical Image Interpretation & AI Evaluation Type: Contract Compensation: $270/hour Location:...SuggestedContract workSummer workRemote work
Mercor
New York, NY4 days ago $60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...SuggestedContract workSummer workRemote workMercor
New York, NY4 days ago$20 - $160 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour Location...SuggestedContract workSummer workRemote workMercor
New York, NY4 days ago$229.9k - $262.4k
...AI Engineer 5 (SDK's: Gen AI Evaluation and MCP) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real‑time, personalized customer experiences...SuggestedFull timePart timeLocal area$197.3k - $245.6k
...per year Requirements: Bachelors degree in Computer Science, AI, Electrical Engineering, Computer Engineering, or a related... ...: Proficiency designing distributed systems for model training, evaluation, and online inference at petabyte scale. Preferred: Experience...SuggestedFull timeCapital One
New York, NY2 days ago$197.3k - $245.6k
AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time,...Full timePart timeLocal area$229.9k - $262.4k
...Overview Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning...Full timePart timeLocal areaCapital One
New York, NY16 days ago$400 per month
Obsidian is seeking contributors for a Frontier Code Agents project, focused on evaluating AI coding models in fraud and risk engineering. Candidates will use AI coding tools to handle complex tasks and provide technical assessments. The role requires 2+ years of experience...$100 - $150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...work , machine learning pipelines , and technical reports. Evaluate AI-generated or human-created work against established criteria...Contract workSummer workRemote workMercor
New York, NY4 days ago$84 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...jailbreaks, unsafe behaviors, hallucinations, and policy failures. Evaluate model robustness across misinformation, cyber, biosecurity,...Contract workSummer workRemote workMercor
New York, NY4 days ago$30 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...accuracy, relevance, response timing, and spatial language use. Evaluate if the assistant appropriately warned users about health, safety...Contract workSummer workImmediate startRemote workMercor
New York, NY4 days ago$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Remote Commitment: 20 hours/week Role Responsibilities Evaluate and enhance AI model capabilities in complex SWE tasks ....Hourly payWeekly payContract workFor contractorsSummer workRemote workMercor
New York, NY4 days ago- ...Capital One is hiring an AI Engineer 5 in New York, NY to partner across engineering, research, and product teams. You will design and deploy AI software components, including model training, inference, agents, and multi‑model workflows, leveraging Open Source and SaaS...
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work- ...time role, requiring strong analytical skills and independent work. The position involves creating historically relevant prompts, evaluating AI outputs, and contributing to research initiatives. Ideal candidates will hold a PhD in History or a related discipline and...Part timeRemote workFlexible hours
Crossing Hurdles
New York, NY1 day ago $37 per unit
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: AI Safety Specialist Type: Contract Compensation: $... ...Location: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated...Contract workSummer workRemote workMercor
New York, NY4 days ago$150k
...topics including Digital Transformation, Data & Digital Platforms, AI at Scale, Agile, Cybersecurity and Digitizing the Tech Function.... ...in ERP and similar solutions. Solution Architecture & Design Evaluating an organization’s IT, enterprise applications, data, analytics,...Work at officeLocal areaThe Boston Consulting Group
New York, NY1 day ago- Supporting ongoing AI and legal technology initiatives, the part-time AI Integration Specialist will work remotely on a project basis, focusing on chatbot development, integration... ...to identify workflow improvements and evaluate AI-driven legal processes Required...Contract workPart timeRemote workFlexible hours
- ...mature its enterprise artificial intelligence roadmap, the remote AI Specialist will serve as a foundational subject matter expert, managing technology portfolios, leading governance teams, and evaluating high-impact vendor tools in a permanent position. Key...Permanent employmentRemote work
- ...Remote Job Summary We are seeking experienced Senior Software Engineers to support an AI training project by creating reinforcement learning environments that evaluate AI models on complex software engineering tasks using Model Context Protocol (MCP) tools....Remote jobFor contractors
$18 per hour
...intelligence. What We Do Toloka connects individuals with Generative AI projects from leading tech innovators. Our mission is to unlock... ...part in online projects such as rating AI-generated content, evaluating factual accuracy, or comparing responses — when projects are...Part timeFreelanceRemote work$8 - $65 per hour
...production expert eager to shape the future of AI? Large‑scale language models are... .... We’re looking for video production specialists who live and breathe cinematography, directing... ...to our prompt engineering and evaluation metrics. A bachelor’s or master’s degree...Hourly payContract workFor contractorsFreelanceImmediate startRemote workInvisible Technologies Inc. Defunct
New York, NY23 hours ago$40 - $60 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...administration with deterministic rubrics for agent performance evaluation. Develop scenarios in patient access including scheduling...Hourly payWeekly payFull timeContract workFor contractorsSummer workRemote workMercor
New York, NYmore than 2 months ago$73 per hour
...CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment...Permanent employmentTemporary workPart timeFreelance10 hours per weekMind Rift
New York, NY3 days ago$110k
...distinctive style of hospitality. Who you are: The Data and AI Engineer will own the data infrastructure that powers reporting... ...and productionize agentic workflows – including retrieval, evaluation, and guardrails – so AI-generated outputs are reliable enough for...Full timeFlexible hoursUnion Square Hospitality Group
New York, NY10 days ago$164.6k - $213.1k
...impact and data into action, together.The OpportunityThe Data & AI Product Engineer is a hands-on technologist who sets the technical... ...or AI solutions.Experience designing data and AI architectures, evaluating and building platforms/vendors, and making build vs. buy...Full timeTemporary workWork at officeLocal areaRelocation packageShift work3 days per weekMetropolitan Life Insurance Company
New York, NY4 days ago$73 per hour
...collective intelligence to ethically shape the future of AI. What We Do The Mindrift platform connects specialists with AI projects from major tech innovators. Our... ...to learn how modern AI systems are tested and evaluated? This is a flexible, project‑based opportunity...Permanent employmentPart timeFreelanceRemote workFlexible hoursMind Rift
New York, NY3 days ago$60 per hour
...in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment...Permanent employmentTemporary workPart timeFreelance10 hours per weekMind Rift
New York, NY3 days ago$20 per hour
...opportunity for independent contributors who are curious about artificial intelligence and excited to learn how advanced AI models are built, evaluated, and improved. These opportunities may be suitable for people who are early in their academic journey, exploring...Hourly payContract workFor contractorsFreelanceH1bRemote workFlexible hours$136k - $192k
...company policy. Job Purpose The AI Value Office Consultant is a US-based individual... ...assumptions and work effectively with specialists, without being a hands-on data scientist... ...Grid utilizes an assessment that evaluates the job qualifications/characteristics using...Full timeWork at officeLocal areaFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Specialist. Be the first to apply!
