Remote AI Policy Specialist - Nuanced Evaluations
Handshake
New York, NY
- Remote job
Handshake, a leading AI-driven education and policy company, seeks an AI Policy Generalist to convert complex customer policies into precise, well-reasoned evaluations of AI model behavior. You will read requests, model outputs, and conversation history, determining the correct policy classification while balancing guidance with practical expectations. You will carefully explain reasoning, calibrate with teams, and continually refine your judgments as new guidance emerges. #J-18808-Ljbffr Handshake
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Remote AI Policy Specialist - Nuanced Evaluations in New York, NY vacancy
$80 per hour
...-time opportunity focused on quality assurance for autonomous AI agents. You will analyze complex systems, review tasks for logic... ...analytical and detail-oriented skills, with experience in policy evaluation or logic puzzles preferred. Compensation can reach up to $80/hour...Remote workPolicyPart timeFlexible hoursMind Rift
United States3 days ago- Mercor is seeking an AI Safety Practitioner to evaluate AI-generated responses for safety, accuracy, and policy compliance. This remote contract role focuses on applying evaluation rubrics for RLHF and SFT, and collaborating with researchers to improve model alignment....Remote jobPolicyContract work
$35 - $42 per hour
Location: USA (Remote) Compensation: $35.00-$42.00 USD per hour(depending... ...Help Shape the Future of AI in Finance, Audit & Risk... ...growing network ofon-call AI Evaluation Specialistssupporting the development... ...Level Agreements (SLAs) Policy Compliance Regulatory Compliance...Remote workPolicyHourly payExtra incomePermanent employmentContract workTemporary workFlexible hours$70 per hour
...hour Join a cutting-edge AI research initiative focused on... ...thinking and communication skills to evaluate AI-generated responses across... ...tasks. This is a fully remote, contract-based opportunity... ...the ability to identify: Nuanced arguments Implicit meaning...Remote workHourly payWeekly payContract workFor contractorsFlexible hoursWeekday AI
United States4 days ago$70 per hour
We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex, policy-sensitive, and ambiguous ("grey-area") topics... .... Ability to consistently evaluate nuanced and policy-sensitive scenarios. Preferred...Remote jobPolicyWorldwide$35 - $40 per hour
...Generalist AI Evaluation Specialist is a remote evaluation track for reviewing generalist evaluation prompts and responses against AuraOne's quality... ...with rationale. Tag an unsafe response with the correct policy category and severity. Audit a 50-row batch for...Remote jobPolicyFor contractors10 hours per week$50 per hour
...Generalist AI Evaluation Specialist is a remote review track for evaluating AI outputs across generalists specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step...Remote jobPolicyFor contractorsWork experience placement10 hours per week- OpenTrain AI is seeking a Film and Television AI Evaluation Expert to craft challenging prompts and evaluate AI responses for accuracy and nuance. You will focus on film, television, streaming, and entertainment history while identifying knowledge gaps and helping build...Remote jobPart timeFor contractors
- ...Supporting diverse AI data and language projects, the hourly contractor AI Trainer and Evaluator will work remotely to generate content, annotate data, and evaluate AI responses... ...judgment in register, tone, and cultural nuance Ability to identify and explain corrections...Remote workHourly payFor contractors
Virtual Vocations Inc
United States5 days ago $20 - $60 per hour
...Role Overview Help train and evaluate next-generation AI systems by creating rigorous, real-world assessments... ...learn, reason, and perform. This remote contract opportunity is open to... ...first language. Ability to write nuanced, well-structured questions that assess...Remote workHourly payContract workFor contractorsSaidGig
United States14 days ago$20 - $80 per hour
...Role Overview Help improve next-generation AI systems by supplying precise, real-world evaluation, annotation, and feedback. This remote contractor role focuses on how AI models... ...communication skills, with the ability to explain nuanced feedback. Experience evaluating or...Remote workHourly payContract workFor contractorsSaidGig
United Statesa month ago- ...Position Summary This is a fully remote, hourly contractor role supporting AI data and language projects on a... ...support AI training datasets. LLM evaluation: reviewing AI-generated responses... ...terminology, tone, cultural nuance, and locale-specific details (units...Remote workHourly payFor contractorsFlexible hours
CNTXT AI
Brooklyn, NY15 days ago $135k - $175k
...building an enterprise AI capability that requires... ...Platform Integration Specialist will define how AI/ML capabilities... .... Ability to evaluate platform fit-for-purpose... ..., paid time off policy, discretionary annual bonus... ...is a hybrid of office/remote workingSkills:Cloud Platforms...Remote workPolicyFull timeWork experience placementWork at officeLocal areaWorldwideRelocationFlexible hours- YO AI Labs seeks Medical Evaluation Specialists, including medical students, residents, physicians, and biomedical professionals, to contribute... ..., synthesize medical information, and apply nuanced clinical judgment. Remote collaboration and strong English writing are...Remote job
- ...What Is An Ai Governance Specialist? An AI governance specialist develops and manages the policies, processes, and frameworks that guide how... ...a professional office or remote environment where they help... ...spend much of their time evaluating risks, improving governance...Remote workPolicyWork at office
Valence
United States2 days ago $100k - $150k
...Generative AI Specialist- Remote Bright Vision Technologies is a technology consulting and... ...of building reusable design patterns, evaluation frameworks, and developer tooling that... ...guardrails, safety filters, and policy enforcement patterns for LLM-powered products...Remote workPolicyFull timeH1bLocal areaImmediate startVisa sponsorshipBright Vision Technologies
United States2 days ago- ...AI Integration Engineer / Specialist Compensation: DOE AI Integration Engineer / Specialist... ...- implement guardrails, evaluation harnesses, and review processes that... ...with broader Mi-Case AI governance, policies, and strategic objectives. Metrics...Remote workPolicy
People, Data & Solutions
United States4 days ago - ...To support AI development, the part-time AI Evaluation Specialist will review and assess AI-generated outputs for quality and usability while collaborating with teams to refine evaluation standards in a remote contract role. Key responsibilities Review and critically...Remote workContract workPart time
Virtual Vocations Inc
United States5 days ago - Handshake is seeking an AI Policy Generalist to turn complex customer policies into consistent evaluations of AI model behavior. You will read user requests, model responses, and conversation history to determine the best policy category. We value precise distinctions,...Remote jobPolicy
$120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Compensation: $120–$200/hour Location: Remote Role Responsibilities Evaluate complex technical tasks using deep language expertise...Remote workHourly payWeekly payFull timeContract workFor contractorsSummer workMercor
Remote1 day ago- ...you! Role Overview The AI Web Optimization Specialist is responsible for... ...performance, generate hypotheses, evaluate content and accelerate optimization... .... Location Remote Mexico Responsibilities... ...Equal Opportunity The policy of Forcepoint is to provide...Remote workPolicy
Forcepoint
Austin, TX2 days ago - ...What Is An Ai Evaluation Specialist? An AI evaluation specialist assesses and tests artificial intelligence systems to ensure they perform... ...an AI evaluation specialist is typically an office-based or remote environment where they spend much of their time working with...Remote workWork at office
Valence
United States1 day ago - ...Applied AI Engineer (Part-Time, Remote – U.S.) Innovation is at the heart of what we do. At Implicit... ...quality through prompt engineering, evaluation frameworks, and retrieval... ...United States. Flexible vacation policy. Opportunity to build cutting-edge...Remote jobPolicyPart timeFlexible hours
Implicit
United States1 day ago - ...Prolific is seeking Product Designers and UX Specialists to join our Expert Network, contributing to the training and evaluation of cutting-edge AI models. This role requires expertise in usability, design systems, and user research, providing essential feedback on design...Remote workWork from homeFlexible hours
Prolific
Tucson, AZ5 days ago $30 per hour
...Prolific is seeking Advanced Dutch Speakers in Chicago, IL to train AI models. You will complete AI tasks and assess AI performance in... ...home with competitive rates and direct payment through PayPal. Pass the evaluation, and you can start within 15 minutes. #J-18808-Ljbffr...Remote workWork from homeProlific
Chicago, IL5 days ago- ...the future. Our services span AI Strategy, Data Intelligence, AI... ...This position is currently remote; however, in accordance with federal... ...requirements and company policy, there may be future changes to... ...retrieval-augmented generation, evaluation, and human-in-the-loop...Remote jobPolicyFull timeContract workWork at office
Anika Systems
Remote1 day ago - ...Senior Software Engineer â AI I \ \ Recruiter:... ...innovative AI use cases \\ Evaluate existing systems and propose architectural... ...with a mix of in\-office and remote work \\ Flexibility & Work\-Life Balance: \ Supportive policies, including work\-from\-...Remote workPolicyFull timeWork experience placementWork at officeFlexible hours
Symbiotic Services
New York, NY1 day ago - ...Prolific is seeking talented Product Designers and UX Specialists to join our Expert Network for training and evaluation of cutting-edge AI models. Candidates should have a relevant educational background and at least one year of experience in design-related fields. Responsibilities...Remote workWork from homeFlexible hours
Prolific
Arizona City, AZ1 day ago $100k - $120k
...AI Implementation Specialist The AI Implementation Specialist embeds directly with DISA's Uranium teams... ...and scalability as DISA grows. • Evaluate and pilot new AI, GIS, and business-... ...operational efficiency. • Abide by all policies and procedures established by DISA....Remote workPolicyContract workShift workDisa Technologies, Inc.
United States1 day ago- ...AI Trainer Position MCI is one of the fastest-growing tech-... ...are powered by both on-site and remote agents, leveraging advanced... ...This role is responsible for evaluating AI outputs, creating training... ...Disabilities Act (ADA), it is the policy of MCI and its affiliates to provide...Remote workPolicyWork at officeLocal area
AI Support Positions
Encino, CA5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Policy Specialist - Nuanced Evaluations. Be the first to apply!
Related searches
- ai scientist New York, NY
- ai data scientist New York, NY
- senior network engineer remote New York, NY
- business intelligence analyst remote New York, NY
- document specialist remote New York, NY
- chief of staff remote New York, NY
- it director remote New York, NY
- remote legal intern New York, NY
- remote accounts receivable New York, NY
- remote construction estimator New York, NY