Vulnerability Triage Security Evaluator [Remote]
AuraOne Human Data
- Remote job
Vulnerability Triage Security Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team can patch the gap.
Why this role matters
Adversarial evaluation is how AuraOne hardens AI models before they ship to customers. Reviewers think like attackers and write up failures with enough rigor that the modeling team can reproduce, fix, and regress-test them.
Responsibilities
- Design adversarial prompts that probe known weakness classes (jailbreak, policy bypass, prompt injection) for Vulnerability Triage Security Evaluator assignments.
- Document every successful attack with reproduction steps and the policy clause it violated.
- Score model defenses across single-turn and multi-turn conversations.
- Triage emerging attack vectors and route them to the safety team with severity ratings.
- Maintain a personal library of attack patterns and propose new red-team rubrics.
- Calibrate against the broader red-team cohort to keep coverage and severity consistent.
Qualifications
- Demonstrated experience red-teaming AI systems, security research, or adversarial ML work for Vulnerability Triage Security Evaluator work.
- Strong written communication — your reports become the patch ticket.
- Comfort working in policy-grey areas with clear documentation of what was attempted and why.
- Familiarity with prompt-injection, jailbreak, and policy-bypass taxonomies.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Construct a 5-turn adversarial conversation that bypasses a specific policy clause and write up the patch ticket.
- Score a model's defenses against a known jailbreak pattern across 20 variants.
- Propose a new red-team rubric category after spotting an emerging attack vector.
- Reproduce a failure another reviewer reported and confirm the severity tag.
Nice to have
- Background in offensive security, AppSec, or trust & safety operations.
- Experience publishing or reproducing public adversarial-ML research.
- Multilingual fluency for cross-language attack testing.
Skills
- Adversarial prompting
- Red-team analysis
- Policy taxonomy
- Failure documentation
- Vulnerability Triage Security evaluation
- Security review
- Adversarial testing
- Threat modeling
- Vulnerability
- Triage
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
- ...Nursing Triage Safety Evaluator is a remote clinical-review track for evaluating AI outputs that touch nursing. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so the...SuggestedRemote jobHourly payFor contractors10 hours per week
$50k
...for considering opportunities with us! Job Title Casualty Evaluator Requisition Number R8039 Casualty Evaluator (Open) Location... ...assignments through designated queues in the claim system. Triage is self-managed and not automated in the claims system. Making...SuggestedHourly payWork experience placementH1bWork at officeRemote workWork from homeFlexible hoursWeekend work$187.18k - $247.25k
...: Yes Job Description: The Expert-Level Network Evaluator / System Vulnerability Analyst provides advanced technical research, systems analysis... ...into authoritative Information Technology (IT) doctrine, security-focused TTPs, HUMINT-enabling methodologies, and cyber...SuggestedFull timeTemporary workPart timeImmediate startRemote workWorldwideFlexible hours- ...safeguarding and refining AI systems. This role centers on evaluating clinical precision, safety standards, and overall response quality... ...stress-testing (Red-Teaming) to identify potential vulnerabilities and safety concerns before public release. We are looking for...SuggestedPart timeFor contractorsWork at officeLocal areaImmediate startRemote workMonday to FridayFlexible hours
- ...Cloud Security AI Evaluator is a remote evaluation track for reviewing cloud security ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can...SuggestedRemote jobHourly payFor contractors10 hours per week
- ...Security Policy Bypass Security Evaluator is a remote review track for evaluating AI outputs in policy review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can...Remote jobHourly payFor contractors10 hours per week
$37.5 per hour
...generated ad copy for various advertisers looking for placements within Amazon originals. In this role you will be responsible for evaluating advertising copy generated by a large language model (LLM) to ensure that it meets their benchmark in quality, structure, brand...Contract workTemporary workRemote work- ...Staffco, LLC · MENTAL HEALTH SERV.Poughkeepsie, NYAllied Health Prof/TechnicalPer DiemAll ShiftsvariedJob SummaryThe Mental Health Evaluator (MHE) is a critical role dedicated to delivering comprehensive, clinically appropriate, culturally competent, and trauma-informed...
- Overview At PAM Health, we care for chronically and critically ill patients who require extended hospital care. PAM Health has over 80 hospital locations and employs over 11,000 people across the country. Our teams work together to deliver the highest level of compassionate...Full timeLocal area
$85 per hour
...complete a 15-20-minute calibration task hosted remotely on our platform. During the session, you will review several image pairs and evaluate how well each matches a set of provided brand guidelines. You will then submit your selections and feedback through a structured...Hourly payRemote work$80 - $120 per hour
...This role is for one of our clients Compensation: $80 - $120 per hour We are hiring expert Evaluators in Special education / IEP to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality...Hourly payContract workFor contractorsWork at officeRemote work- Appsierra Group is seeking an experienced Registered Nurse to evaluate and rank AI-generated clinical content for safety and accuracy.... ...contractor role focuses on nursing expertise to ensure education, triage guidance, and clinical documentation meet high standards. You...Remote jobFor contractors
$80 - $120 per hour
...Special Education Iep Evaluator This role is for one of our clients. Compensation: $80 - $120 per hour. We are hiring expert Evaluators in special education / IEP to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy...Hourly payRemote work- ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...Contract workTemporary workImmediate startRemote work
$9 - $30 per hour
ALTA Language Services, Inc. is looking for a remote Testing Evaluator in Atlanta, GA. This part-time position involves assisting with language proficiency examinations and evaluating tests according to set criteria. Candidates should have strong communication skills, professionalism...Part timeRemote work- ...Seeking a full-time Remote AI Research Evaluator with a PhD in Quantitative Finance to assess and enhance AI models' capabilities in financial reasoning and quantitative analysis through flexible, contract-based work. Key responsibilities Assessing the factuality and...Full timeContract workRemote workFlexible hours
- ...Shore Staffing, Inc. is seeking aMaryland Licensed Psychologistto join our growing behavioral health evaluation team. This flexible opportunity allows you toconduct comprehensive psychological evaluationsorprovide remote clinical supervision and oversight to a Psychology...ReliefPrivate practiceRemote workFlexible hours
$16 - $20 per hour
...Research Evaluator Position The research team at Penn State Ross and Carol Nese College of Nursing is hiring part-time research evaluators for projects focused on dementia care in assisted living settings. The research evaluator will assist with in-person recruitment...Hourly payPart timeSummer work$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...Hourly payPart timeRemote workFlexible hours$120 - $175 per hour
Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...Contract workSummer workRemote work- ...Luxury Brand Evaluator Turn your passion for luxury into a career opportunity. Explore the world of premium brands and make a lasting impact in fashion, beauty, jewelry, or automobiles. Join CXG, the global leader in customer experience, and work alongside iconic names...Remote workWorldwideFlexible hours
$14.5 per hour
...training and domain experts. Key Responsibilities Analyze search result performance and provide insights on relevance and quality. Evaluate and rate the effectiveness of search engine results to ensure they meet user expectations. Offer feedback and recommendations to...Hourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hours- ...Folioworks in California seeks an experienced MD/DO to conduct independent medical-legal evaluations for injured workers under the workers' compensation system. You will review records, perform examinations as needed, and provide objective QME opinions. In-person and remote...Remote workFlexible hours
$75 per hour
...Role Responsibilities Develop and deliver sociology content through AI training initiatives for postsecondary education. Evaluate and review AI-generated coursework, assignments, and simulated academic papers. Create lectures, learning modules, and case studies...Hourly payContract workRemote work$100 per hour
...Job Description Job Description WELLBOUND Certified Home Health Agency is seeking a dedicated Physical Therapist (PT) Field Evaluator to join our growing team. In this role, you will deliver high-quality therapeutic services that support patients’ mobility, self-care...Work from home$425 per month
...values, we welcome you to explore our contractor opportunities! Requirements: This position pays up to 50% of revenue , 1-Day Evaluation $350 for remote and $425 for in person, 2-Day Evaluation $742.00 for in person. POSITION SUMMARY The Vocational Evaluator...Full timeFor contractorsWork experience placementWork at officeLocal areaRemote work- ...Description Description: Position Summary The Certified Evaluator delivers a convenient, professional, five-star selling... ...guidelines, and ensures purchases are accurately documented, securely transported, and prepared for resale. Essential Duties and...Minimum wageTemporary workWork at officeLocal areaRelocation
- A premier customer experience assessment company seeks a Freelance Luxury Brand Evaluator in Newtown Square. This role requires evaluating high-end automotive brands through in-store visits or online assessments. Candidates should be 18+, possess a strong understanding...FreelanceFlexible hours
- AI Trainer Jobs is conducting a fully remote study for image quality assessment and brand alignment. You will evaluate pairs of photographs and judge alignment with brand guidelines, then complete a calibration task before onboarding for additional tasks. Payments are...Remote jobFor contractors
- ...will be doing: Choose your assignments - align your missions with your personal preferences and profile Visit luxury stores or evaluate online - engage with sales associates in the stores or online from the comfort of your home Observe carefully - check the ambiance...Contract workWorldwideFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Vulnerability Triage Security Evaluator [Remote]. Be the first to apply!




