Repository Navigation Evaluation Specialist [Remote]
AuraOne Human Data
Remote
- Remote job
Repository Navigation Evaluation Specialist is a remote review track for evaluating AI outputs across repository navigation evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.
Why this role matters
Repository Navigation Evaluation specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.
Responsibilities
- Review AI outputs against current repository navigation evaluation specialist operations workflows, playbooks, and firm policy for Repository Navigation Evaluation Specialist assignments.
- Grade tone, escalation logic, and stakeholder fit on a structured rubric.
- Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
- Capture the right next step so the modeling team can train on it.
- Adjudicate disputed treatments against published playbooks or firm guidance.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Direct working experience in repository navigation evaluation specialist operations on real teams for Repository Navigation Evaluation Specialist work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the policy or workflow being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Grade a model's response to a real repository navigation evaluation specialist operations ticket and rate workflow, tone, and escalation.
- Flag a missed escalation with the right severity tag and corrected next step.
- Adjudicate a disputed playbook call between two reviewers using firm guidance.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Prior experience training, calibrating, or QA-ing operations teams.
- Familiarity with AI-assisted workflow tooling and its failure modes.
- Bilingual experience for cross-region operations.
Skills
- Operational review
- Policy adherence
- Workflow judgment
- Stakeholder communication
- Repository Navigation Evaluation specialist operations
- Code review
- Software testing
- Agent evaluation
- Repository
- Navigation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the Repository Navigation Evaluation Specialist [Remote] in Remote vacancy
- ...Monitoring and Evaluation Support for El Salvador Team Lead Peace is our commitment. The United States Institute of Peace represents... ...working closely with Home Office technical staff. Maintain a repository of project lessons learned, offering strategic recommendations...SuggestedFull timeContract workFor contractorsWork experience placementWork at officeLocal areaWork from homeWorldwideHome officeNight shift
US Institute of Peace
Washington DC1 day ago $100k - $300k
...employer recognized by the Detroit Free Press as a Top 100 Workplace and honored as a Crain's Cool Place to Work. Housing Navigation Specialist Department: Housing Navigation Reports to: Housing Navigation Supervisor Employment Status Classification: Full-...SuggestedFull timeTemporary workWork at officeRemote workWork from homeHome officeFlexible hoursCommunity Housing Network
Troy, MI1 day ago- ...given the means and mentorship needed to succeed, and your creativity will be rewarded.About the PositionThe Monitoring and Evaluation (M&E) Specialist will provide technical support to the Office of Monitoring and Evaluation (OME), Division of Global Health Protection (...SuggestedWork at officeRemote work
- ...Government Agencies. Learn More About ProSidian Consulting at .Job DescriptionProSidian Seeks a Infrastructure Project Evaluation and Coordination Specialist [DOE0139138] for Program Support on a Exempt W2: No Overtime Pay Basis Contract Contingent generally located across...SuggestedFull timeContract workTemporary workFor contractorsFor subcontractorWork at officeRemote workFlexible hours
$75.22k - $94.02k
...Impact Evaluation Specialist IIWhen you join Wounded Warrior Project® (WWP), you're committing to making a difference. We make a commitment to you, too – helping you to find that spark, ignite your passion to serve, and embark on a career with meaning and purpose.The Wounded...SuggestedTemporary workWork at officeWounded Warrior Project
Chicago, IL4 days ago$70k - $140k
...Program Evaluation SpecialistWe are seeking driven, thoughtful candidates with experience in program evaluation to support impactful initiatives... ...the U.S. Department of State (DOS). As a Program Evaluation Specialist with Censeo, you will support quantitative and qualitative...Full timeWork at officeRemote workWork from homeFlexible hoursNight shiftCenseo Consulting Group
Washington DC1 day ago$70k - $95k
...Patient Navigation Specialist When you join the growing BILH team, you're not just taking a job, you're making a difference in people's lives. The Patient Navigation Specialist supports Beth Israel Lahey Health (BILH) and its Community Hospital Division by providing...Remote work3 days per week$20.45 - $22.5 per hour
...Financial Navigator Ensemble is a leading provider of technology-enabled revenue cycle management solutions for health systems, including... .../Medicaid advocate/eligibility, state case/eligibility specialist or other applicable roles Prior Patient Access or Physician...Local areaImmediate startRemote work$24 - $30 per hour
Mercor is seeking a Moonlight MCQA - Arabic Generalist to draft Arabic MCQs for an AI evaluation dataset. This contract role offers remote work with flexible hours and a process including AI interview based on your resume. Must hold a master's or PhD, 2-6 years of experience...Remote jobHourly payContract workFreelanceFlexible hours- Care Atlas in the United States seeks a compassionate Care Navigator to support Medicare patients with remote monitoring and chronic care management. You will review RPM data, call patients to guide their care, and document health updates in the EHR while escalating urgent...Remote jobHourly payFull timeFlexible hours
$5,540 - $5,780 per month
...Reporting to the Director of Graduate Admissions and Program Evaluations (GAPE) in the College of Graduate Studies, this position is responsible... ...are completed efficiently and on schedule. The Evaluation Specialist interprets and applies complex policies, handles exceptions,...Permanent employmentFull timeWork experience placementInternshipWork at officeVisa sponsorship- OpenTrain AI is seeking a GPU Kernel Evaluation Expert to assess the quality, correctness, and completeness of GPU and accelerator kernel development tasks used to train and evaluate frontier AI models. You will evaluate numerical correctness, benchmarking fairness, task...Remote jobContract work
- ...professional to join a GenAI red-team within a leading AI lab network. The role involves probing frontier models, designing robust evaluation tasks, and documenting findings for reproducibility. This full-time W-2 position is remote within the United States, requiring...Full timeRemote work
- ...consulting firm in Colby, Kansas, is seeking a Nurse Practitioner or Physician Assistant to conduct examinations for veteran compensation evaluations. Responsibilities include conducting assessments, reviewing medical histories, and completing reports, all while collaborating...Remote jobWork from homeFlexible hours
$5,540 - $5,780 per month
...Reporting to the Director of Graduate Admissions and Program Evaluations (GAPE) in the College of Graduate Studies, this position is responsible... ...are completed efficiently and on schedule. The Evaluation Specialist is responsible for interpreting and applying complex policies...Permanent employmentFull timeWork experience placementInternshipWork at officeImmediate startVisa sponsorship- The Contract Management Specialist (\u201cCMS\u201d) will be responsible... .... This includes drafting, evaluating, and executing contracts,... ...process, and managing contract repository and reporting. The CMS... ...communication and interpersonal skills Navigate responses to redlines to...Full timeContract workPart timeWork at officeLocal areaRemote workWork from homeRelocationMonday to Thursday
- YO AI Labs is seeking experienced Producers across film, television, digital, or live events to support an AI evaluation project in a fully remote capacity. You will craft realistic industry scenarios, evaluate AI outputs, and develop benchmarks that reflect real-world...Remote work
- ...our purpose. Every day, we help clients navigate complexity, support recovery, and deliver... ...retaining the advice of outside specialists.Handle claims consistent with clients’ and... ...Claims Adjuster team, you will:Investigate, evaluate, and resolve complex general liability claims...Local areaRemote work
- ...Corporation is seeking a Senior Dissemination Specialist (Team Lead) to support INSCOM Data... ...developed Create Organize Retrieve and Evaluate (CORE) tool, Government-developed DIA... ...product(s) into the approved intelligence repositories.Collaborate and work with other IC...Contract workRemote work
$21 - $28.3 per hour
...payers. They are comfortable working directly with patients and can navigate sensitive financial conversations with empathy and clarity,... ...and career success. These competencies are used to evaluate performance, make hiring decisions, identify development needs,...Hourly payRemote workShift workMass General Brigham
Boston, MA23 hours ago$230k - $270k
...Safeguards Enforcement Analyst, Safety Evaluations Remote-Friendly (Travel-Required) | San Francisco, CA | Washington, DC; San Francisco... ...role requires someone who is detail-oriented, comfortable navigating ambiguity, and capable of coordinating across teams to break...Work at officeRemote workVisa sponsorshipFlexible hoursShift workAnthropic
New York, NY16 hours ago$107.6k - $130k
...and the final offer will depend on an evaluation of the selected candidate's experience,... ...for:ASCO is seeking a Senior Guideline Specialist to help lead the development and modernization... ...to evaluate conflicting clinical data, navigate methodological ambiguity, and drive...Full timeWork at officeRemote workWorldwideFlexible hours$19.39 per hour
...careers. We are seeking a Bilingual Outreach Specialist who enjoys community engagement,... ...recruitment plan. Prioritize and periodically evaluate the effectiveness of recruitment and... ...an d emails from possible students, and navigates them to the proper solution. NON-ESSENTIAL...Hourly payFull timeSeasonal workWork at officeLocal areaRemote workNight shiftAfternoon shiftKlamath Community College
Klamath Falls, OR8 days ago- A national recruiting firm is seeking an Online Candidate Specialist in Austin, TX. The role involves evaluating resumes for technical positions, partnering with recruiters, and maintaining data integrity in an applicant tracking system. Ideal candidates will have 2-4...
- ...Office Specialist Vi: Transcript Evaluator St. Edward's University of Austin, Texas invites applications for the position of Office Specialist VI: Transcript Evaluator. The Office Specialist VI: Transcript Evaluator provides accurate, timely evaluations of academic...Temporary workInternshipWork at officeImmediate startFlexible hours
$108.48k - $184.41k
Senior Specialist, C&I BuildingsLocation: US RemoteReady to make a difference? To support our... ...grid-interactive efficient buildings.Evaluate emerging technologies, market trends, and... ...clients solve their most complex challenges, navigate change, and shape the future.We can only...Full timeContract workWork experience placementWork at officeLocal areaRemote workFlexible hours- ...company initiatives. The Senior Sourcing Specialist will lead the assessment of the... ...success include the ability to prioritize, navigate nebulous situations, build consensus across... ...operations, legal concepts, and risk evaluation will enable this individual to lead the...Ongoing contractFull timeContract workTemporary workWork experience placementRemote workVisa sponsorshipRelocation package
$70 - $110 per hour
...Role Overview Evaluate and help improve AI systems by applying practical project management expertise to AI-generated content and workflows... ...~4+ years of experience as a Project Management Specialist, Project Manager, Technical Project Manager, Program Manager,...Hourly payImmediate startRemote workSaidGig
United States10 days ago- Rise Data Labs is seeking advanced Mathematics and Statistics experts to support the training and evaluation of state-of-the-art AI systems. We need subject-matter experts who can apply deep quantitative knowledge to AI evaluation problems, assess AI-generated reasoning...Remote jobContract workImmediate startFlexible hours
- ...AI-focused manufacturing project. You will apply your expertise in Fusion 360, CAM, CNC machining, and manufacturing workflows to evaluate processes and AI-generated recommendations. No prior AI experience is required, but you will provide critical feedback, develop evaluation...Remote jobFor contractors
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Repository Navigation Evaluation Specialist [Remote]. Be the first to apply!
Related searches
- configuration specialist Remote
- localization specialist Remote
- intervention specialist Remote
- investigations specialist Remote
- ecommerce specialist Remote
- workforce management specialist Remote
- community health promotion specialist Remote
- title specialist Remote
- infectious disease specialist Remote
- credit specialist Remote