Software Engineers: Share Your AI Coding Sessions for Model Evaluation [Remote]

AuraOne Human Data

Remote job

Software Engineers: Share Your AI Coding Sessions for Model Evaluation is a remote review track for evaluating AI outputs across builders developers share your ai coding sessions specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.

Why this role matters

Builders Developers Share Your AI Coding Sessions specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.

Responsibilities

Review AI outputs against current builders developers share your ai coding sessions specialist operations workflows, playbooks, and firm policy for Software Engineers: Share Your AI Coding Sessions for Model Evaluation assignments.
Grade tone, escalation logic, and stakeholder fit on a structured rubric.
Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
Capture the right next step so the modeling team can train on it.
Adjudicate disputed treatments against published playbooks or firm guidance.
Maintain reviewer-quality scores in inter-rater calibration cycles.

Qualifications

Direct working experience in builders developers share your ai coding sessions specialist operations on real teams for Software Engineers: Share Your AI Coding Sessions for Model Evaluation work.
Comfort applying multi-page rubrics consistently across long batches.
Clear written reasoning that names the policy or workflow being applied.
Strong attention to detail and the ability to flag when a prompt itself is the problem.
Reliable async availability for at least 10 hours per week.

Example tasks

Grade a model's response to a real builders developers share your ai coding sessions specialist operations ticket and rate workflow, tone, and escalation.
Flag a missed escalation with the right severity tag and corrected next step.
Adjudicate a disputed playbook call between two reviewers using firm guidance.
Audit a 25-row batch for rubric consistency and report drift to the program lead.

Nice to have

Prior experience training, calibrating, or QA-ing operations teams.
Familiarity with AI-assisted workflow tooling and its failure modes.
Bilingual experience for cross-region operations.

Skills

Operational review
Policy adherence
Workflow judgment
Stakeholder communication
Builders Developers Share Your AI Coding Sessions specialist operations

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

$200 / hr

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Apply

Vacancy posted 2 days ago

Similar jobs that could be interesting for youBased on the Software Engineers: Share Your AI Coding Sessions for Model Evaluation [Remote] in Remote vacancy

Software Engineer - Hosted Model Infrastructure
$145k - $200k
...world’s leading software for data-driven... ...are a software engineering team with expertise... ...in enabling ML models in production. We deploy AI models to run in... ...from application code through... ...ability to quickly evaluate and integrate new... ...submit a payment or share financial information...
Suggested
Full time
Work experience placement
Work at office
Remote work
Work from home
Relocation package
Palantir Technologies
Washington DC
1 day ago
Software Engineers: Paid Code Review for AI Agent Evaluation
$135.2k
...paid study on the coding environments and programming... .... Creating robust evaluation harnesses ensures that AI models are tested against realistic software engineering scenarios. This... ...During this remote session, you will review a... ...and screen-sharing exercises. Who This...
Suggested
Hourly pay
Contract work
Remote work
Terac
Remote
11 days ago
Builders & Developers: Share Your AI Coding Sessions
$500 per month
...hiring a remote Builders & Developers: Share Your AI Coding Sessions contractor (pay $500 total). Contribute to frontier AI research and evaluation work. Ideal candidates: Build or... ...project using the open-weight and Gemini models we provide; Choose which model to use...
Suggested
Temporary work
For contractors
Remote work
Gridnaut Recruiting
Remote
5 days ago
Software Engineer L5/L6 — Model Evaluations & Data Curation (MEDC)
...steps. Our partner is looking for a Software Engineer L5/L6 — Model Evaluations & Data Curation (MEDC) based in the... ...infrastructure that accelerates how AI teams create, evaluate, and improve... ...curation infrastructure, including shared libraries, components, and workflows...
Suggested
Full time
Remote work
Flexible hours
jobgether
United States
6 days ago
Research Engineer - Code Generation & Model Evaluation
...Role Title: Research Engineer - Code Generation & Model Evaluation Role Type: Contractor Location... ...train next-generation AI systems. Your work will... ...transparency and knowledge sharing. Required Skills and... ...participation in collaborative software projects. Strong...
Suggested
Temporary work
For contractors
Remote work
micro1
Remote
24 days ago
Research Engineer, Code Generation & Model Evaluation
$50 - $100 per hour
...Apply advanced software engineering and problem-solving expertise to code generation and model evaluation work that helps improve how next-generation AI systems learn, reason, and perform. This remote... ...support transparency and knowledge sharing. Qualifications Expertise...
Hourly pay
Contract work
For contractors
Remote work
SaidGig
United States
28 days ago
Principal, Data Scientist, Agentic AI Systems Engineering & Model Post-Training
$110k - $220k
...Supply Chain AI Lab & Innovation... ...rigorous evaluation and model post-training.... ...on AI systems engineering position focused... ...operating production software. As a... ...domains within the shared agentic... ...retrieval, durable sessions and workflows,... ...uncontrolled code-execution...
Full time
Contract work
Temporary work
Part time
Walmart
Bentonville, AR
3 days ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...technical talent with leading AI research labs. Headquartered in... ...: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type: Contract... ...agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
13 days ago
Senior Software Engineer - Session Replay
$154.45k - $185.33k
...Grafana Cloud brings AI to observability and... ...time zones. Senior Software Engineer - Session Replay The Opportunity... ...Defining data models, ingestion pipelines,... ...You can use modern AI coding assistants as part of... ...success. We believe in shared outcomes—RSUs help us...
Full time
Local area
Remote work
Grafana Labs
Remote
18 days ago
Senior Software Engineer - LLM Evaluation
...looking for a Senior Software Engineer – LLM Evaluation based in United States... ...evaluation of large language models. You will curate high-quality code, develop technical solutions, and evaluate AI-generated software... ...shortlist is then shared directly with the hiring...
Full time
Part time
For contractors
Remote work
10 hours per week
Flexible hours
jobgether
United States
7 days ago
Sr. Software Development Engineer, AI/ML, AWS Neuron, Model Inference
$193.3k - $261.5k
...builds AWS Neuron, the software development kit... ...running a wide range of models and supporting... ...software boundary, our engineers build systematic... ...what's possible in AI acceleration.As... ...As you design and code solutions to help our... ...celebrates knowledge-sharing and mentorship. Our...
Work experience placement
Internship
Local area
Flexible hours
Amazon
Cupertino, CA
5 days ago
Senior Software Engineer - AI Evaluation / Coding Agents
...working with frontier AI labs to accelerate model development through... ...training data, evaluations, and engineering talent. About the... ...experienced, hands-on software engineers to help... ...and improve AI coding models . Rather... ...scalable processes Share clear, actionable findings...
Hourly pay
Temporary work
For contractors
Freelance
Immediate start
Remote work
Turing
Remote
a month ago
Senior Full Stack Software Engineer
...Senior Full Stack Software Engineer TypeScript ·... ...Serverless · AI-Assisted... ...around agentic AI: coding agents plan, implement... ...agent sessions, and integrate... ...degradation. Evaluate what you ship.... ...Team-Centric. Share patterns, prompts... ...data leakage into models, and over-permissioned...
Contract work
Remote work
Life Line Screening
Cleveland, OH
3 days ago
Sr. Full-Stack Software Engineer
$175k - $215k
...Sr. Full-Stack Engineer Team: Engineering... ...the most AI-native startup... ...senior full-stack software engineer who... ...product design sessions, architecture discussions... ..., and code reviews. Design... ...goal and evaluating effort, risk, and... ...and knowledge sharing - including how...
Full time
Work at office
Remote work
Relocation
Relocation package
Plug Technologies Inc
Los Angeles, CA
5 days ago
Software Engineer for AI Model Training, Open Source Contributions
$50 - $100 per hour
...Contribute to open source software projects that help train next-generation AI systems. You will apply real-world engineering expertise to create, improve... ...production-quality code. Prior AI experience is not... ...multifaceted constraints and evaluate multiple solution paths....
Hourly pay
For contractors
Remote work
SaidGig
United States
27 days ago
Senior Software Engineer for AI Model Evaluation
$65 - $105 per hour
...Apply deep engineering judgment to help frontier AI models reason more accurately about real-world software development and engineered systems. You... ...quality engineering work, evaluate model performance, and turn... ...reasoning, subtly incorrect code, unaddressed edge cases,...
Hourly pay
Full time
Freelance
Internship
Live in
Relocation
Relocation package
SaidGig
Bay County, FL
a month ago
Software Engineer for AI Model Training
$100 per hour
...Apply your software engineering expertise to improve advanced AI systems through rigorous technical review... ..., prompt development, evaluation, and high-quality... ...instructions that guide AI models to produce high-quality... ...postmortems, technical blogs, or code reviews. ~ Excellent...
Hourly pay
Part time
For contractors
Remote work
SaidGig
United States
more than 2 months ago
Senior Murex Software Engineer
$94k - $115k
...functions.The Senior Software Engineer will serve as a... ...hybrid delivery model.Lead design sessions and architecture... ...design, coding standards, troubleshooting... ...Leverage modern AI-assisted... ...be contacted to evaluate the accommodation... ...formed around a shared interest or a common...
Full time
Contract work
Work at office
Remote work
Worldwide
Work visa
Relocation package
3 days per week
Transamerica
Denver, CO
3 days ago
Senior Software Engineer
$136.8k - $205.2k
...hiring a senior engineer to lead... ...today centers on AI systems in... ...make them safe; evaluation of systems... ...often as with code we write ourselves... ...the working sessions, and carry... ...them: working software against real... ...based scoring, model-graded... ...test, learn, share, and repeat.Preferred...
Full time
Temporary work
Work at office
Immediate start
Remote work
Northwestern Mutual
New York, NY
4 days ago
Senior Software Engineer
...experienced Senior Software Engineer to join our dynamic,... ...verify, or override AI-assisted output as part... ...automation (code review, triage, docs... ...record of critically evaluating their output rather... ...modern stack. Knowledge Sharing: Regular sessions where the team trades...
Remote work
Home office
Flexible hours
Marigold
Oregon State
1 day ago
Software Engineer
$125.7k - $194.8k
...It all started when engineer Fred Luddy wrote code that automated a... ...ServiceNow is the AI control tower for business... ...Area: Software Development Develop... ...integration, and basic evaluation Work with assigned... ...Participate in knowledge-sharing sessions and team learning...
Work at office
Immediate start
Remote work
Flexible hours
SmartRecruiters
Eastern, KY
2 days ago
Specialist Software Engineering
$131k - $165k
...SummaryThe Specialist Software Engineer is focused on the... ...agile environment.Code, test, identify,... ...large team design sessions.Serve as a subject... ...pipeline-as-code, shared templates, and... ...)Familiarity with AI Development Tools... ...should be contacted to evaluate the accommodation...
Full time
Contract work
Work at office
Remote work
Worldwide
Work visa
Relocation package
3 days per week
Transamerica
Denver, CO
4 days ago
Software Engineer L5 - AI Observability & Agent Evaluation
$388k
...part of what’s next.AI and ML power... ...workloads, capturing model inputs, features,... ...a hands-on senior engineer to build the frameworks... ...performance, evaluation, and vendor integration... ...:Experience in software, AI/ML, or platform... ...use case.Strong coding skills in Python and...
Hourly pay
Full time
Immediate start
Flexible hours
Netflix
Los Gatos, CA
1 day ago
Senior Software Engineer
$200k - $220k
...Senior Software Engineer (Full-Stack Mastermind... ...figuring out how AI changes the way... ...generating code and shipping whatever... ...constraints, evaluate what comes back... ...debugging session to a junior engineer... ...not to use a model. Our customers... ...reviews, mentorship, shared techniques and...
Full time
Remote work
RTA: The Fleet Success Company
United States
4 days ago
Senior Software Engineer
$91.7k - $163.7k
...Senior Software Engineer UnitedHealth Group... ...with a team that shares your passion... ...efforts (AI 1.0) have already... ...patterns, and secure coding practices... ...solutioning sessions Mentor junior... ...practices for model deployment,... ...model registries, evaluation frameworks,...
Minimum wage
Full time
Work experience placement
Local area
Remote work
UnitedHealthcare
United States
3 days ago
Senior Software Engineer
...Senior Software Engineer AccuLynx is a fast-growing... ...major planning sessions and meetings, but... ...written, well-tested code to complete new... ...product ~ Leverages AI-powered... ...standards ~ Actively evaluates and integrates emerging... ...process, sharing best practices to...
For contractors
Remote work
Flexible hours
Night shift
AccuLynx
United States
2 days ago
Software Engineer
...Software Engineer Roy Jorgensen Associates... ..., and agentic-AI platform that... ...equipped with AI coding tools as a... ...video/LiDAR AI model training, and... ...Participate in the shared on-call... ...team working sessions across the region... ...for performance evaluations. Safety/...
Contract work
Work at office
Local area
Remote work
Work from home
Home office
Night shift
Roy Jorgensen Associates
United States
4 days ago
Senior Software Engineer
$160k - $190k
...Background: Software development... ...instructional model that leverages... ..., hands-on, AI-native Senior Software Engineer to help transform... ...and agentic coding to build... ...outages, with an evaluation harness,... ...so that every session produces better... ...in graphs — shared state flowing...
Full time
For contractors
Immediate start
Remote work
Worldwide
Flexible hours
Shift work
Modern Classrooms Project
United States
2 days ago
Senior Software Engineer - Blockchain Mastercard
$132k - $212k
...And Summary Senior Software Engineer - Blockchain... ...alongside cutting-edge AI tools and agentic... ...architecture reviews, code reviews, and technical design sessions to drive... ...support engineers by sharing knowledge, promoting... ...across the team. Evaluate emerging blockchain...
Full time
Contract work
Part time
Worldwide
Flexible hours
The Bitcoin Street Journal
Eastern, KY
4 days ago
Remote South Asia Software Engineers - Evaluation Study
AI Trainer Jobs is seeking experienced software engineers who can review coding tasks and evaluation harnesses for realism in a fully remote role. You will walk through proposed tasks, assess structure and difficulty, and provide actionable feedback to improve test suites...
Remote job
Weekly pay
For contractors
AI Trainer Jobs
New York, NY
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineers: Share Your AI Coding Sessions for Model Evaluation [Remote]. Be the first to apply!