STEM Professional for AI Coding Agent Evaluation [Remote]
$80 - $100 per hourSaidGig
Remote
- Remote job
Role Overview
Contribute your STEM expertise to shape and improve next-generation AI coding agents. In this contractor role you will describe and evaluate how you use AI coding tools in real technical work, supply data-driven feedback on agent behavior, and help train models to handle complex engineering tasks. This opportunity is remote and focused on applying real-world domain knowledge rather than prior AI research experience.
Key Responsibilities- Share detailed accounts of how you use AI coding agents, including the specific tools, workflows, and technical challenges you address.
- Provide comprehensive descriptions and evaluations of AI agents tackling complex, real-world problems, for example navigating large codebases, advanced debugging, and architectural changes.
- Document step-by-step approaches for using agents to build end-to-end features, resolve ambiguous requirements, or collaborate on multi-person projects.
- Critically assess strengths and limitations of AI coding agents across scenarios, and offer data-driven insights and practical suggestions for improvement.
- Deliver written and verbal feedback that informs model and product development.
- Participate in structured interviews, surveys, and knowledge-sharing sessions to capture your experience as a technical practitioner.
- Extensive hands-on experience as a STEM professional, regularly using AI coding agents as a core tool in your technical toolkit.
- Required skills: experience with AI coding agents, workflow documentation, technical problem-solving, advanced debugging, feature development, architectural refactoring, written communication, verbal communication, stakeholder communication, knowledge sharing, and data-driven analysis.
- Ability to clearly articulate complex technical workflows, challenges, and solutions in writing and verbally.
- Demonstrated track record applying AI assistants to sophisticated tasks, such as end-to-end feature development, architectural refactoring, or advanced debugging in real projects.
- Experience communicating technical concepts to both technical and non-technical stakeholders, and familiarity with popular AI coding tools and emerging best practices.
- Engagement type: Independent contractor role, performed remotely.
- Work will involve contributing to a customer project aimed at advancing AI coding agent capabilities.
- Activities include creating documentation, providing written feedback, and participating in interviews, surveys, and knowledge-sharing sessions as requested by the project team.
Hourly rate, $80.00 to $100.00 per hour.
Eligibility- No prior AI research or model-building experience required, domain expertise in STEM and practical use of AI coding agents are the primary qualifications.
- Open to professionals who currently incorporate AI coding agents into their workflows and can provide concrete examples of that use.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the STEM Professional for AI Coding Agent Evaluation [Remote] in Remote vacancy
$80 - $100 per hour
...Role Overview Help train and evaluate next-generation AI coding agents by documenting real-world technical workflows... ...role you will convert hands-on STEM experience into structured examples... ...include current STEM professional experience using AI coding agents in...SuggestedRemote jobHourly payFor contractorsWork visaSaidGig
Canada1 day ago$80 - $100 per hour
...Role Overview Help train and evaluate AI coding agents by contributing real-world STEM workflows, problem examples, and candid feedback. In this contractor... ...accuracy and usability Qualifications Active professional experience in a STEM field, regularly engaging...SuggestedHourly payFor contractorsRemote workSaidGig
United States1 day ago$80 - $100 per hour
...Role Overview Help train next-generation AI coding agents by contributing expert, real-world... ...with project stakeholders using clear, professional written and verbal communication. Qualifications... ...problem-solving. Preferred: current STEM professional actively using AI coding...SuggestedRemote jobHourly payFor contractorsSaidGig
Remote1 day ago$50 per hour
...Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is... ...: ~We're building a dataset to evaluate AI coding agents - how well a model handles real-world...SuggestedPermanent employmentTemporary workPart timeMindrift
Remote2 days ago$80 per hour
...technology firm is seeking QAs for autonomous AI agents to validate and improve task structures... ...attention to detail and experience in evaluating scenarios. This flexible, project-based... ..., allowing integration with academic or professional commitments. #J-18808-Ljbffr MindriftSuggestedRemote jobFlexible hours- We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. You'll create challenging tasks and evaluation criteria within realistic simulated environments: Build realistic developer environments - a virtual company with...Contract work
Mindrift
Remote4 days ago - ...which will expand your professional network and build... ...developing and researching AI Agent decision-quality... ...degree or above in a STEM field (CS, Statistics,... ...training and classification evaluation metrics (precision, recall... ...documentation and code are in English) Strong...Full timeContract workInternshipLocal areaWork from home
$225k - $285k
...Description We build agentic AI products that our... ...modalities. These agents sit on top of the same... ...architecture and hands-on coding. Requirements ~... ...and operating evaluation infrastructure. ~Experience... .... ~Fast-paced and professional work culture. ~...Full timeWork from homeFlexible hoursPayNearMe, Inc.
Remote7 days ago- ...enablement with agentic AI. Our platform tightens... ...to join the Systems Agents engineering team. This team builds the coding harness, runtime layer,... ...compliant, and have strong evaluations. This role is ideal... ...solutions. ~5+ years of professional software development...Full timeRemote work
Vannevar
Remote6 days ago - ...STEM AI Agent Research Specialist is a remote review track for evaluating AI outputs across stem research reasoning, calculations, and research workflows. Reviewers... ...publishing, teaching, or advising on the topic at a professional level. Comfort applying multi-page rubrics...Remote jobHourly payFor contractors10 hours per week
- ...Description We are looking for an AI Video Agent Engineer for Elva. This role... ...types of users: ~professional video creators ~casual users... ...services ~Rapidly evaluate available APIs and select the... ...Programming ~Strong practical coding skills with vibe coding...Full timeCasual work
Toogeza
Remote1 day ago $60 per hour
...contributors for a part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and... ...with structured data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define expected AI behaviors...Part timeRemote workFlexible hoursMind Rift
Kansas City, MO4 days ago$250k - $280k
...Shape the Future of AI At Labelbox, we're building the critical... ...factory for advancing frontier agent capabilities. We build the data, environments, and evaluations that frontier labs use to train... ...is right. You ship production code with coding agents daily. You know...Full timeWork at officeFlexible hours3 days per weekLabelbox
Remote1 day ago- ...leading enterprises orchestrate AI-powered work. Our vision is to... ...are building and deploying AI agents that are grounded in their... ...decisions and participate in rigorous code reviews to uphold quality and... ...What you need ~3+ years of professional software engineering...Full timeWork at officeLocal areaFlexible hours
Writer
Remote1 day ago - A forward-thinking technology company is looking for a detail-oriented individual to design structured evaluation scenarios for AI agents. This entry-level part-time role allows you to contribute remotely, creating test cases for LLM-based agents. The ideal candidate will...Remote jobPart time
$80 per hour
A leading AI consultancy is seeking a detail-oriented individual to design evaluation scenarios for LLM-based agents. This part-time, remote role allows you to create structured test cases while working flexibly around your commitments. The ideal candidate holds a relevant...Part timeRemote work$80 per hour
...-time opportunity focused on quality assurance for autonomous AI agents. You will analyze complex systems, review tasks for logic, and... ...analytical and detail-oriented skills, with experience in policy evaluation or logic puzzles preferred. Compensation can reach up to $80/...Remote jobPart timeFlexible hours$85 per hour
...technical talent with leading AI research labs. Headquartered... ...Position: Security Engineer (Coding Agent Experience) Type:... ...coding agents to complete and evaluate complex security engineering... ...strengths and weaknesses. Apply professional engineering judgment to realistic...Contract workSummer workRemote work- ...them with exceptional professionals for this role. JOB DESCRIPTION... ...- ML Engineer for Agent Platform at JPMorgan... ...communicate, remember, and get evaluated in a secure, stable,... ...-quality production code, and reviews and debugs... ...enterprise-authorized AI-assisted engineering practices...
J.P. Morgan
Austin, AR2 days ago $60 per hour
...A leading AI firm in Austin is looking for QA experts to validate and improve AI systems. This... ...to detail. Candidates will review AI evaluation tasks, identify inconsistencies, and define expected behaviors for agents. Ideal applicants have experience in policy evaluation...FreelanceRemote workMind Rift
Austin, TX2 days ago- ...As a Senior Software Engineer, Agents on our Delivery team, you make... ...specific customers: ~Own AI agents for your assigned customers... ...: discovery, design, build, evaluation, launch, and live operation.... ...Requirements ~General-purpose coding experience (TypeScript, Python...Full timeRemote workFlexible hours
Replicant
Remote3 days ago $166.5k - $291.4k
...when engineer Fred Luddy wrote code that automated a tedious task... ...work. Today, ServiceNow is the AI control tower for business... ....Job DescriptionThe teamBuild Agent is ServiceNow's AI coding assistant... ...for the Build Agent evaluation framework, model support, and...Work at officeImmediate startRemote workFlexible hours$200k - $220k
...technical direction for our endpoint agents that run on every protected... ...you level up as from the code you write yourself. ~Design,... ...good Go looks like here. ~Use AI tools to move faster and... ...Annual allowance for education and professional development assistance. ~$7...Full timeRemote workHome officeHuntress
Remote1 day ago$224k - $302.4k
...hire a Staff Software Engineer - Agent Optimization. Temporal... ...reliable foundation powering AI leaders such as OpenAI, NVIDIA... ...performance of Codex, Claude Code, and similar tools for developers... ...What You’ll Bring ~8+ years of professional work experience. ~Proven experience...Full timeWork experience placementWork at officeRemote workWork from homeHome officeTemporal Technologies
Remote4 days ago$85 per hour
...technical talent with leading AI research labs. Headquartered... ...Position: Backend Engineer (Coding Agent Experience) Type: Contract... ...agents to complete and evaluate complex engineering tasks.... ...strengths and weaknesses. Apply professional engineering judgment to realistic...Contract workSummer workRemote work$180k - $225k
About Scale AIScale AI is the data foundation for AI, helping... ...telecommunications to build production AI agents that automate complex... ...customers to design, evaluate, and deploy intelligent systems... ...and use personal data for our professional business purposes, including...Full time$147.6k - $274k
...love. That’s what makes us Roche.Advances in AI, data, and computational sciences are... ...Senior Machine Learning Scientist building agents for applied small-molecule drug design. You... ...You are fluent in Python and modern agentic coding environments such as LangChain, ML...Full timeLocal areaWorldwideRelocation package$142.8k - $193.2k
...engineers, analysts, product professionals and more, all with the shared... ...advanced software tools, new AI systems, and machine learning... ...toughest hiring problems.Recruiting Agents and Candidate Voice team is... ...journey.• Develop rigorous evaluation frameworks to measure agent...WorldwideFlexible hours- ...security-first enterprise AI company. We build... ...to securely deploy AI agents and automations within... ...agent delegation, sandbox code execution, and failure... ...cornerOwn North's agentic evaluation framework, ensuring evals... ...tools in both professional and personal workHands-...Full timeWork at officeLocal areaRemote workHome officeFlexible hours
$10k
...About the Role Ramp's Agent Developer Platform team builds... ...interfaces -- that let developers and AI agents read from and write to... ...and guardrails, and evaluation and monitoring for agentic workflows... ...Demonstrated hands-on, daily use of AI coding agents as the primary...Full timeWork at officeHome officeRelocation packageFlexible hoursThe Ramp
Remote1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to STEM Professional for AI Coding Agent Evaluation [Remote]. Be the first to apply!
