AI Agent Trajectory Annotator and Reviewer
$20 - $30 per hourBespoke Labs
Type: Contract, hourly
Location: Remote
Hours: 20–30 per week
Pay: $20–30/hour, based on experience and language coverage
Start: Immediate
ABOUT THE ROLE
We evaluate how well advanced AI coding agents solve real engineering problems. An agent is given a real open source codebase inside a container and a hard task, then works on its own for 80 to 250 steps. A trajectory is the full record of that run — every command, result, and decision.
You will do two jobs, and you should expect either on any given day.
•Annotate — Read a trajectory nobody has looked at yet and judge it step by step.
• Review — Take an existing annotation, written by our AI tooling or another person, and confirm, correct, or reject it.
TASKS YOU'LL SEE
• Feature build — Add a working feature to a live library without breaking anything that already worked.
• Rebuild — Work out what a compiled tool does by running it, then rebuild it to match its output, exit codes, and file effects.
• Bug hunt — Find and fix twenty undocumented bugs across a dozen files with no test suite, then record what caused them.
Mostly Python and Go, with some Rust, C, and Ct+. A trajectory runs about 100 steps.
WHAT YOU JUDGE IN A TRAJECTORY
• Was the command right for the state the environment was actually in?
• Did the agent read the previous output correctly?
• Was the step wrong, or only inefficient — these are scored differently.
• Where did the run first go off course — usually earlier than where it visibly broke.
• Did the agent notice its own mistake and recover, or keep building on a false assumption?
• Did it game the grader instead of solving the task (e.g., weakening a test or hardcoding an expected value)?
WHAT WE NEED FROM YOU
• Experience — 2+ years in software engineering, DevOps, or site reliability, with real debugging in real codebases.
• Languages — Strong in Python or Go, and able to read a language you've never used.
• Linux — Comfortable with logs, running processes, build failures, and containers.
• Workflow — Everyday Git, diffs, pull requests, and issue tracking.
• Debugging — Able to work with no test suite and no error message pointing at the cause.
• Focus — Able to hold context across a long run, because step 74 can depend on step 12.
• Writing — Clear English, since every judgement needs an explanation another engineer can check.
$89.2k - $209.5k
Job Description Oracle Health is seeking a Senior AI Agent Engineer to build production AI agents and workflow automation capabilities that accelerate analytics delivery, improve insight generation, and reduce manual engineering work across our analytics and reporting...SuggestedTemporary workFlexible hours$43k - $48k
...Advertising Agent Brixton Corp is a U.S.-based marketing and talent solutions company dedicated to helping businesses grow through strategic execution and structured processes. We work with organizations across multiple industries to strengthen their market presence...SuggestedBrixton
Baton Rouge, LA4 days ago$40k - $45k
Company Description Beloform Craft is a growing company focused on creating meaningful connections, maintaining a strong professional presence, and delivering exceptional experiences for the people and organizations we work with. We believe great communication ...SuggestedFull timeWork at officeAlphabe Insight Inc
Baton Rouge, LA4 days ago- ...and every team member has the opportunity to make a lasting impact.Job DescriptionBeloform Craft is seeking a motivated Advertising Agent to support the planning, coordination, and execution of advertising initiatives. In this role, you will collaborate with internal teams...SuggestedFull time
Alphabe Insight
Baton Rouge, LA1 day ago - We're Changing Entertainment. Come Join Us.We're always looking for talent that believes in having fun. At PENN Entertainment, you'll get to be a part of an exciting industry, where the days and nights are fast paced. You'll work with an incredibly diverse set of coworkers...SuggestedLocal areaNight shift
PENN Entertainment
Baton Rouge, LA2 days ago - ...Special Agent Position Joining the United States Secret Service (USSS) as a Special Agent offers a unique and rewarding career dedicated to serving the nation through our dual mission of criminal investigations and protection. Special Agents play a critical role in...
United States Secret Service
Baton Rouge, LA2 days ago $15 per hour
...diploma or GED equivalent is preferred.Equal Opportunity Employer This employer is required to notify all applicants of their rights pursuant to federal employment laws. For further information, please review the Know Your Rights notice from the Department of Labor....Flexible hoursShift workNight shiftBally’s Corporation
Baton Rouge, LA2 days ago$100k
...and property damage claims.We are currently hiring Surveillance Agents to conduct stationary/mobile surveillance across the country. Over... ...the preparation for and during surveillance operationsPrepare, review, and submit detailed reports of findings, incidents, and...Hourly payFull timeCurrently hiringFlexible hoursMarshall Investigative Group
Baton Rouge, LA2 days ago$97.3k - $125.54k
...JOB DESCRIPTION The position advertised has been exempted from the federal civilian hiring freeze. As an FBI special agent, you'll directly impact national security. By harnessing your background to transition into federal law enforcement, you can help shape the Bureau...Work at officeLocal areaFederal Bureau of Investigation (FBI)
Baton Rouge, LA3 days ago$44k - $49k
...Advertising Agent The Advertising Agent will help represent client brands, communicate campaign messages effectively, and support advertising initiatives designed to increase brand awareness and customer engagement. This position is ideal for someone who enjoys working...Full timeIGL4US
Baton Rouge, LA1 day ago$70k - $100k
...management systems (EDMS). This is an on-site position that will report to our Baton Rouge, LA Office. Responsibilities Manage receipt, review, processing, distribution, and archiving engineering and construction project documentation. Maintain project document registers,...Contract workTemporary workFor subcontractorWork at officeRemote work- Dynamic Group is a national general contractor with divisions spanning heavy civil construction, building construction, and emergency response. Our civil construction capabilities include grading, drainage, subsurface utilities, pile driving, marine construction, and deep...Contract workFor contractorsFor subcontractor
- ...Procore Submittals as outlined in contract documents Maintain distribution list of responsible parties for various stages of document review process Assign workflow to responsible parties to review vendor documents within Procore or other document control systems After...Contract workTemporary workWork at officeLocal area
- Location 11585 Lake Sherwood Avenue North,Baton Rouge, LA, 70816,United States WHAT WE NEED: Document Controller The Document Controller is responsible for managing the document control process within the organization. This role involves overseeing the creation, revision...Work experience placementWork at officeShift work
- Introduction ... Overview We're always looking for talent that believes in having fun. At PENN Entertainment, you'll get to be a part of an exciting industry, where the days and nights are fast paced. You'll work with an incredibly diverse set of coworkers...Local areaNight shift
Penn Entertaiment
Baton Rouge, LA1 day ago $55k - $62k
...Job Description Job Description We are seeking a highly motivated and detail-oriented Advertising Agent to join our dynamic team. This role involves planning and executing advertising campaigns, analyzing market trends, and coordinating with internal teams and...Work at officeEntertainment Travel Associates
Baton Rouge, LA4 days ago- United Weld Holdings has an exciting opportunity available for a Document Controller to be an integral part of our Engineering team at our corporate location in Baton Rouge, LA. UWH takes pride in our quality of work and has a goal of building our workforce with the most...Work at office
EPIC PIPING, LLC
Baton Rouge, LA5 days ago $36.06 - $40.87 per hour
.... Understands and applies as necessary all relevant styles and standards for developing documentation materials. Assists in peer reviews of all documentation materials created by other team members. Analyzes requirements of project to determine scope of work and required...Contract workWork at office- ...you have aspirations to run your own business? If so, you may want to consider working in the office of Steven Brooksher - State Farm Agent. As a member of our agency team, you have an opportunity to experience first-hand what it takes to be a State Farm Agent. You will...Hourly payFor contractorsInternshipWork at officeLocal areaFlexible hours
State Farm
Baton Rouge, LA5 days ago $14 - $20 per hour
...Looking for a part-time job or internship that provides meaningful work and competitive compensation? Consider a position in a State Farm Agent's office. As an Agent Team Member, you will receive: Hourly pay Flexible hours Valuable experience Growth...Hourly payPart timeFor contractorsInternshipWork at officeMonday to FridayFlexible hoursDrew Dileo
Baton Rouge, LA4 days ago- ...you have aspirations to run your own business? If so, you may want to consider working in the office of Steven Brooksher - State Farm Agent. As a member of our agency team, you have an opportunity to experience first-hand what it takes to be a State Farm Agent. You will...Hourly payFor contractorsInternshipWork at officeLocal areaFlexible hours
Steven Brooksher - State Farm Agent
Baton Rouge, LA3 days ago $10k
Border Patrol Agent (BPA) - in the Federal Security and Public Safety Sector Experienced (GL-9 GS-11) SAME MISSION, NEW DRIVE! You love... ...employment site, to complete your application. Be certain to review ALL details of the job opportunity announcement and follow all instructions...Full timeLocal areaImmediate startRelocationNight shiftUS Customs and Border Protection
Baton Rouge, LA2 days ago- United Weld Holdings is seeking a Document Controller to support the Engineering team in Baton Rouge, LA. You will compile and manage engineering data in the document management system, ensuring accurate storage, organization, and retrieval for a large department. The role...
- ...agency opened in 2000 and is a strong, experienced team of six. I bring a well-rounded background, having worked both for State Farm agents and in corporate, and I’m proud to be a third-generation State Farm agent. Outside of the office, I’m a big sports fan,...Work at officeFlexible hours
Kelly Elkins - State Farm Agent
Baton Rouge, LA3 days ago - Job Description A Day in the Life: As a File Maintenance Clerk, you'll play a key role in ensuring pricing accuracy and signage compliance throughout the store. You'll be responsible for implementing corporate price changes, maintaining pricing records, and ensuring all...Weekly payLocal areaFlexible hoursShift workDay shiftAfternoon shift
$55k - $75k
...opened in April 2016 and is a close-knit team of four focused on growth, service, and strong relationships. Before becoming a State Farm agent, I participated in the Agent Aspirant program, which helped build a strong foundation for leading and developing a successful team....For contractorsWork at officeLocal areaFlexible hoursState Farm
Baton Rouge, LA1 day ago$18 per hour
...and competitive compensation? Consider a position in a State Farm Agent's office.Responsibilities:Establish customer relationships and... ...and friendly customer serviceUse a customer-focused, needs-based review process to educate customers about insurance options.As an Agent...Full timeFor contractorsWork at officeAlex Mcgehee - State Farm Agent
Baton Rouge, LA4 days ago- ...you're newly licensed and ready to commit to real estate full time, this is your opportunity to skip the trial-and-error most new agents face. Join the top 20% of real estate professionals in North America and build your first year on a proven foundation instead of guesswork...Full timeSecond job
- Benefits Flexible schedule Opportunity for advancement Training & development Ross Garbarino - State Farm Agent is seeking an organized and efficient specialist to fulfill administrative duties and support insurance agents in gaining and keeping customers. As an Intern...Full timeInternshipWork at officeLocal areaFlexible hours
$14 - $20 per hour
...Looking for a part-time job or internship that provides meaningful work and competitive compensation? Consider a position in a State Farm Agent's office. As an Agent Team Member, you will receive: Hourly pay Flexible hours Valuable experience Growth potential/...Hourly payPart timeFor contractorsInternshipWork at officeMonday to FridayFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Agent Trajectory Annotator and Reviewer. Be the first to apply!
- tsa agent Baton Rouge, LA
- operations agent Baton Rouge, LA
- executive protection agent Baton Rouge, LA
- airport agent Baton Rouge, LA
- agent assistant Baton Rouge, LA
- telemarketer - state farm agent team member Baton Rouge, LA
- import export agent Baton Rouge, LA
- agent Baton Rouge, LA
- state farm agent Baton Rouge, LA
- cruise agent Baton Rouge, LA