Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Applied Engineering Benchmark Specialist

$61 - $77 per hour

AuraOne Human Data

Applied Engineering Benchmark Specialist is a remote review track for evaluating AI outputs across applied engineering benchmark specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.

Why this role matters

Applied Engineering Benchmark specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.

Responsibilities

  • Review AI outputs against current applied engineering benchmark specialist operations workflows, playbooks, and firm policy for Applied Engineering Benchmark Specialist assignments.
  • Grade tone, escalation logic, and stakeholder fit on a structured rubric.
  • Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
  • Capture the right next step so the modeling team can train on it.
  • Adjudicate disputed treatments against published playbooks or firm guidance.
  • Maintain reviewer-quality scores in inter-rater calibration cycles.

Qualifications

  • Direct working experience in applied engineering benchmark specialist operations on real teams for Applied Engineering Benchmark Specialist work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the policy or workflow being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Example tasks

  • Grade a model's response to a real applied engineering benchmark specialist operations ticket and rate workflow, tone, and escalation.
  • Flag a missed escalation with the right severity tag and corrected next step.
  • Adjudicate a disputed playbook call between two reviewers using firm guidance.
  • Audit a 25-row batch for rubric consistency and report drift to the program lead.

Nice to have

  • Prior experience training, calibrating, or QA-ing operations teams.
  • Familiarity with AI-assisted workflow tooling and its failure modes.
  • Bilingual experience for cross-region operations.

Skills

  • Operational review
  • Policy adherence
  • Workflow judgment
  • Stakeholder communication
  • Applied Engineering Benchmark specialist operations

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

$61–$77 / hr

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Vacancy posted 27 days ago
Similar jobs that could be interesting for youBased on the Applied Engineering Benchmark Specialist in Remote vacancy
  •  ...detailsDomainResearch, design and developmentJob field / Job profileMechanics - Mechanical part or subassembly designJob titleDesign Engineering Specialist IIIEmployment typePermanentProfessional categoryEmployees / StaffPart time / Full timeFull-timeJob descriptionUses CAD... 
    Suggested
    Permanent employment
    Local area
    Remote work
    Flexible hours

    Safran Group

    Washington DC
    4 days ago
  • $40 - $50 per hour

     ...research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $40–$50/... 
    Suggested
    Full time
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    Remote
    1 day ago
  • YO AI Labs seeks experienced Mechanical Engineering Technicians to support an advanced AI benchmark project entirely remotely. You will craft realistic testing scenarios, interpret data, and diagnose failures to build credible datasets for validation. Role emphasizes hands... 
    Suggested
    Remote job

    YO AI Labs

    Chicago, IL
    4 days ago
  • $50 - $150 per hour

     ...research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Software Engineering Expert Type: Contract Compensation: $50–$150... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Remote
    1 day ago
  •  ...unify diverse financial data sources (Excel, ERPs, CRMs) into coherent, queryable schemas that agents can reason over. Design benchmarking and evaluation suites that quantify Sapien's accuracy, reliability, and business impact across different customer workflows.... 
    Suggested
    Full time

    Sapien

    Remote
    1 day ago
  • $280k - $320k

     ...growing group of committed researchers, engineers, policy experts, and business leaders working...  ...the Role We're looking for an Applied AI Engineer to join our Beneficial Deployments...  ...ecosystem-level tooling (MCP servers, benchmarks, reusable agent skills) that extends... 
    Full time
    Work at office
    Visa sponsorship
    Flexible hours
    Shift work

    Anthropic

    Remote
    1 day ago
  •  ...researchers to create evaluations, publish benchmarks, and push the boundary of data. We’ve...  ...educational institutions Work together with engineers, scientists, operators, and more from...  .... About the Role As a Senior Applied AI Engineer at Handshake AI Enterprise,... 
    Full time
    Work at office
    Remote work
    Flexible hours

    Handshake

    Remote
    1 day ago
  •  ...building the next generation of evaluation benchmarks for frontier AI models. We are seeking experienced QA and Test Engineers to ensure every benchmark is reliable, reproducible...  ...against rigorous, real-world scenarios. Apply your software testing and quality... 
    Full time
    Contract work
    For contractors
    Remote work
    Flexible hours

    Weekday

    Remote
    a month ago
  •  ...these problems look solved in academic benchmarks, but remain deeply unsolved in real-world...  .... What We’re Looking For Strong engineering fundamentals and ability to build reliable...  ...on hard AI problems are encouraged to apply! Why Join Work on the full stack of... 
    Full time
    Work at office

    Salient

    Remote
    1 day ago
  • $188k - $275k

     ...inference stack. About the role: We are looking for an Applied AI Engineer to help us understand, measure, and improve the real-world...  ..., this role will focus on building and running rigorous benchmarks, profiling model and system behavior, identifying bottlenecks... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    Core Weave

    Remote
    1 day ago
  •  ...time with patients. The role As an Applied AI Engineer on the Research team, you will work...  ...lands in front of clinicians, not by a benchmark in isolation. Our agents are...  ...Sully's stack — decomposition, parallel specialists, dynamic output contracts, and per-agent... 
    Full time
    Work at office
    Remote work
    Night shift

    Sully.ai

    Remote
    1 day ago
  • $182k - $242k

     ...company (Nasdaq: CRWV) in March 2025. Learn more at About this role We're looking for a Senior Engineer for CoreWeave's Benchmarking & Performance team. You will have an integral part in our planet-scale performance data warehouse: Ingesting,... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Remote work
    Flexible hours

    Coreweave

    Remote
    1 day ago
  • KPFF Consulting Engineers is seeking a Structural Engineer specializing in façade design in Portland, Oregon. Your role will involve individual and collaborative contributions throughout the design and construction phases of various curtainwall projects. Candidates with... 
    Remote job

    KPFF Consulting Engineers

    Portland, OR
    1 day ago
  • $115k - $155k

     ...performance, Made in America production.About the team: The Solutions Engineering Team is responsible for delivering robotics-driven automation...  ...PhilosophyFormic’s pay and equity packages are thoughtfully benchmarked against peer companies at a similar growth stage. Equity... 
    Full time
    Temporary work
    Work at office
    Local area
    Immediate start
    Remote work
    Flexible hours
    Shift work

    Formic Technologies

    Chicago, IL
    1 day ago
  •  ...is hiring a Full-Time Day-shift Senior Process Improvement Engineer for the Ambulatory Operations Improvement team Responsible...  ..., Reengineering, Process Redesign, CQI, Business Development, Benchmarking, Productivity Improvements , Technology Breakthroughs, Prosci... 
    Full time
    Temporary work
    Remote work
    Shift work
    Day shift

    Sentara Health

    Virginia Beach, VA
    7 days ago
  •  ...Sr. Data Science Engineer Spec-Data Scientist Concurrent Technologies Corporation Johnstown, PA or Telecommute Concurrent Technologies Corporation (CTC) an independent, nonprofit applied scientific research and development organization. Is seeking a Sr. Data... 
    Remote work

    Concurrent Technologies Corporation

    United States
    1 day ago
  • $160k - $200k

     ...ubiquitous. We build the foundation for agent engineering in the real world, helping developers...  ...M raised at Series B from IVP, Sequoia, Benchmark, CapitalG, and Sapphire Ventures, we’re...  .... About The Team: The Applied AI team builds the agents that show the... 
    Full time
    Work at office
    Relocation
    Flexible hours

    LangChain

    Remote
    1 day ago
  •  ...building fast. Overview of Role We're looking for a Staff Applied AI Engineer to architect the ML systems at the core of Arcade's product...  ...and quality bar — design measurement systems (not one-off benchmarks) that real engineering decisions are made against, so model... 
    Full time

    Arcade

    San Francisco, CA
    1 day ago
  • $236.3k - $277.7k

     ...Are you ready to make an impact? Applied AI Engineer/Architect  West Monroe is seeking an Applied AI Engineer/Architect to lead the...  ...workflows. Establish AI evaluation methodologies using benchmark datasets, regression testing, red-team scenarios, quality metrics... 
    Full time
    Local area
    Immediate start
    Flexible hours

    West Monroe

    Remote
    1 day ago
  •  ...help make a better world of workWe're looking for a Senior Applied AI Engineer to join the Frontier team, helping build Agentic AI Solutions...  ...from paper to productionExperience with evaluation/benchmarking frameworks for AI systemsExperience implementing GraphRAG or... 
    Work at office
    Local area
    Shift work
    2 days per week

    Culture Amp

    Melbourne, FL
    5 days ago
  • $129k - $193.5k

     ...at MPD! We are seeking an experienced Radar Signal Processing Engineer with data fusion and target tracking expertise in the Radar and...  ...projects. What You Need to be Successful - Engineering Specialist Minimum Requirements: Eight or more years related experience,... 
    Full time
    Work at office
    Immediate start
    Remote work
    Relocation package
    Flexible hours

    The Aerospace Corporation

    El Segundo, CA
    4 days ago
  • $162k - $191k

     ...at the core of every business process. Join our mission to make processes work for people, companies, and the planet.Senior Applied Value Engineer - Automotive ManufacturingRole DescriptionAs a Senior Applied Value Engineer, you will push the envelope in solving... 
    Full time
    Work experience placement
    Work at office
    Local area
    Immediate start
    Remote work
    Worldwide
    Flexible hours
    Shift work

    Celonis

    New York, NY
    1 day ago
  • $80 - $100 per hour

     ...Before Applying This role is open to contractors in accepted locations only. Please confirm...  ...Be Doing Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate... 
    Remote job
    Full time
    Contract work
    For contractors

    G2i

    Miami, FL
    1 day ago
  • $152.13k - $192.71k

     ...enjoy living in it.Caterpillar Inc. seeks Embedded Software Engineering Specialist at its facility located at Alpharetta Georgia EPD Plant, 495...  ...leaveAdoption benefitsTuition Reimbursement * These benefits also apply to part-time employeesApplicants who are interested in this... 
    Full time
    Part time
    Immediate start
    Remote work
    Flexible hours

    Caterpillar

    Alpharetta, GA
    2 days ago
  • $70 - $85 per hour

     ...original, graduate-level computational engineering problems that challenge advanced AI systems...  ...partial data. Your work will create benchmark tasks that reveal whether AI can plan...  ...nonlinear solution techniques. Experience applying these tools to workflows like beam,... 
    Hourly pay
    Remote work

    SaidGig

    United States
    29 days ago
  •  ...business outcomes across the organization. As a Senior Applied AI/ML Engineer, you will design, build, and improve production AI systems...  ...experimentation frameworks. Create testing methodologies, benchmarks, and feedback loops that improve model quality, system performance... 
    Permanent employment
    Full time
    Remote work
    Visa sponsorship
    Flexible hours

    Pivotal Health

    Los Angeles, CA
    1 day ago
  •  ...sustainable for everyone. Embedded Software Engineer Role Summary We are looking for a...  ...under extreme loads. Performance Benchmarking: Analyze and optimize ECU system...  ...Benefits Site . External candidates can apply for this role through the Rivian and Volkswagen... 
    Full time
    Contract work
    Local area

    Rivian and Volkswagen Group Technologies

    Remote
    1 day ago
  •  ...decades now.    Job Description Title: Embedded Software Engineer, Cadence/Tensilica Location: Plantation, FL Duration: 6-...  ...workloads. The candidate will be responsible for helping benchmark software across various configurations, and potentially hand-optimize... 
    Full time
    Immediate start

    Aditi Consulting

    Remote
    1 day ago
  • What We Are Looking ForKinder Morgan is seeking an Automation Engineer/Specialist to plan, design, implement and support monitoring and control...  ...hydraulics for liquid pipelines.Knowledge of and ability to apply codes and standards such as UL508A, NEC (NFPA 70), API 500, and... 
    For contractors
    Work experience placement
    Local area
    Work from home
    Weekend work

    Kinder Morgan

    Doraville, GA
    2 days ago
  • $175k - $200k

     ...Senior Applied AI Engineer At Curai, we believe that access to high-quality healthcare is a fundamental human right, not a privilege....  ...guardrails. Develop robust evaluation frameworks — offline benchmarks, human-in-the-loop review, online experiments — that give... 
    Remote work
    Work from home
    Flexible hours

    Curai

    United States
    19 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Applied Engineering Benchmark Specialist. Be the first to apply!