Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Operations Manager - AI Systems Evaluator

$70 - $110 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: General and Operations Managers
Type: Contract
Compensation: $70–$110/hour
Location: Remote
Duration: 4–6 weeks
Commitment: 20+ hours/week

Role Responsibilities

  • Evaluate AI-generated operational plans , staff schedules, and performance reviews for quality and accuracy.
  • Compare AI-generated responses based on operational soundness, financial rigor, and cross-department feasibility.
  • Review operational scenarios involving budgeting, staffing, workflow coordination, and performance improvement decisions. Provide expert feedback.
  • Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

  • 4+ years of experience as a General Manager, Operations Manager, Operations Director, Business Manager, District Manager, or another closely related general operations management role.
  • Experience directing multi-department or multi-location operations, including budgeting, staffing, and performance oversight.
  • Ability to commit 20+ hours per week .
  • Strong written English communication skills.

Start Date

  • Immediate

Compensation & Legal

  • Task-based payment upon first task approval. Experts who complete their first task within the required timeline will then be qualified to be paid hourly for the remainder of the project.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 26 days ago
Similar jobs that could be interesting for youBased on the Operations Manager - AI Systems Evaluator in San Francisco, CA vacancy
  • $80k - $125k

     ...industrializing services through AI, starting with one of...  ...built to solve the systemic data problem that's...  ...% of the Fortune 100, manages over $30B in supply...  ...the Role The AI Operations team owns the agents and...  ...lives, building and evaluating the agents that remove... 
    Suggested
    Hourly pay
    Contract work
    Work experience placement
    Work at office

    Loop, Inc.

    San Francisco, CA
    3 days ago
  • $40.4 - $44.89 per hour

     ...reframe old problems, seek new solutions and operate comfortably in areas that are unknown....  .... Role Summary: Our Autonomy Systems Validation's team is expanding, and we are...  ...and perform data analytics. Develop and manage detailed operational reports, dashboards... 
    Suggested
    Full time
    Temporary work
    Part time
    Flexible hours
    Shift work
    Afternoon shift

    Rivian

    San Francisco, CA
    2 days ago
  • $150 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  •  ...Provider Growth & AI Agent Operations Lead San Francisco Location: San Francisco (Rincon Hill) Company...  ...of marketplace, SaaS, and public sector systems, increasingly powered by AI. Wonderschool helps providers manage operations, governments run large-scale childcare... 
    Suggested
    Work at office
    Flexible hours

    Wonderschool

    San Francisco, CA
    3 days ago
  • Nen Creative is seeking an Operations Lead to own day-to-day business operations, spanning finance, contracts, people operations, and internal systems. You’ll be a natural problem solver who thrives in a fast-paced, high-growth environment, taking initiative and building... 
    Suggested
    Remote job

    Nen Creative

    San Francisco, CA
    4 days ago
  • $85 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks...  ...cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    3 days ago
  • $170k - $220k

     ...reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for...  ...About the role Anthropic's legal team operates at the frontier of some of the most...  ...functional partners such as Security and IT to manage retention, preservation, collection, and... 
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    3 days ago
  •  ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use Gemini, Instinct, or Muse and can assess prompts and responses in English. A part-time commitment of 20 hours per week is available... 
    Full time
    Part time
    Immediate start

    Braintrust

    San Francisco, CA
    13 hours ago
  • $140k - $190k

     ...Job Description Position: Product Manager - AI-Native Business Systems Location: South San Francisco...  ...beyond R&D into a phase of massive operational scale. As our breakthrough "Boosted...  ...Evals & Quality Standards: Stand up evaluation frameworks for operations and... 
    Full time
    Local area
    Visa sponsorship

    Ohalo

    San Francisco, CA
    11 days ago
  •  ...About the Team The User Operations team (Support) is...  ...orchestrated engine powered by AI-native workflows,...  ...increasingly interconnected systems. We are looking for a...  ...Technical Systems Program Manager to help drive the...  ...define scalable solutions, evaluate technical tradeoffs,... 
    Full time
    Work at office
    Relocation package

    OpenAI

    San Francisco, CA
    3 days ago
  •  ...together all of the workforce systems that are normally...  ...time ever, you can manage and automate every part...  ...You'll sit on our Growth Operations team, working across Marketing...  ...and Sales, using AI to make both orgs best-...  ..., prompted, automated, evaluated, or deployed AI tools and... 
    Live in
    Work at office
    3 days per week

    Rippling

    San Francisco, CA
    4 days ago
  • $8 - $65 per hour

    Are you a general operations expert eager to shape the future of AI? Large‑scale language models are evolving...  ...models on topics such as inventory systems, vendor management, SOP development, cost‑control...  ...to our prompt engineering and evaluation metrics. A bachelor's or... 
    Hourly pay
    Contract work
    For contractors
    Remote work

    Remote

    San Francisco, CA
    1 day ago
  •  ...world At Bedrock were moving AI out of the lab and onto real...  ...to work build the field and operations teams behind every deployment...  ...the organization and the systems that keep data flowing ensuring...  ...and experience either directly managing non-exempt employees or... 
    Full time
    For contractors

    Bedrock Robotics

    San Francisco, CA
    a month ago
  •  ...designed for fast, efficient AI inference. We combine large...  ...a Data Center Facilities Operations Lead to own the critical facilities...  ...make sure the facility-side systems that support Gimlet's compute...  ...facility vendors, OEMs, Site Managers, Data Center Technicians,... 
    Full time
    Work at office

    gimlet

    San Francisco, CA
    3 days ago
  • $80 - $120 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ...Larry Summers , and Jack Dorsey . Position: Incident management / reliability / SRE Evaluator Type: Contract Compensation: $80–$120/hour... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    28 days ago
  • $10 - $30 per hour

     ...feedback digital delivery platforms ai-driven tools familiarity independent...  ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments...  ...performance. Our pipeline performance system ensures both quality and velocity.... 
    Part time
    For contractors
    Remote work
    Shift work

    micro1

    San Francisco, CA
    21 days ago
  •  ...Dispatch Role Nevoya is North America's leading next-generation electric trucking carrier, transforming logistics through AI-native operations and relentless customer focus. The company builds and uses proprietary technology to deliver zero-emissions carrier services... 
    Work experience placement
    Internship
    Shift work

    Nevoya

    San Francisco, CA
    3 days ago
  • $99k - $232k

     ...a Cloud Infrastructure and Operations Delivery Manager you will oversee client-facing...  ..., digital tooling, and AI-enabled workflows- Building...  ...Information Technology, Information Systems, Engineering- At least one...  ...with team members. We evaluate these factors thoughtfully... 
    Full time
    H1b

    PwC

    San Francisco, CA
    1 day ago
  • $290k - $365k

     ...create reliable interpretable and steerable AI systems. We want AI to be safe and beneficial...  ...Anthropic gives independent external evaluators access to our models our people and our...  ...policies. Were hiring a Technical Program Manager to establish and deepen our partnerships... 
    Full time
    Contract work
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    2 days ago
  •  ...digital core, optimize their operations, accelerate revenue growth and...  ...expertise, advanced analytics, AI-powered tools, and a global delivery...  ...Solution Architect Senior Manager, Source to Pay to play a...  ...on integration requirements, system connectivity, and the role each... 
    Contract work
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    4 days ago
  • $181.7k - $304.7k

     ...Salesforce Salesforce is the #1 AI CRM, where humans with...  ...lead our EMEA Treasury Operations team while also owning our global Treasury Systems function. This is a...  ...regional operational management with a strategic, cross...  ...recruiters assess and evaluate candidates’ resumes and... 
    Full time
    Work at office
    Local area
    3 days per week

    Salesforce

    San Francisco, CA
    11 days ago
  • $25 - $45 per hour

     ...Skills ChatGPT Muse AI agents Attention to...  ...agents. We provide frontier evaluations and reinforcement learning...  ...Our pipeline performance system ensures both quality and velocity...  ...in engineering, business management, or office/operations Detail-oriented, great... 
    Part time
    For contractors
    Work at office
    Weekend work

    micro1

    San Francisco, CA
    12 days ago
  •  ...Head Of Infrastructure Operations (US)Houston; New York; San Francisco...  ...the GPU cloud engineered for AI. We provide cost-effective,...  ...business outcomes, including cost management, rapid innovation, and...  ...power distribution, cooling systems, security, and environmental... 
    Contract work
    For contractors

    Nscale

    San Francisco, CA
    5 days ago
  • $85k - $120k

     ...What is Verse? The race to AI has become the race to power. Every breakthrough...  ...most ambitious organizations access and manage energy. The Role Verse operates a growing fleet of IoT-connected battery energy storage systems under a demanding availability SLA — with... 
    Hourly pay
    Remote work
    Flexible hours
    Night shift
    Afternoon shift

    Verse

    San Francisco, CA
    3 days ago
  • $40.4 - $44.89 per hour

     ...reframe old problems, seek new solutions and operate comfortably in areas that are unknown....  ...generations. Role Summary Our Autonomy Systems Validation team is expanding, and we are...  ...and perform data analytics. Develop and manage detailed operational reports, dashboards... 
    Full time
    Temporary work
    Part time
    Local area
    Flexible hours
    Shift work
    Afternoon shift

    Rivian Automotive

    South San Francisco, CA
    3 days ago
  • $134k - $179k

     ...CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology,...  ...more at  . What You’ll Do: The DC Program and Cost Management team is the operational planning team responsible for the program and cost management... 
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    San Francisco, CA
    7 days ago
  • $77k - $202k

     ...Strategy / Technology / Operations Consultant, Senior...  ...analyze complex problems, manage assigned workstreams, build...  ...payer functions and systems such as Oracle OHI, Epic...  ...workstreams- Using AI fluency, digital tooling...  ...with team members. We evaluate these factors thoughtfully... 
    Full time
    H1b

    PwC

    San Francisco, CA
    3 days ago
  • $100k - $120k

     ...clients on how to improve our operations. Hire, develop, train, and...  ..., expense reviews, and managing the operation's budget.  Identify...  ...computer applications/systems (e.g., Microsoft Office Suite...  ...decision tool (AEDT) to assess or evaluate your candidacy for employment... 
    Temporary work
    Work at office
    Flexible hours

    Metropolis Technologies

    San Francisco, CA
    18 days ago
  •  ...products that bring powerful AI into people’s everyday lives....  ...researchers, engineers, designers, and operators who care deeply about creating...  ...AI in the future. The Systems Integration team is critical...  ...the Role As a Lab Operations Manager, Systems Integration , you will... 
    Full time

    OpenAI

    San Francisco, CA
    3 days ago
  • $159k - $196k

     ...Waymo’s Technical Program Managers and Program Managers...  ...complex technical and operational challenges with no...  ...machine learning and/or AI ~ Experience with machine...  ...pipelines, or evaluation frameworks ~ Excellent...  ...environments for autonomous systems, model validation... 
    Full time
    Remote work

    Waymo

    San Francisco, CA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Operations Manager - AI Systems Evaluator. Be the first to apply!