Operations Manager - AI Systems Evaluator
$70 - $110 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: General and Operations Managers
Type: Contract
Compensation: $70–$110/hour
Location: Remote
Duration: 4–6 weeks
Commitment: 20+ hours/week
Role Responsibilities
- Evaluate AI-generated operational plans , staff schedules, and performance reviews for quality and accuracy.
- Compare AI-generated responses based on operational soundness, financial rigor, and cross-department feasibility.
- Review operational scenarios involving budgeting, staffing, workflow coordination, and performance improvement decisions. Provide expert feedback.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- 4+ years of experience as a General Manager, Operations Manager, Operations Director, Business Manager, District Manager, or another closely related general operations management role.
- Experience directing multi-department or multi-location operations, including budgeting, staffing, and performance oversight.
- Ability to commit 20+ hours per week .
- Strong written English communication skills.
Start Date
- Immediate
Compensation & Legal
- Task-based payment upon first task approval. Experts who complete their first task within the required timeline will then be qualified to be paid hourly for the remainder of the project.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$80k - $125k
...industrializing services through AI, starting with one of... ...built to solve the systemic data problem that's... ...% of the Fortune 100, manages over $30B in supply... ...the Role The AI Operations team owns the agents and... ...lives, building and evaluating the agents that remove...SuggestedHourly payContract workWork experience placementWork at office$40.4 - $44.89 per hour
...reframe old problems, seek new solutions and operate comfortably in areas that are unknown.... .... Role Summary: Our Autonomy Systems Validation's team is expanding, and we are... ...and perform data analytics. Develop and manage detailed operational reports, dashboards...SuggestedFull timeTemporary workPart timeFlexible hoursShift workAfternoon shift$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and...SuggestedContract workSummer workRemote work- ...Provider Growth & AI Agent Operations Lead San Francisco Location: San Francisco (Rincon Hill) Company... ...of marketplace, SaaS, and public sector systems, increasingly powered by AI. Wonderschool helps providers manage operations, governments run large-scale childcare...SuggestedWork at officeFlexible hours
- Nen Creative is seeking an Operations Lead to own day-to-day business operations, spanning finance, contracts, people operations, and internal systems. You’ll be a natural problem solver who thrives in a fast-paced, high-growth environment, taking initiative and building...SuggestedRemote job
$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks... ...cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure...Contract workSummer workRemote work$170k - $220k
...reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for... ...About the role Anthropic's legal team operates at the frontier of some of the most... ...functional partners such as Security and IT to manage retention, preservation, collection, and...Work at officeLocal areaVisa sponsorshipFlexible hours- ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use Gemini, Instinct, or Muse and can assess prompts and responses in English. A part-time commitment of 20 hours per week is available...Full timePart timeImmediate start
$140k - $190k
...Job Description Position: Product Manager - AI-Native Business Systems Location: South San Francisco... ...beyond R&D into a phase of massive operational scale. As our breakthrough "Boosted... ...Evals & Quality Standards: Stand up evaluation frameworks for operations and...Full timeLocal areaVisa sponsorship- ...About the Team The User Operations team (Support) is... ...orchestrated engine powered by AI-native workflows,... ...increasingly interconnected systems. We are looking for a... ...Technical Systems Program Manager to help drive the... ...define scalable solutions, evaluate technical tradeoffs,...Full timeWork at officeRelocation package
- ...together all of the workforce systems that are normally... ...time ever, you can manage and automate every part... ...You'll sit on our Growth Operations team, working across Marketing... ...and Sales, using AI to make both orgs best-... ..., prompted, automated, evaluated, or deployed AI tools and...Live inWork at office3 days per week
$8 - $65 per hour
Are you a general operations expert eager to shape the future of AI? Large‑scale language models are evolving... ...models on topics such as inventory systems, vendor management, SOP development, cost‑control... ...to our prompt engineering and evaluation metrics. A bachelor's or...Hourly payContract workFor contractorsRemote work- ...world At Bedrock were moving AI out of the lab and onto real... ...to work build the field and operations teams behind every deployment... ...the organization and the systems that keep data flowing ensuring... ...and experience either directly managing non-exempt employees or...Full timeFor contractors
- ...designed for fast, efficient AI inference. We combine large... ...a Data Center Facilities Operations Lead to own the critical facilities... ...make sure the facility-side systems that support Gimlet's compute... ...facility vendors, OEMs, Site Managers, Data Center Technicians,...Full timeWork at office
$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Larry Summers , and Jack Dorsey . Position: Incident management / reliability / SRE Evaluator Type: Contract Compensation: $80–$120/hour...Contract workSummer workWork at officeRemote work$10 - $30 per hour
...feedback digital delivery platforms ai-driven tools familiarity independent... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ...performance. Our pipeline performance system ensures both quality and velocity....Part timeFor contractorsRemote workShift work- ...Dispatch Role Nevoya is North America's leading next-generation electric trucking carrier, transforming logistics through AI-native operations and relentless customer focus. The company builds and uses proprietary technology to deliver zero-emissions carrier services...Work experience placementInternshipShift work
$99k - $232k
...a Cloud Infrastructure and Operations Delivery Manager you will oversee client-facing... ..., digital tooling, and AI-enabled workflows- Building... ...Information Technology, Information Systems, Engineering- At least one... ...with team members. We evaluate these factors thoughtfully...Full timeH1b$290k - $365k
...create reliable interpretable and steerable AI systems. We want AI to be safe and beneficial... ...Anthropic gives independent external evaluators access to our models our people and our... ...policies. Were hiring a Technical Program Manager to establish and deepen our partnerships...Full timeContract workWork at officeLocal areaVisa sponsorshipFlexible hours- ...digital core, optimize their operations, accelerate revenue growth and... ...expertise, advanced analytics, AI-powered tools, and a global delivery... ...Solution Architect Senior Manager, Source to Pay to play a... ...on integration requirements, system connectivity, and the role each...Contract workWork experience placementLive inWork at officeLocal area
$181.7k - $304.7k
...Salesforce Salesforce is the #1 AI CRM, where humans with... ...lead our EMEA Treasury Operations team while also owning our global Treasury Systems function. This is a... ...regional operational management with a strategic, cross... ...recruiters assess and evaluate candidates’ resumes and...Full timeWork at officeLocal area3 days per week$25 - $45 per hour
...Skills ChatGPT Muse AI agents Attention to... ...agents. We provide frontier evaluations and reinforcement learning... ...Our pipeline performance system ensures both quality and velocity... ...in engineering, business management, or office/operations Detail-oriented, great...Part timeFor contractorsWork at officeWeekend work- ...Head Of Infrastructure Operations (US)Houston; New York; San Francisco... ...the GPU cloud engineered for AI. We provide cost-effective,... ...business outcomes, including cost management, rapid innovation, and... ...power distribution, cooling systems, security, and environmental...Contract workFor contractors
$85k - $120k
...What is Verse? The race to AI has become the race to power. Every breakthrough... ...most ambitious organizations access and manage energy. The Role Verse operates a growing fleet of IoT-connected battery energy storage systems under a demanding availability SLA — with...Hourly payRemote workFlexible hoursNight shiftAfternoon shift$40.4 - $44.89 per hour
...reframe old problems, seek new solutions and operate comfortably in areas that are unknown.... ...generations. Role Summary Our Autonomy Systems Validation team is expanding, and we are... ...and perform data analytics. Develop and manage detailed operational reports, dashboards...Full timeTemporary workPart timeLocal areaFlexible hoursShift workAfternoon shift$134k - $179k
...CoreWeave is The Essential Cloud for AI™. Built for pioneers by pioneers, CoreWeave delivers a platform of technology,... ...more at . What You’ll Do: The DC Program and Cost Management team is the operational planning team responsible for the program and cost management...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hours$77k - $202k
...Strategy / Technology / Operations Consultant, Senior... ...analyze complex problems, manage assigned workstreams, build... ...payer functions and systems such as Oracle OHI, Epic... ...workstreams- Using AI fluency, digital tooling... ...with team members. We evaluate these factors thoughtfully...Full timeH1b$100k - $120k
...clients on how to improve our operations. Hire, develop, train, and... ..., expense reviews, and managing the operation's budget. Identify... ...computer applications/systems (e.g., Microsoft Office Suite... ...decision tool (AEDT) to assess or evaluate your candidacy for employment...Temporary workWork at officeFlexible hours- ...products that bring powerful AI into people’s everyday lives.... ...researchers, engineers, designers, and operators who care deeply about creating... ...AI in the future. The Systems Integration team is critical... ...the Role As a Lab Operations Manager, Systems Integration , you will...Full time
$159k - $196k
...Waymo’s Technical Program Managers and Program Managers... ...complex technical and operational challenges with no... ...machine learning and/or AI ~ Experience with machine... ...pipelines, or evaluation frameworks ~ Excellent... ...environments for autonomous systems, model validation...Full timeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Operations Manager - AI Systems Evaluator. Be the first to apply!
- program operations manager San Francisco, CA
- business operations manager San Francisco, CA
- director of network operations San Francisco, CA
- director operations support San Francisco, CA
- director revenue operations San Francisco, CA
- director of corporate operations San Francisco, CA
- entry level operations manager San Francisco, CA
- senior director clinical operations San Francisco, CA
- operations finance manager San Francisco, CA
- senior manager warehouse operations San Francisco, CA







