Operations Manager - AI Systems Evaluator
$70 - $110 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: General and Operations Managers
Type: Contract
Compensation: $70–$110/hour
Location: Remote
Duration: 4–6 weeks
Commitment: 20+ hours/week
Role Responsibilities
- Evaluate AI-generated operational plans , staff schedules, and performance reviews for quality and accuracy.
- Compare AI-generated responses based on operational soundness, financial rigor, and cross-department feasibility.
- Review operational scenarios involving budgeting, staffing, workflow coordination, and performance improvement decisions. Provide expert feedback.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- 4+ years of experience as a General Manager, Operations Manager, Operations Director, Business Manager, District Manager, or another closely related general operations management role.
- Experience directing multi-department or multi-location operations, including budgeting, staffing, and performance oversight.
- Ability to commit 20+ hours per week .
- Strong written English communication skills.
Start Date
- Immediate
Compensation & Legal
- Task-based payment upon first task approval. Experts who complete their first task within the required timeline will then be qualified to be paid hourly for the remainder of the project.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Operations Manager - AI Systems Evaluator in San Francisco, CA vacancy
- BridgeBio Pharma is looking for a Senior People Operations Systems Specialist to manage HR operations across platforms like Workday and ServiceNow. This role involves handling employee lifecycle transactions, ensuring compliance, and improving self-service tools in a dynamic...Suggested
- Domu is seeking an Operations Manager to enhance our operational model for outbound calling. The right candidate will have over 5 years of... ...dialing infrastructure and the ability to build operational systems. You will work closely with the engineering and client success...Suggested
- ...organizations use increasingly capable AI systems effectively, responsibly, and with confidence... ..., and programs grow, we need an operating foundation that keeps the content portfolio... ..., learning operations, knowledge management, or a related field. - Experience owning...SuggestedFull timeWork at office
OpenAI
San Francisco, CA18 days ago - About the Team The User Operations team (Support) is... ...orchestrated engine powered by AI-native workflows,... ...increasingly interconnected systems. We are looking for a... ...Technical Systems Program Manager to help drive the... ...define scalable solutions, evaluate technical tradeoffs,...SuggestedWork at officeRelocation package
- Pilot.com, Inc. is seeking a data‑driven operations professional to own work orchestration, KPI reporting, and forecasting across the Customer Operations function. You will build scalable models, automate workflows, and partner with senior leadership to drive efficiency...Suggested
- YouTube, part of Google, seeks a Manager of Systems Administration and Full Stack Operations to lead Platform Technology Leads responsible for stability and integrity... ...of critical customer support tech. You will drive AI/ML initiatives and scale operations across...
$245k - $265k
Ironclad is the leading AI contracting platform that transforms... ...for Contract Lifecycle Management, a Fortune Great Place to Work... ...at Ironclad is building the operating foundation that helps the company... ...next chapter requires stronger systems, sharper data, cleaner...Full timeContract work- ...together all of the workforce systems that are normally... ...time ever, you can manage and automate every part... ...You'll sit on our Growth Operations team, working across Marketing... ...and Sales, using AI to make both orgs best-... ..., prompted, automated, evaluated, or deployed AI tools and...Live inWork at office3 days per week
- ...offering a part-time role for PhD-level Chemistry experts to contribute to AI safety and evaluation projects. The work involves applying scientific expertise to understand and improve how AI systems handle specialized chemistry topics, with training provided for the...Remote jobPart time
$136.85k - $185.15k
...You'll Do As the Marketing Operations and Analytics Program Manager, you’ll work closely with... ...Customer Engagement, GTM Systems, and wider Marketing team... ...for measuring and evaluating marketing performance, supporting... ...who can apply AI and machine learning to automate...$207.4k - $372.7k
AIML - Senior Operations Program Manager - Responsible AI and Safety San Francisco Bay Area, California, United... ...data operations behind our Safety Evaluation and Red Teaming programs, scaling... ...current documentation of processes, systems, and data flows across these functional...Relocation- Welo Data in San Francisco is hiring Data Labeling Associates for Project Perseus. This role focuses on evaluating Arabic AI systems, requiring professional proficiency in Portuguese and experience in AI safety. Responsibilities include assessing AI outputs, identifying...Full time
- ...About Broccoli AI Broccoli AI is building the AI operating system for home service businesses . We work with plumbing, HVAC, and electrical contractors —... ...opportunities and prioritize ruthlessly. Set up the evaluation, monitoring, and iteration loops that keep AI agents...For contractors
Broccoli AI
San Francisco, CA2 days ago $80k - $125k
...Loop AI Ops Team MemberLoop is industrializing services through... ...company built to solve the systemic data problem that's held logistics... ...20% of the Fortune 100, manages over $30B in supply chain spend... ...are disrupting traditional operations with AI powered workflows that...Work experience placementLoop Defunct
San Francisco, CA1 day ago- ...across the full model and systems stack, we integrate cutting... ...seamlessly blend high‑level AI capabilities with the physical... ...Role We are looking for an Operations Program Manager - Robotics Data Acquisition... ...and ready for rollouts and evaluations. Lead root cause analysis...
- ...social justice, dedicated to fighting systemic injustices, creating pathways out of... ...and marginalization The Church Operations & Ministry Support Manager (Internally known as Office Manager)... ...We may use artificial intelligence (AI) tools to support parts of the hiring...Full timeWork at officeWeekday work
GLIDE
San Francisco, CA16 days ago $148.84k - $175.1k
Coinbase is seeking a Technical Program Manager focused on improving knowledge systems within customer support operations. You'll work cross-functionally to bring programs from concept to execution, ensuring efficient workflows and customer satisfaction. Applicants should...$290k - $365k
...interpretable, and steerable AI systems. We want AI to be safe and... ...Role GTM Systems builds and operates the platform Anthropic's sales... ...hiring a Technical Program Manager to own program delivery for... ...build-vs-buy calls, and vendor evaluation Help build the operating...Contract workFor contractorsFor subcontractorWork at officeVisa sponsorshipFlexible hours- OpenAI is seeking a Technical Systems Program Manager in San Francisco to drive strategy, alignment,... ...will partner with Engineering, Revenue Operations, Data, and Enterprise Platform teams... ...operationalize scalable CRM workflows and AI-native automation, influencing cross-...Work at officeRelocation package
$99k - $232k
...Description & SummaryThe OpportunityAs an AI P&C Insurance Operations Solutions Architect, Manager, you will specialize in... ..., Management Information Systems, Mathematics, Mechanical Engineering... ...and RAG / GraphRAG, and treating evaluation as a discipline through guardrails...Full timeTemporary workH1b- About the Team OpenAI’s Hardware organization develops AI-native silicon and system-level solutions for the unique demands of advanced AI... ...platform. About the Role We are seeking a Operations Program Manager (OPM) to serve as the single-threaded operational leader...Contract work
- OpenAI seeks a Sales Strategy & Operations Lead to build the GTM operating system for ChatGPT Ads. You will work across sales strategy, segmentation, forecasting... ...models, build analytical frameworks, and program manage high-priority sales initiatives for a global, multi-...
- ...a doctor's office, the system forgets you the moment... ...for every patient. An AI agent that knows who you... ...is embedded into how we operate, not layered on top. We... ...Standardize how we build, evaluate, and scale AI... ...prioritization instincts; you've managed a roadmap with more...Work at office
- Hover is seeking a Customer Experience leader to scale AI-powered support and knowledge systems. You will own the automation roadmap, design HITL workflows, and collaborate with CX, Engineering, and BizOps to hit key KPIs while guiding a hybrid US team across our SF and...
$190k - $220k
...in Los Angeles is seeking a Revenue Operations Lead to enhance its AI-driven revenue model. This role requires ownership of revenue signals management and collaboration across product and... ...with hands-on experience in building systems. The position offers a salary range...- Canva is seeking a driven Partnerships Operations leader to own the end-to-end Partner Relationship Management platform and related processes. You will manage an onshore... ...contract operations, while building scalable systems for partner enablement and co-sell. You will...Contract work
- Kastle is building the AI employee for consumer lending in San Francisco. We seek a mortgage originations expert to map end-to-end... ...define every touchpoint and data move between LOS, POS, and related systems. You will run QA and error triage, feed structured corrections...
$168k - $210k
...grows, so does the need for the systems, structure, and data that... .... As Talent Acquisition Operations AI Enablement Lead, you are the... ...Guide, Claude and Zapier. Evaluate and recommend new tooling as... ...and metrics that give hiring managers and TA leaders real‑time visibility...Summer holidayLocal areaFlexible hoursShift work- ...charging network and digital media services in the United States. The Operations Specialist will troubleshoot hardware, monitor systems, and ensure reliable station performance while leveraging AI tools to optimize operations and customer outcomes. Ideal candidates have...
- About Overview AI Overview AI is a VC‑funded, YC‑backed company building edge AI vision systems for manufacturing quality inspection. Our all‑in‑one, edge‑enabled cameras... ...Role We’re looking for a high‑ownership GTM Operations Lead to work closely with sales leadership...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Operations Manager - AI Systems Evaluator. Be the first to apply!
Related searches
- strategy & operations manager San Francisco, CA
- program operations manager San Francisco, CA
- operations manager supervisor San Francisco, CA
- web operations manager San Francisco, CA
- partner operations manager San Francisco, CA
- managing director of operations San Francisco, CA
- hvac operations manager San Francisco, CA
- senior director of operations San Francisco, CA
- network operations center manager San Francisco, CA
- associate director clinical operations San Francisco, CA