Operations Manager - AI Systems Evaluator
$70 - $110 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: General and Operations Managers
Type: Contract
Compensation: $70–$110/hour
Location: Remote
Duration: 4–6 weeks
Commitment: 20+ hours/week
Role Responsibilities
- Evaluate AI-generated operational plans , staff schedules, and performance reviews for quality and accuracy.
- Compare AI-generated responses based on operational soundness, financial rigor, and cross-department feasibility.
- Review operational scenarios involving budgeting, staffing, and workflow coordination. Provide expert feedback.
- Leverage real-world expertise in directing multi-department operations, staffing, and financial oversight.
- Work independently and asynchronously to improve AI systems and meet deadlines.
Qualifications
Must-Have
- 4+ years of experience in a general operations management role.
- Experience directing multi-department or multi-location operations.
- Ability to commit 20+ hours per week .
- Strong written English communication skills.
Start Date
- Immediate
Compensation & Legal
- Task-based payment upon first task approval. Qualified experts will be paid hourly for the remainder of the project.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the Operations Manager - AI Systems Evaluator in New York, NY vacancy
$400 per month
Obsidian is seeking contributors for a project with a leading AI research lab. The role involves using frontier AI coding agents to... ...identify bugs, and compare outputs from different models. Strong systems engineering experience is essential, coupled with familiarity...Suggested- Mission North is seeking a strategic, systems-minded leader to own the People technology & operations function. You’ll shape the HRIS/ATS ecosystem, drive automation, and deploy AI-powered solutions to elevate employee experience and operational efficiency. You’ll partner...Suggested
$14.5 per hour
...Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology companies to provide datasets that are... ...understanding of pop culture in the US Reliable computer system and internet connection Familiar understanding of how to...SuggestedHourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
New York, NY5 days ago$20 - $30 per hour
A remote-focused technology firm is looking for an individual proficient in evaluating large language model outputs. The role involves assessing AI systems, reviewing workflows, and providing actionable feedback to enhance product quality. Ideal candidates will have strong...SuggestedRemote jobHourly pay$30 per hour
...Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems using defined rubrics and quality standards.... ...Requirements Strong experience in LLM evaluation, AI output analysis, QA/testing, UX research, or...SuggestedRemote jobHourly payContract work- A leading AI research accelerator is hiring a position focused on contributing to projects that evaluate and enhance AI systems. You will design community service scenarios, write structured... ...and a background in Recreation Management or Public Administration. This role...Remote jobFull timeFor contractors
$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Financial Managers Type: Contract Compensation: $... ...+ hours/week Role Responsibilities Evaluate AI-generated financial plans , budgets,...Hourly payContract workSummer workWork at officeImmediate startRemote work$85k - $110k
...AI Operations Associate The AI Operations Associate monitors AI-driven... ...internally built workflows, managing vendor SLAs, update cycles,... ...~ Strong analytical skills evaluating nuanced workflow/output... ...analytics, or workflow-heavy systems ~ Strong written communication...Work at officeLocal areaRemote work2 days per weekAccordion USA
New York, NY2 days ago$80k - $125k
...Loop AI Ops Team Member Loop is industrializing services... ...native company built to solve the systemic data problem that's held... ...serves 20% of the Fortune 100, manages over $30B in supply chain spend... ...are disrupting traditional operations with AI powered workflows that...Work experience placementLoop Defunct
New York, NY2 days ago- JPMorganChase AI Research (AIR) is a global team of research... ...scientists, engineers and product managers that develops novel AI... ...researchers across multi-agent systems, foundation model training, reinforcement... ...and responsibly.As Business Operations Lead, VP/ED, supporting AI...Work at office
JP Morgan Chase
New York, NY15 hours ago $7.5k
...Job Description Mission Operations & AI Enablement Lead OceanX... ...the adoption of AI-enabled systems, workflow automation, and operational... ...in operations, project management, systems implementation, or... ...improvement. Research and evaluate international maritime regulations...Summer workCasual workSeasonal workWork at officeRemote workFlexible hoursAfternoon shiftEarly shiftOceanX
New York, NY28 days ago- ...neobanks, platforms, developers, and AI agents. Our technology allows... ...Rain's foundational AI systems, tooling, and other primitives... ...long-term impact. Establish Operational Rigor: Define success metrics... ...Excellent communicator. You can manage expectations, maintain...Work at officeWork from homeWorldwideFlexible hoursShift work
Fern
New York, NY3 days ago - ...AI Operations AssociateWe are looking for a detail-oriented AI Operations Associate to support the day-to-day operations of... ...scientists, product teams, and business stakeholders to help manage AI workflows, monitor system performance, maintain data quality, and ensure smooth...InternshipFlexible hours
Ova Technologies
New York, NY3 days ago - A mission-driven organization is looking for a Senior Director of Operations to lead core business functions in a remote-first setting. This role focuses on strategic execution, building scalable systems, and optimizing workflows, while partnering with leadership. Ideal...Remote work
Jobgether
New York, NY3 days ago $90k - $153k
...Systems Operations SpecialistAs a Systems Operations Specialist reporting to the VP - Systems and... ...infrastructure provisioning, configuration management, and operational workflows across cloud... ...automation, with experience leveraging AI-assisted tools such as GitHub Copilot,...Temporary workH1bWork at officeHome officeFlexible hours3 days per weekNasdaq
New York, NY15 hours ago- About the Opportunity A leading AI research organization is seeking advanced LLM power users with strong experience using... ...for real-world personal life tasks. This project focuses on evaluating how well AI systems handle personalized, multi-step life tasks that require...Trial period
$100k - $155k
...with the world's most advanced AI-native platform. We work on large scale distributed systems, processing almost 3 trillion... .... We are seeking an AI Operations Manager to help establish the operating... ...intake process, aggregating and evaluating requests such as AI system...Remote jobFull timeWork experience placementWork at officeLocal area$60k - $65k
...Information Systems Operations Specialist In collaboration with firm support professionals, the Operations Specialist will deploy, secure, maintain, track, and assess the firm’s telecommunications, networking, server, and storage equipment infrastructure inventory across...Full timeWork at officeDavis Polk
New York, NY3 days ago- MetroPlusHealth is seeking a Core Operations Manager to evaluate and improve core systems and applications. You will partner with Provider Contracting, Credentialing, Claims Operations, Utilization Management, and Network Management to align systems with department and...Remote job
- We are Allvue Systems, the leading provider of software solutions for the... ...Job Summary This role owns the operational engine of the Legal department:... ...stand up the ticketing and matter management systems behind it, and apply our approved AI and automation tools to take...Contract workWork at officeFlexible hours
$192k - $278k
Technical Program Manager, Systems Security Portfolio, Product Security PMO Share Technical Program... .... Experience in infrastructure, operating systems, hardware platforms (GPUs/TPUs),... ...management. Familiarity with securing AI/agentic environments, ML fleet platforms...Work experience placement$60 per hour
A leading project-based AI consultancy is seeking legal consultants to evaluate AI systems and improve their reasoning. This non-permanent role requires a law degree and at least 2 years of experience in US law. Candidates must have strong written English skills and a...Permanent employmentPart timeFreelanceFlexible hours- To support the growth of the Enterprise AI team, the full-time AI Operations Manager will design and implement AI-native processes, oversee operational reporting... ...the publication of key metrics across various systems Administer the Enterprise AI intake process and facilitate...Full timeRemote work
$120k - $160k
...Job Description Job Description IT Operations and Service Delivery Lead- NYC Area IT Manager - High Growth AI Infrastructure Company Location: NYC Area. Primarily... ...of progressive experience in IT operations, systems administration, endpoint engineering, or IT service...Ongoing contractLocal areaWork from homeCyberCoders
New York, NY6 days ago$51.2k - $74.15k
...assessment, we’re advancing AI to move from insight to action... ...efficiency. Interest in exploring AI systems and a willingness to develop... ...AI practices, including risk management and ethical use. Education... ...Responsibilities As a Data Operations Specialist II, you will...Full timeWork experience placementInternshipWork at officeMoody's
New York, NYa month ago$270k - $365k
...Data Operations Manager, Human DataSan Francisco, CA | New York City, NYAbout... ...teams working on frontier AI capabilities. You'll partner... ...scaleDesign and implement operational systems that translate research... ...quality data pipelinesBuild evaluation frameworks and quality...Work at officeVisa sponsorshipFlexible hoursShift workAnthropic
New York, NY3 days ago- ...talent function and build a world-class team. This role combines recruiting, HR operations, onboarding, compliance, and culture initiatives, with an emphasis on leveraging internal AI systems to streamline processes. The position requires strong organizational skills,...Work at office
- Flagler Health is building the clinical operating system for modern musculoskeletal care. We partner... ...phase of growth. Role We are hiring an AI Operations Associate to design, build,... ...operational leverage. This is not a "sit back and manage" role. As AI Operations Associate, you...
- Crowe in New York seeks a Head of Revenue Operations to design and lead the firmwide revenue operating system, uniting lead generation, opportunity development, close... ...with practical insights, streamlined processes, and AI-enabled tools. You will partner across partners,...
- Checkbox, a Sequoia-backed tech startup, is seeking a Revenue Operations Lead in New York (hybrid, full-time). You will own GTM systems, playbooks, data, and cross-functional collaboration to drive growth and operational excellence. You will partner with Finance and GTM...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Operations Manager - AI Systems Evaluator. Be the first to apply!
Related searches
- print operations manager New York, NY
- senior manager customer operations New York, NY
- director of network operations New York, NY
- regulatory operations manager New York, NY
- development operations manager New York, NY
- operations compliance manager New York, NY
- manager creative operations New York, NY
- business operations director New York, NY
- director of facility operations New York, NY
- retail operations manager New York, NY