DevOps Engineer - AI Model Evaluator - AI Trainer
$85 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)
Type: Contract
Compensation: $85/hour
Location: Remote
Role Responsibilities
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks.
- Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability , and infrastructure automation .
- Identify bugs, edge cases, reliability issues, and failure modes in model outputs.
- Compare outputs from multiple frontier models to assess their strengths and weaknesses.
- Apply professional engineering judgment to realistic infrastructure engineering scenarios.
Qualifications
Must-Have
- 2+ years of professional DevOps , SRE , or Cloud Engineering experience.
- Experience with AWS , Azure , GCP , Kubernetes , Terraform , CI/CD pipelines , or observability tooling.
- Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
- Ability to evaluate model-generated infrastructure and reliability engineering solutions.
Preferred
- Experience supporting production-scale systems.
Compensation & Legal
- $400 per accepted task
- Compensation tied to accepted work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
#hiringmercorVacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the DevOps Engineer - AI Model Evaluator - AI Trainer in Miami, FL vacancy
$400 per month
...partnering with a leading AI research lab to support a... .... Contributors help evaluate and improve frontier AI coding models through structured technical... ...realistic infrastructure engineering workflows and model evaluation... ...2+ years of professional DevOps, SRE, or Cloud...Suggested$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Incident management / reliability / SRE Evaluator Type: Contract Compensation: $... ...independently to meet deadlines and improve AI model performance. Qualifications Must-...SuggestedContract workSummer workWork at officeRemote work$160k - $200k
...LinkedIn, X (Twitter), and Instagram. Role OverviewBlackstone's DevOps Engineering team enables application teams to securely build, deploy, and... ...solutions, and standardized development environments.Manage AI coding assistants, including Claude, and context management platforms...SuggestedFull timeLocal areaRemote work$16 - $17 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...independently and asynchronously to meet deadlines and improve AI model performance . Qualifications Must-Have Access to a personal...SuggestedContract workSummer workRemote workMercor
Miami, FLa month ago- ...secures Carnival Corp’s adoption of AI by ensuring GenAI, model, and agentic systems are deployed, validated... .... This position is a hands-on engineering role focused on AI security posture,... ...build, and run model validation and evaluation harnesses to test AI models and...SuggestedFull timePart timeWork at officeLocal areaWork from homeRelocationMonday to Thursday
$100k - $160k
...the place for you!We are looking for a DevOps Engineer to join our amazing Cloud Engineering team... ...smooth operations.Participate in evaluating and integrating new technologies to enhance... ...role.We may use artificial intelligence (AI) tools to support parts of the hiring process...Full timeWork experience placementLocal areaImmediate startRemote work- Rex.zone is seeking a Senior AI data annotator to perform data labeling and evaluation for NLP tasks, RLHF assessments, and prompt QA to improve training data quality and model performance. This is a US-based remote, full-time role aligned with Miami talent demand. You...Remote jobFull time
$50 per hour
Senior AI Interaction Evaluator (Codex / Claude Code) Contract | $50-200/hr | 10+ hrs/week | Project-... ...looking for highly experienced software engineer (SR+) to help evaluate the quality of... ...something harder: whether the model thinks like a great engineer. What This...Contract workCurrently hiring$40 per hour
AI Trainer - Materials Science Specialists (Remote Advisory) We are seeking AI Trainers - Materials Science Specialists to join our expert... ...graphical abstract doesn't accurately represent the scientific paper. Improving cutting‑edge AI models. #J-18808-Ljbffr ProlificRemote job$78 - $92 per hour
Overview Senior DevOps Engineer - Multi-Cloud (AI Platform & FinOps) - Job #7350 - 12-month W-2 contract (likely extension). Rate: $78 - $92 per hour... ...workflows in Argo CD or Jenkins to retrain and deploy models for multiple tenants. Package workloads in Docker and deploy...Hourly payContract workRemote work$40 per hour
A leading AI research organization is seeking AI Trainers specializing in Materials Science for remote advisory roles. Candidates should possess a Masters... ...reviewing scientific papers for accuracy and enhancing AI models. Participants can earn $40+/hr for tasks requiring...Remote job- ...defining it. Harnessing the full power of AI and digital innovation, we deliver... ...cloud infrastructure solutions using modern DevOps IaC and CICD processes; integrate solutions... ...recent experience working in a DevOps/System Engineering roleBachelor's Degree from an accredited...H1bLocal area
- G2i Inc. is seeking a Senior AI Interaction Evaluator to join our rolling talent bench for contract work. You will assess AI coding interactions... ...provide critical feedback on usefulness, correctness and engineering judgment. You will evaluate real-world AI behavior, judge...Contract work
$200k
...growing technology company who is in search for an seasoned Senior DevOps Engineer to join a crucial team that is integral for the day-to-day... ...in a new way. Experts add insights directly into each article, started with the help of AI. #J-18808-Ljbffr Forsyth BarnesFull timeRemote workRelocation package- ...a strong, hands‑on developer who can take internally built AI tools and turn them into secure, production‑ready applications... ...for Strong experience as a full‑stack developer, software engineer, or dev/devops hybrid Experience taking prototypes or messy internal tools...Immediate startRemote work
- ...is looking for fluent Spanish speakers from Spain, Argentina, Chile, Mexico, or Colombia for a remote freelance role in a high-impact AI training initiative for Hume. Participants will engage in live, conversational video calls to help train AI on human emotions and...Remote jobFreelance
- ...digital core and unleashing the power of AI to create value at speed across the enterprise... ....We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs,... ...clients on their biggest business challenges.Evaluate actionable solutions to these challenges,...Full timeWork experience placementLive inWork at officeLocal area
- ...digital core and unleashing the power of AI to create value at speed across the enterprise... ....We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs,... ...clients on their biggest business challenges.Evaluate actionable solutions to these challenges,...Full timeWork experience placementLive inWork at officeLocal area
$128k - $249k
....We are seeking a Senior Data Engineer to join the firm. The Senior Data... ...Applications, Data, and AI Platforms group, designing scalable... ...schema design, dimensional modeling, Medallion Architecture implementation... ...collaborating within an Agile DevOps model. Supporting CI/CD...Full timeTemporary workLocal areaRemote workFlexible hoursAfternoon shift- ...technology and leadership in cloud, data and AI with unmatched industry experience,... ...industry knowledge and applied AI and data engineering. We help the world’s leading Resources and... ..., AI platforms, and governance models that turn data into trusted, production-grade...Full timeWork experience placementLive inWork at officeLocal area
- ...and pressure-testing the models and proposals that drive... ...risk-based pricing engine, the application that powers... ...and propose & implement AI/ML solutions, and you will partner with DevOps, Product, and Engineering... .... Perform model evaluation and validation on a regular...Local areaFlexible hours
One Park Financial
Miami, FL3 days ago $102.5k - $188.9k
...We invite applications from highly skilled and experienced AI and Data Science Engineer III professionals ready to drive the development of our... ...scalable data pipelines, application programming interfaces, and model deployment workflows in cloud environmentsPartnering with...Visa sponsorship$134.5k - $265.1k
...At Deloitte, Forward Deployed Engineers (FDE) don’t just build AI solutions, they help clients turn AI... ...quality, safety, latency, cost, and model risk.Deliver production-quality code... ...evaluationExperience with MLOps/LLMOps practices: evaluation frameworks, model monitoring, and...Local area- ...-time analytics capabilities, and responsibly scale AI. As part of Accenture Cloud First and the Accenture... ...including BigQuery, Looker, Vertex AI, Gemini Foundation Models, and Gemini Enterprise.You AreA hands-on Engineer with foundational experience in Data Engineering,...Full timeWork experience placementLive inWork at officeLocal area
$128k - $252.5k
...services covering valuation modeling, cost optimization, restructuring... ...of tools, products, and AI services. You will work closely... ...interdisciplinary data scientists, engineers, architects, and consultants,... ...helping them understand and evaluate new and essential areas for...Local area- ...technology-driven organizations that drive true change.Our Data Architecture & Engineering team sees data as the foundation for decision-making, artificial intelligence, and transformation. We embrace AI to scale with quality, innovate without fear, and use technology with...Temporary workLocal area
$95k - $150k
...environment? As an experienced Senior Data Engineer you will have the ability to share new... ...Deloitte under our Project Delivery Talent Model. Project Delivery Model (PDM) is a talent... ...Serverless Compute, event hubs) The Team AI & Engineering leverages cutting-edge engineering...Permanent employment- ...experiences and drive long-term business growth.The Senior Data Engineer will play a key role in the design, development, and deployment... ...dbt to support business requirementsBuild and optimize data models and workflows to support analytics, reporting, machine learning...Full timeWork at office
$105.4k - $207.8k
...power the next generation of AI-enabled cyber defense?If yes,... ...are looking for a hands-on Data Engineer to build and operate the governed... ...data stores.Designing data models for risk and controls domains,... ...retrieval, tool or function calling, evaluation or monitoring, prompt-...Local areaVisa sponsorship- ...applying machine learning and AI techniques to real-world... ...develop and validate predictive models, and contribute to the implementation... .... Perform feature engineering, data exploration, and hypothesis... ...datasets. Build and evaluate models using appropriate AI/ML...Temporary workWork experience placementRemote workFlexible hours
United Automobile Insurance Company
Miami, FL1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to DevOps Engineer - AI Model Evaluator - AI Trainer. Be the first to apply!
Related searches
- devops engineer remote Miami, FL
- senior devops engineer remote Miami, FL
- devops engineer Miami, FL
- senior devops cloud engineer Miami, FL
- devops engineer full time Miami, FL
- devops aws developer (remote) Miami, FL
- evaluator Miami, FL
- ai evaluator Miami, FL
- quality evaluator Miami, FL
- education evaluator Miami, FL