DevOps Engineer for AI Model Evaluation

Contribute to evaluations that improve frontier AI coding models by completing and assessing realistic infrastructure engineering tasks. You will work with model-generated code and implementations that touch cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation, applying professional engineering judgment to identify failure modes and reliability issues.

• Use frontier AI coding agents to complete complex infrastructure engineering tasks, then evaluate the results.
• Review model-generated implementations for cloud platforms, Kubernetes, CI/CD, observability, and infrastructure-as-code.
• Identify bugs, edge cases, reliability problems, and failure modes in model outputs.
• Compare and contrast outputs from multiple frontier models to assess strengths and weaknesses.
• Apply professional engineering judgment to realistic scenarios to judge correctness, safety, and operational readiness.

• At least 2 years of professional experience in DevOps, SRE, or Cloud Engineering.
• Hands-on experience with one or more cloud providers, such as AWS, Azure, or GCP.
• Familiarity with Kubernetes, Terraform, CI/CD pipelines, and observability tooling.
• Regular use of AI coding agents, for example Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools.
• Ability to evaluate model-generated infrastructure and reliability engineering solutions.
• Experience supporting production-scale systems is preferred.

• Remote engagement.
• Hourly employment model.
• Work is organized in sprint windows that run in 12 to 24 hour stretches, scheduled per client requirement.
• Typical tasks require about 2 to 3 hours of work after an initial ramp-up.
• Spots are limited and are filled on a first come, first serve basis.
• Project participation and assignment depend on client project availability.

• Hourly rate stated: $85 per hour.
• Alternate task-based pay: $400 per accepted task.
• Payment is tied to accepted work.

• Meets the qualifications listed above, including the 2+ years of relevant professional experience.
• Capability to use and evaluate outputs from AI coding agents.
• No additional work-authorization details were specified, applicants should ensure they are eligible to work remotely under the engagement terms.