Senior MLOps Engineer

Job Title

Pay Range: $60hr - $65hr

Requirement/Must Have

• 3+ years of experience building production backend or distributed systems with pre-AI experience.
• Proven experience shipping AI/LLM features serving real users at scale, not just prototypes or demos.
• Experience building AI agents, skills, tools, or MCP (Model Context Protocol) integrations.
• Strong proficiency in Python for backend development.
• Working knowledge of Go, TypeScript, or Rust.
• Deep experience with AWS, GCP, or Azure, including cost optimization and compute decisions.
• Hands-on experience with Docker and Kubernetes, including building, deploying, debugging, and scaling services.
• Strong understanding of LLM integration, including token economics, context limits, rate limiting, structured outputs, and API failure modes.
• Strong understanding of LLM evaluation, including non-determinism, quality measurement, and regression detection.
• Hands-on engineering mindset with the ability to write code, debug production issues, and deploy work independently.

Responsibilities

• Build intelligent, data-driven platform capabilities for next-generation test analytics and test agents.
• Develop automated evaluation tools for AI and human-based assessment systems.
• Conduct rigorous statistical analyses to ensure reliability and performance.
• Benchmark, adapt, and integrate AI/ML models into existing software systems.
• Independently run and analyze ML experiments to drive real improvements.
• Build scalable infrastructure for Generative AI systems connecting test stations, line-level data, and pipelines.
• Deploy, manage, and scale AI services in production environments.

Should Have

• Experience building multi-step agentic workflows with tool use and function calling.
• Experience with agent orchestration frameworks such as LangGraph, CrewAI, or custom frameworks.
• Experience building guardrails, fallbacks, or graceful degradation for AI systems.
• Experience with streaming inference and async agent orchestration.
• Experience with cost and latency optimization techniques such as caching, batching, and prompt compression.
• Familiarity with ML observability tools such as Langfuse, Arize, Braintrust, and Weights & Biases.
• Experience with retrieval systems such as vector search and hybrid search.

Skills

• Python.
• Go.
• TypeScript.
• Rust.
• AWS.
• GCP.
• Azure.
• Docker.
• Kubernetes.
• AI/LLM integration.
• Agentic systems.
• MCP integrations.
• ML experimentation.
• Statistical analysis.
• CI/CD and scalable infrastructure.

Qualification and Education

• Strong engineering background with hands-on experience in backend systems, AI systems, and cloud-native infrastructure.