ML Infrastructure Engineer

Job Description

Job Description

About the Role

This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context layer that makes AI agents reliable, accurate, and secure for mission-critical business operations. You'll own the systems that keep those agents running fast and reliably in production — from design through deployment — working closely with ML and infrastructure teams to scale inference at increasing concurrency.

• Own inference and model-serving infrastructure end to end, from architecture design through production deployment.

• Build and scale systems that enable AI agents to run reliably and efficiently under high concurrency in production environments.

• Collaborate with ML and infrastructure teams to ensure seamless integration and drive performance optimization.

• Identify infrastructure bottlenecks and lead the engineering effort to resolve them.

• 5+ years of experience building and operating machine learning inference systems, model-serving platforms, or ML infrastructure in production environments.

• Hands-on experience designing and scaling inference-serving infrastructure using tools such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom systems.

• Demonstrated ability to optimize production ML systems for latency, throughput, and reliability at scale.

• Strong proficiency with containerization and orchestration technologies — Docker and Kubernetes — for deploying ML workloads.

• Experience building or maintaining distributed systems that handle concurrent requests and manage resource allocation under load.

• Solid command of monitoring, observability, and debugging tooling for production systems (e.g., Prometheus, Grafana, ELK, distributed tracing).

• Experience deploying and managing ML systems on cloud platforms such as AWS, GCP, or Azure.

• Proficiency in at least one systems or backend language: Python, Go, Rust, C++, or Java.

• Experience with knowledge graphs, semantic search, or graph databases (e.g., Neo4j, Amazon Neptune) is a plus.

• Familiarity with real-time or low-latency inference systems, agentic AI pipelines, or enterprise data infrastructure is a plus.

On-site in San Mateo, California, United States. Visa sponsorship is not available for this role.