AI Infrastructure Engineer

Job Title: AI Infrastructure Engineer

Job Summary

We are seeking an AI Infrastructure Engineer to design, implement, and manage the infrastructure that powers AI and machine learning workloads. In this role, you will build scalable, secure, and high-performance environments for model training, inference, and deployment while optimizing compute, storage, networking, and GPU resources. You will work closely with AI/ML engineers, platform engineers, data engineers, and DevOps teams to ensure reliable and efficient AI operations.

Key Responsibilities

• Design, deploy, and maintain AI infrastructure across cloud and on-premises environments.
• Build and manage GPU-enabled compute clusters for machine learning training and inference.
• Implement scalable infrastructure for distributed AI workloads.
• Deploy and manage Kubernetes clusters for containerized AI applications.
• Automate infrastructure provisioning using Infrastructure as Code (IaC).
• Develop and maintain CI/CD pipelines for AI infrastructure and services.
• Optimize compute, storage, networking, and GPU utilization to improve performance and reduce costs.
• Monitor infrastructure health, availability, capacity, and performance using observability tools.
• Implement security best practices, identity management, secrets management, and compliance controls.
• Support AI model deployment platforms and inference infrastructure.
• Troubleshoot infrastructure, networking, and performance issues affecting AI workloads.
• Collaborate with AI engineers, ML engineers, data engineers, and cloud teams to improve platform reliability and scalability.
• Evaluate and implement emerging infrastructure technologies for AI workloads.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4+ years of experience in Infrastructure Engineering, Cloud Engineering, Platform Engineering, or DevOps.
• Strong experience with Linux system administration.
• Proficiency in Python, Bash, or Go for infrastructure automation.
• Hands-on experience with Docker and Kubernetes.
• Experience with one or more cloud platforms: AWS, Microsoft Azure, or Google Cloud Platform.
• Experience with Infrastructure as Code tools such as Terraform or Pulumi.
• Strong understanding of networking, storage, load balancing, and security.
• Experience with CI/CD tools such as GitHub Actions, GitLab CI, or Jenkins.
• Knowledge of monitoring and logging tools such as Prometheus, Grafana, ELK Stack, or OpenTelemetry.

Preferred Qualifications

• Experience managing NVIDIA GPU infrastructure and CUDA environments.
• Experience with distributed computing frameworks such as Ray, Apache Spark, or Slurm.
• Experience with AI model serving frameworks such as NVIDIA Triton Inference Server, KServe, or Ray Serve.
• Familiarity with MLOps tools such as MLflow, Kubeflow, or Airflow.
• Experience with vector databases and Generative AI infrastructure.
• Knowledge of storage technologies for AI workloads, including object storage and distributed file systems.
• Experience with high-performance computing (HPC) environments.
• Familiarity with infrastructure security, compliance, and governance standards.

Technical Skills

• Linux
• Python
• Bash
• Go (preferred)
• Docker
• Kubernetes
• Terraform or Pulumi
• AWS / Azure / Google Cloud Platform
• NVIDIA GPU Technologies
• CUDA
• Prometheus
• Grafana
• OpenTelemetry
• Git
• GitHub Actions / GitLab CI / Jenkins
• PostgreSQL
• Redis
• Ray
• Apache Spark
• Slurm
• NVIDIA Triton Inference Server
• KServe

Soft Skills

• Strong analytical and troubleshooting abilities
• Excellent communication and collaboration skills
• Ownership and accountability
• Ability to work in cross-functional teams
• Strong documentation and operational excellence mindset
• Continuous learning and adaptability

Nice to Have

• Experience supporting Large Language Models (LLMs) and Generative AI platforms
• Experience with Retrieval-Augmented Generation (RAG) infrastructure
• Knowledge of AI infrastructure cost optimization strategies
• Experience with multi-cloud or hybrid-cloud deployments
• Cloud, Kubernetes, or Linux certifications

Location

Hybrid / Remote / On-site (as applicable)

Employment Type

Full-time