Senior AI Architect - Private AI

Make an impact with NTT DATA
Join a company that is pushing the boundaries of what is possible. We are renowned for our technical excellence and leading innovations, and for making a difference to our clients and society. Our workplace embraces diversity and inclusion – it’s a place where you can grow, belong and thrive.

Private AI Architect
Experience: 10 to 13 years (5+ years in AI/ML & Enterprise AI Architecture)

Locations: Bangalore/Chennai/Delhi NCR/Pune/Hyderabad

ROLE OVERVIEW
We are seeking an experienced Private AI Architect to design and deliver enterprise-scale AI and GenAI platforms in private cloud, on-premises, and hybrid environments. This role spans architecture, presales, and delivery, with a focus on building secure, scalable, and sovereign AI platforms using open-source technologies, self-hosted LLMs, and agentic AI systems. The candidate will act as a trusted advisor, driving AI transformation programs while ensuring data privacy, governance, and regulatory compliance.
KEY RESPONSIBILITIES & EXPECTED EXPERIENCE
• Demonstrate hands-on experience in designing, building, configuring, deploying and troubleshooting enterprise-grade Private AI/GenAI platforms using open-source and self-hosted technologies. Anirban: To emphasize the hands-on nature of the role.
• Creation of High-Level Design, Low-Level Design, platform blueprints, deployment patterns, and Private AI reference architectures for on-prem, private cloud, hybrid, and sovereign AI environments.
• Demonstrate hands-on experience in developing and implementing LLM, RAG and agentic AI solutions, including ingestion pipelines, vector databases, tool integrations, orchestration and evaluation components architectures for on-premises, private cloud, and hybrid deployments. Anirban: To emphasize the hands-on nature of the role.
• Build and operationalize AI platforms using Kubernetes, containerization, and modern platform engineering practices
• Implement MLOps/LLMOps frameworks for model training, deployment, monitoring, and governance
• Demonstrate hands-on experience in deploying, configuring and optimizing self-hosted foundation models and enterprise AI workloads, including GPU utilization, quantization, batching, autoscaling, latency tuning and production troubleshooting. Anirban: To emphasize the hands-on nature of the role.
• Develop reference architectures, accelerators, and reusable frameworks for Private AI adoption
• Lead RFPs, proposals, and estimations for AI engagements
Advise clients on AI strategy, platform modernization, infrastructure readiness, and AI operating models
• Collaborate with infrastructure, cloud, security, and data engineering teams to establish enterprise AI foundations
• Drive POCs, MVPs, and enterprise AI adoption initiatives
REQUIRED SKILLS & QUALIFICATIONS
Experience
• 10 to 13 years IT experience; 5+ years in AI/ML architecture
• Proven delivery of enterprise AI/GenAI solutions in private cloud, on-premises, or hybrid environments
Private AI & GenAI Expertise
• Strong understanding of self-hosted LLMs, open-source AI ecosystems, and enterprise AI platforms
• Expertise in LLMs, RAG, vector databases, prompt engineering, and agentic frameworks
• Experience with frameworks such as LangChain, LlamaIndex, Haystack, Semantic Kernel, or similar
• Familiarity with open-source models such as Llama, Mistral, DeepSeek, Qwen, Falcon, or equivalent
Architecture & Infrastructure
• Expertise in MLOps/LLMOps lifecycle and governance - prompt/version management, model evaluation, hallucination testing, tracing, monitoring, drift detection, feedback loops, quality metrics, and production support.
• Strong understanding of Kubernetes, Docker, OpenShift, platform engineering, and distributed systems
• Experience with AI infrastructure including GPUs, model serving frameworks, inference optimization, and vector databases
• Knowledge of data platforms, data pipelines, and enterprise integration architectures
• Experience with technologies such as MLflow, Kubeflow, KServe, Ray, vLLM, Ollama, or similar is preferred
• Quantization, batching, GPU utilization, latency optimization, autoscaling, and high-availability inference design.
• GPU sizing, NVIDIA stack, CUDA, MIG, distributed inference/training, storage throughput, networking, capacity planning, workload scheduling, and infrastructure readiness assessment.
Security, Privacy, and Sovereign AI Controls
• Data isolation, zero-trust architecture, IAM/RBAC, encryption, secrets management, audit logging, secure model hosting, tenant isolation, data residency, and regulatory compliance.
Client & Leadership
• Experience in RFPs, proposals, estimations
• Strong stakeholder management and communication skills
PREFERRED QUALIFICATIONS
• Experience building enterprise AI platforms from the ground up using open-source technologies
• Exposure to NVIDIA AI Enterprise, Red Hat OpenShift AI, VMware Private AI, or similar ecosystems
• Experience with air-gapped, sovereign, or highly regulated AI environments
• Exposure to Responsible AI and regulatory environments (BFSI, Healthcare, Government, etc.)

About NTT DATA
NTT DATA is a $30+ billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long-term success. We invest over $3.6 billion each year in R&D to help organizations and society move confidently and sustainably into the digital future. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure, and connectivity. We are also one of the leading providers of digital and AI infrastructure in the world. NTT DATA is part of NTT Group and headquartered in Tokyo.

Equal Opportunity Employer
NTT DATA is proud to be an Equal Opportunity Employer with a global culture that embraces diversity. We are committed to providing an environment free of unfair discrimination and harassment. We do not discriminate based on age, race, colour, gender, sexual orientation, religion, nationality, disability, pregnancy, marital status, veteran status, or any other protected category. Join our growing global team and accelerate your career with us. Apply today.