Real-Time Inference Engineering Lead (FTE / Hybrid)

Req ID: 388176

NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now.

We are currently seeking a Real-Time Inference Engineering Lead (FTE / Hybrid) to join our team in Charlotte , North Carolina (US-NC) , United States (US) .

Job Duties and Responsibilities:

Platform Context

The Cortex Predictive AI Platform accelerates predictive AI modernization and enterprise adoption across the full model lifecycle: governed data and features; model build, training, and validation; deployment and inference; and ongoing monitoring and operations. The Real-Time Services portfolio provides standardized, scalable model-serving capabilities for applications requiring reliable, performant online inference.

Position Summary

The Real-Time Inference Engineering Lead will design, build, and industrialize low-latency, resilient model-serving services for real-time predictive AI use cases. This role provides technical leadership for online inference architecture, deployment patterns, API services, capacity controls, observability, and operational practices across public cloud and on-premises environments.

The successful candidate will establish reusable patterns that enable application, data science, and ML engineering teams to deploy and operate predictive models safely and efficiently at scale. This is a hands-on engineering role requiring strong experience with model serving, Kubernetes, APIs, performance optimization, reliability engineering, CI/CD, and production operations.

Key Responsibilities

• Define the target architecture and engineering standards for real-time predictive model-serving services across cloud and on-premises environments.
• Design, build, test, deploy, and operate scalable online inference services that meet latency, throughput, availability, resiliency, and security requirements.
• Establish reusable model-serving patterns for synchronous APIs, asynchronous inference, batch-adjacent processing, and event-driven real-time use cases where appropriate.
• Build standardized deployment approaches for predictive models, including model packaging, versioning, release promotion, canary deployment, rollback, and retirement.
• Design and implement secure API patterns for inference services, including authentication, authorization, traffic management, rate limiting, auditability, and integration with enterprise systems.
• Engineer Kubernetes-based serving platforms using GKE, OpenShift, and related container orchestration capabilities.
• Implement autoscaling, resource allocation, quota management, capacity planning, and workload-isolation controls for variable inference demand.
• Conduct performance engineering, load testing, stress testing, and failure testing to validate service behavior under expected and peak production workloads.
• Identify and implement latency-optimization opportunities across model initialization, feature retrieval, network paths, API handling, runtime configuration, and infrastructure utilization.
• Define and implement monitoring, telemetry, dashboards, alerts, SLIs, SLOs, and error-budget practices for real-time inference services.
• Partner with ML platform, data engineering, application engineering, security, and operations teams to integrate model services with governed data, feature, network, and identity capabilities.
• Implement CI/CD and automated validation for model-serving services, infrastructure configuration, APIs, performance benchmarks, and release-readiness checks.
• Build operational runbooks, incident-response procedures, support models, and production-readiness artifacts for real-time services.
• Drive reliability improvements through root-cause analysis, capacity reviews, resiliency testing, disaster-recovery planning, and continuous operational improvement.
• Mentor engineers and establish reusable technical documentation, reference implementations, and knowledge-transfer materials for real-time inference capabilities.

Required Qualifications

• 8+ years of software engineering, platform engineering, cloud engineering, SRE, or infrastructure engineering experience.
• 4+ years of experience designing, building, or operating production APIs, distributed systems, platform services, or real-time data and ML workloads.
• Demonstrated experience leading technical design and engineering delivery for highly available, performance-sensitive production services.
• Strong experience with online inference architecture, model-serving frameworks, or predictive-model deployment patterns.
• Hands-on experience designing and operating RESTful, gRPC, or event-driven APIs.
• 4+ years satrong experience with Kubernetes and container platforms in production, including GKE, OpenShift, or comparable environments.
• Experience with autoscaling, resource management, capacity planning, performance testing, and load testing for distributed services.
• Experience implementing observability, monitoring, dashboards, alerts, SLIs, SLOs, and incident-management practices.
• Experience with CI/CD, Git-based development, automated testing, deployment automation, and production-release practices.
• Strong understanding of resiliency, high availability, fault tolerance, disaster recovery, and operational support for critical services.
• Ability to work effectively with data science, ML engineering, platform engineering, application teams, security, and business stakeholders.

Required Skills / Knowledge

• Online inference and low-latency model-serving architecture.
• Model deployment, versioning, routing, rollout, rollback, and lifecycle management.
• REST APIs, gRPC, API gateways, authentication, authorization, traffic management, and API observability.
• Kubernetes, GKE, OpenShift, containers, service meshes, ingress, workload scheduling, and autoscaling.
• Performance engineering, load testing, stress testing, benchmarking, profiling, and latency optimization.
• Monitoring, telemetry, distributed tracing, dashboards, alerting, SLIs, SLOs, and error budgets.
• CI/CD, automated testing, deployment automation, infrastructure-as-code, and release controls.
• Resiliency engineering, high availability, capacity controls, incident response, root-cause analysis, and operational runbooks.
• Cloud and on-premises platform operations, networking, identity, data protection, and secure production delivery.

Preferred Qualifications

• Experience with Vertex AI endpoints, KServe, Seldon, NVIDIA Triton Inference Server, MLflow deployments, or comparable model-serving technologies.
• Experience deploying and operating models on GCP, Azure, AWS, private cloud, or hybrid-cloud environments.
• Experience with service mesh, API gateway, traffic-routing, or edge-serving technologies.
• Experience serving high-volume, customer-facing, fraud, risk, personalization, decisioning, or other latency-sensitive predictive models.
• Experience with feature-serving, online feature stores, caching, streaming platforms, or real-time data enrichment.
• Experience with Terraform, Helm, Argo CD, Jenkins, GitHub Actions, GitLab CI, or similar automation tooling.
• Experience in banking, financial services, healthcare, insurance, or another regulated enterprise environment.
• Experience participating in a 24x7 operational support model for high-priority production services.

Expected Outcomes

• Standardized, production-ready real-time inference architecture and reusable model-serving patterns.
• Reliable online inference services that meet defined latency, throughput, availability, and resiliency objectives.
• Automated deployment, testing, monitoring, capacity-management, and rollback capabilities for predictive models.
• Clear operational dashboards, SLOs, alerts, runbooks, and readiness evidence for real-time services.
• Improved engineering productivity and faster adoption of secure, scalable real-time predictive AI capabilities across the Cortex portfolio.

#LI-NorthAmerica

NTT DATA provides a reasonable range of compensation for U.S.-based positions. The starting pay range for this role is $83,520.00 - $125,280.00. Actual compensation will depend on a number of factors, including the candidate’s relevant experience, technical skills, and other qualifications.

This position may also be eligible for incentive compensation based on individual and/or company performance. 

This position is eligible for company benefits including medical, dental, and vision insurance with an employer contribution, flexible spending or health savings account, life and AD&D insurance, short and long term disability coverage, paid time off, employee assistance, participation in a 401k program with company match, and additional voluntary or legally-required benefits.

About NTT DATA

NTT DATA is a $30 billion business and technology services leader, serving 75% of the Fortune Global 100. We are committed to accelerating client success and positively impacting society through responsible innovation. We are one of the world's leading AI and digital infrastructure providers, with unmatched capabilities in enterprise-scale AI, cloud, security, connectivity, data centers and application services. our consulting and Industry solutions help organizations and society move confidently and sustainably into the digital future. As a Global Top Employer, we have experts in more than 50 countries. We also offer clients access to a robust ecosystem of innovation centers as well as established and start-up partners. NTT DATA is a part of NTT Group, which invests over $3 billion each year in R&D.

Whenever possible, we hire locally to NTT DATA offices or client sites. This ensures we can provide timely and effective support tailored to each client’s needs. While many positions offer remote or hybrid work options, these arrangements are subject to change based on client requirements. For employees near an NTT DATA office or client site, in-office attendance may be required for meetings or events, depending on business needs. At NTT DATA, we are committed to staying flexible and meeting the evolving needs of both our clients and employees. NTT DATA recruiters will never ask for payment or banking information and will only use @nttdata.com, @nttdatafed.com and @talent.nttdataservices.com email addresses. If you are requested to provide payment or disclose banking information, please submit a contact us form,

NTT DATA endeavors to make accessible to any and all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please contact us at .  This contact information is for accommodation requests only and cannot be used to inquire about the status of applications. NTT DATA is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. For our EEO Policy Statement, please click here . If you'd like more information on your EEO rights under the law, please click here . For Pay Transparency information, please click here .