Principal Software Engineer - AI ADC

Senior Staff Engineer - AI ADCRequired Skills & Experience: Advanced degree in Computer Science, Networking, or related field (MS/PhD preferred). Deep knowledge of L4–L7 protocols (TCP/TLS/ QUIC), load balancing algorithms and GSLB strategies (DNS-based, redirect, anycast). Strong programming skills in C/C++ (data plane), Go/Python (control/ML), and performance profiling (perf, eBPF, flamegraphs, VTune/nvprof). Hands-on experience with AI/ML systems, including: -Model inference pipelines, LLM API integrations -Token-level behavior and performance characteristics -Understanding of AI workload routing challenges (latency, caching, batching, multi-model orchestration)Job Description: A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations.This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference-aware load balancing, and Layer-8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution—working closely with product management, platform teams, and AI strategy groups.Key Responsibilities Include:Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.Design and implement LLM-aware load balancing, incorporating semantic, token, latency, and model-level insights into traffic decisions.Develop enhancements for ADC and GSLB platforms to support AI workloads, including:Token-aware rate limitingInference latency-based routingAI model endpoint discovery and health checksIntegration with vector databases, model registries, and ML observability systemsContribute to advanced features including Layer-8 context-aware routing, adaptive traffic shaping, and AI-driven anomaly detection.Drive system-wide architecture: data plane, control plane, configuration, and distributed state management.This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $200,000-$220,000 base salary. Insight Global