Sr. Software Engineer

Senior Engineer, AI Inference

At Spectro Cloud, we are looking for a senior engineer to own AI inference at scale. Our customers run models on infrastructure they control, from data centers to edge sites. You will build the systems that serve those models in production and keep them fast, reliable, and efficient. You will bring new models into service, improve how they perform under real demand, and set the standard for how we test and release them. Your work will shape how our customers use AI in production.

We are looking for an engineer who enjoys difficult problems, works well across teams, and can turn a deep technical issue into a clear plan. Experience with Kubernetes is highly desired.

Qualities

As a software engineer at Spectro Cloud, you will succeed by embracing adaptability, tackling complex challenges, and navigating dynamic environments with confidence. You can break down intricate problems into manageable steps, deliver value iteratively, and explain your work clearly to the people who depend on it. You take ownership of outcomes and see work through to production. At Spectro Cloud, we champion innovation and collaboration, fostering a culture where shared ideas drive progress.

Key Responsibilities

• Build and operate AI inference services in production, with engines such as vLLM, SGLang, or TensorRT-LLM
• Bring new models into service, from first test to full rollout
• Improve throughput, response time, and memory use across the stack
• Find the root cause of performance and reliability issues, then fix them
• Design benchmarks and release criteria for every model we ship
• Set reliability targets and meet them in production
• Scale inference across many GPUs and nodes
• Run inference on modern GPU platforms, including the latest accelerators
• Automate deployment and operations on Kubernetes
• Write runbooks, tuning guides, and clear technical documentation

Qualifications

While we don't expect candidates to check every box, we do look for the following when assessing applicants:

• Hands-on experience running AI or LLM inference in production, with an engine such as vLLM or TensorRT-LLM
• A working understanding of inference engines, including batching, caching, and memory management
• Experience improving the performance, reliability, and efficiency of production systems
• Experience with distributed systems and multi-GPU workloads
• Experience designing benchmarks and release criteria
• Bachelor's degree in Computer Science or a related field
• 4+ years of software development experience, or 2+ years with a Master's degree
• Strong Python and hands-on PyTorch
• Golang for backend services and APIs
• Working knowledge of Kubernetes
• Experience with modern GPU platforms, drivers, and monitoring tools
• Understanding of high-speed networking and interconnects
• Experience with cloud-native applications on AWS, Azure, or GCP
• Comfort in Linux environments and with common command line tools
• Experience with automated testing
• Familiarity with AI-assisted development tools
• Clear written communication in English
• Experience with model optimization, quantization, and acceleration
• GPU scheduling in Kubernetes
• Experience with monitoring and observability tools
• Kubernetes certification such as CKA or CKAD

The Hiring Process

At Spectro Cloud, we highly value your time and dedication throughout the hiring process. Therefore, we aim to ensure that our interactions are efficient and focused on delivering value.

Our interview process for Engineering roles typically comprises three to four stages:

• Initial screening interview
• One or two technical interviews, including hands-on coding assessments
• Final round to assess team fit and engage in detailed discussions

The majority of our interviews are conducted via Zoom. We recommend having a laptop with a reliable internet connection, and if possible, a functional camera for a more engaging experience.

Become a Spectronaut and join us in shaping the future!

2026-113