Senior Inference Runtime Engineer

Bitdeer is a world-leading technology company for Bitcoin mining and AI cloud.

We are seeking a Senior Inference Runtime Engineer to own the performance-critical serving layer of the MaaS platform. This role focuses on making self-hosted LLMs faster, cheaper, and more stable by optimizing the runtime stack behind OpenAI- and Anthropic-compatible APIs.

• Optimize prefill/decode scheduling, continuous batching, KV cache behavior, speculative decoding, long-context serving, and streaming smoothness.
• Tune and operate vLLM/Dynamo/SGLang/TensorRT-LLM-style runtimes for model-specific latency, throughput, GPU utilization, and cost efficiency.
• Profile bottlenecks across GPU memory, HBM bandwidth, NCCL/network, tokenizer, frontend/proxy, and model worker paths.
• Lead high-value model onboarding, including runtime selection, tensor/pipeline parallelism, quantization, context length, and rollback strategy.
• Define runtime playbooks and safe defaults for reasoning, tool calling, multimodal, prompt cache, and provider-specific parameters.
• Partner with SRE and performance/evaluation engineers to turn benchmark findings into production runtime improvements.

• 6+ years of systems, ML infrastructure, or high-performance backend engineering experience.
• Hands-on experience with LLM serving runtimes such as vLLM, Dynamo, SGLang, TensorRT-LLM, TGI, or Triton.
• Strong understanding of GPU memory, CUDA/NCCL basics, KV cache, batching, streaming, and distributed inference tradeoffs.
• Proficient in Go or Python and comfortable reading runtime source code, profiling traces, and production metrics.
• Experience operating production inference services with strict latency, availability, and cost targets.
• Able to translate low-level performance work into customer-visible reliability, latency, and margin improvements.

• A culture that values authenticity and diversity of thoughts and backgrounds;
• An inclusive and respectable environment with open workspaces and exciting start-up spirit;
• Fast-growing company with the chance to network with industrial pioneers and enthusiasts;
• Ability to contribute directly and make an impact on the future of the digital asset industry;
• Involvement in new projects, developing processes/systems;
• Personal accountability, autonomy, fast growth, and learning opportunities;
• Attractive welfare benefits and developmental opportunities such as training and mentoring.

--------------------------------------------------------------------

#LI-ST1