Lead Site Reliability Engineer

BMC empowers nearly 80% of the Forbes Global 100 to accelerate business value, faster than humanly possible. Our industry-leading portfolio unlocks human and machine potential to drive business growth, innovation, and sustainable success. BMC does this in a simple and optimized way by connecting people, systems, and data that power the world's largest organizations so they can seize a competitive advantage.

About the Role You build, operate, and continuously improve reliable, scalable, secure, and observable AI/ML and Generative AI platforms in production - supporting the full lifecycle from deployment and monitoring through incident management, optimization, and continuous improvement. At this level you execute defined work with guidance and grow through review and mentorship. Scope at this level: Completes defined operations/observability tasks with guidance; learns AI failure modes and on-call basics. Organizational impact expected: Contributes reliable work to a project.

Key Responsibilities Core responsibilities for this role: *Build and operate production-grade infrastructure and operational frameworks for LLM, GenAI, RAG, and Agentic AI applications. *Define and defend SLOs/SLIs for agent and model behaviour - latency, availability, quality, and cost. *Monitor production AI systems for performance, drift, hallucination rates, and quality regressions; act before customers are affected. *Lead incident response for AI systems: detection, triage, mitigation, and blameless post-mortems. *Build runbooks, automation, and self-healing for common AI operational issues; participate in on-call as required. *Operate model/agent lifecycle in production: versioning, rollout, rollback, and safe promotion with evaluation gates. *Manage LLM cost, capacity, and scaling; manage model serving/inference infrastructure. *Maintain observability/tracing for agents, tools, and prompts (Langfuse, OpenTelemetry, OpenSearch). *Operate AI/ML workloads on IBM Z where required; harden the AMI Platform for reliability, scalability, and security. *Partner with AI Engineers, Data Scientists, DevOps, and Security to embed reliability from design onward.

Must-Have Skills & Experience * SRE fundamentals - SLOs/SLIs, monitoring, alerting, incident management (depth scales with level). * Linux, networking, distributed systems, troubleshooting, performance analysis. * CI/CD and DevSecOps practices; containers/orchestration (Docker, Kubernetes, OpenShift); cloud platforms. * Generative AI/LLM literacy; RAG/agents/tooling concepts as level requires. * MLOps/LLMOps tooling and AI observability as level requires. * Scripting/automation (Python, Bash); security-first, calm incident handling. * Linux/scripting fundamentals; exposure to monitoring or cloud (project/internship OK). * Curiosity about how LLM/agent systems fail in production. * Calm, careful work under guidance; readiness for eventual on-call with support. * Past experience: Contributed to monitoring, runbooks, or operational fixes within an existing workflow; basic exposure to cloud/containers. * Delivery evidence: Reliable execution of defined ops tasks; clear notes; careful changes under guidance. * Shared expectation: Completes defined tasks with guidance and checks work carefully.

Nice-to-Have Skills * Agent frameworks and AgentOps for multi-agent systems. * LLM cost-optimization and inference-serving stacks (vLLM, Ollama). * Enterprise on-prem or hybrid deployments; Agile/Atlassian.

Our commitment to you!

BMC's culture is built around its people. We have 6000+ brilliant minds working together across the globe. You won't be known just by your employee number, but for your true authentic self. BMC lets you be YOU!

If after reading the above, You're unsure if you meet the qualifications of this role but are deeply excited about BMC and this team, we still encourage you to apply! We want to attract talents from diverse backgrounds and experience to ensure we face the world together with the best ideas!

BMC is committed to equal opportunity employment regardless of race, age, sex, creed, color, religion, citizenship status, sexual orientation, gender, gender expression, gender identity, national origin, disability, marital status, pregnancy, disabled veteran or status as a protected veteran. If you need a reasonable accommodation for any part of the application and hiring process, visit the accommodation request page.

BMC Software maintains a strict policy of not requesting any form of payment in exchange for employment opportunities, upholding a fair and ethical hiring process.

The annual base salary range represents the low and high end of the BMC salary range for this position. Actual salaries depend on a wide range of factors that are considered in making compensation decisions, including but not limited to skill sets; experience and training, licensure, and certifications; and other business and organizational needs.

The range listed is just one component of BMC's employee compensation package. Other rewards may include a variable plan and country specific benefits.

At BMC, it is not typical for an individual to be hired at /near the top of the range. A reasonable estimate of the current range is $152,925 - $254,875