Staff Backend Engineer - Monitoring and Anomaly Detection

Role Description

The Observability, Monitoring, and Integrations team manages observability, monitoring, and detection for systems that support customer purchasing and use of GitLab. As a Staff Backend Engineer on the Fulfillment Workflow Monitoring (Catch All) team, you'll set the technical direction for the telemetry, detection, and reconciliation tooling that identifies billing, data, and event anomalies across CustomersDot, Salesforce, and Zuora before they can affect revenue or the customer experience.

This is a greenfield team. You'll help build it from the ground up, shaping its operating rhythm, incident response model, and quality bar. You'll use telemetry and machine learning to identify issues early and help resolve them. Because Monetization changes can directly affect revenue and often involve confidential or financial information, you'll use strong judgment, careful rollout practices, and a high bar for reliability. This role is opening at the Staff Backend Engineer level (grade 8).

What you’ll do

• Design, build, and operate observability across the Monetization stack, including metrics, logs, and traces, using tools such as Prometheus and Grafana.
• Implement automated detection for billing, data, and event anomalies, and route alerts to the responsible feature teams.
• Develop reconciliation and data integrity checks across usage and billing pipelines.
• Define and track service level objectives and service level indicators, write runbooks, and take part in incident response to measure and improve Monetization system reliability.
• Explore artificial intelligence and machine learning techniques to predict system anomalies and accelerate resolution.
• Review and offer feedback on merge requests from other Monetization engineers.
• Collaborate with Product, Finance, Support, and other partners to turn operational needs into reliable tooling.

Qualifications

• Professional experience with Ruby on Rails, including building and operating applications such as CustomersDot.
• Experience in site reliability or observability engineering, including monitoring, alerting, service level objectives and indicators, runbooks, and incident response.
• Experience setting technical direction for observability, reliability, or detection work and bringing other engineers along.
• Experience building anomaly detection, monitoring, or risk management tooling.
• Familiarity with observability tools such as Prometheus, Grafana, and OpenTelemetry.
• Exposure to analytical data stores such as ClickHouse and the change data capture and event-streaming pipelines that feed them, including Siphon and NATS JetStream.
• Experience owning projects from concept through production, communicating clearly about complex technical and organizational problems, and proposing iterative solutions.
• Working knowledge of Python for anomaly detection and data work, or experience with billing, financial, or other business-critical systems, including Zuora or Salesforce, is a plus.

About the team

The Observability, Monitoring, and Integrations team is part of GitLab's Monetization section, a highly cross-functional group that partners with Sales, Finance, IT, and Support to deliver secure and reliable purchasing, billing, provisioning, and utilization workflows. The Monetization section includes the Purchase, Subscription Lifecycle, Billing Engine, Platform and Infrastructure, Observability, Monitoring, and Integrations, and Compliance teams.

This role is on the Fulfillment Workflow Monitoring (Catch All) team and reports to the Engineering Manager, Observability, Monitoring, and Integrations. We work asynchronously, plan and reflect regularly, and share responsibility for deployments, incident management, testing, and safe rollouts. We're increasingly focused on how artificial intelligence and machine learning can improve our Monetization infrastructure and provide predictive insights.

Benefits

• Flexible Paid Time Off
• Team Member Resource Groups
• Equity Compensation & Employee Stock Purchase Plan
• Growth and Development Fund
• Parental Leave