Sr Software Engineer, Infrastructure

Senior Software Engineer, Infrastructure (L5)

GAQ127R160

Location: San Francisco, CA (Hybrid)

Team: IT Infrastructure and Operations

About the Role

At Databricks Information Technology, we are a product-led organization transforming the way we work, from how easy it is to use our IT services to the applications we develop that help us scale seamlessly in the face of incredible growth.

As a Senior Software Engineer (Infrastructure) , you will be a core technical contributor on the IT Infrastructure team, owning and driving the evolution of our core infrastructure and observability platforms. This role requires a strong software engineering mindset, deep technical breadth across SRE and infrastructure worlds, and the ability to deliver high-quality, scalable solutions for currently "immature" system problems. You will be responsible for building resilient, scalable, and automated infrastructure that empowers our development teams. As a senior member of the team, you will bridge the gap between software engineering and systems architecture, ensuring our AWS environment is cost-optimized, secure, and highly available.

The Impact You Will Have

• Own and Evolve Observability Pipelines: Serve as the subject matter expert driving the design and implementation of automated telemetry workflows that integrate into infrastructure pipelines.


• Architect Bidirectional Data Flow: Design and deliver end-to-end software solutions to ensure data flows seamlessly into systems like Datadog and eventually "dogfooded" back into the Databricks platform.


• Build for Scale and Predictability: Develop data hooks for various SaaS services to pick up logs and metrics that are not currently collectible, pushing them into spaces where they are needed for long-term analysis.


• Drive "Observable by Default" Frameworks: Create underlying infrastructure to ensure new internal applications are secure and have logging and metrics enabled by default.


• Partner Cross-Functionally: Collaborate with stakeholders across Security, Engineering, Infrastructure, and Support to deliver impactful projects with real business outcomes.


• Mentor and Document: Document solutions and failure triage playbooks, and mentor junior engineers on the platforms you own.

What We Look For

• Software Engineering Expertise: 5+ years of production-level experience with a strong proficiency in Python (non-negotiable).


• IaC: Expert-level proficiency in Terraform (modules, state management) or Pulumi(Preferred) .


• Cloud & Infrastructure Breadth: Hands-on experience with AWS (or Azure/GCP), Kubernetes , Docker and containerization concepts.


• Automation & Integration Mindset: Experience building and troubleshooting integrations between infrastructure, data pipelines, and observability platforms.


• CI/CD: Advanced knowledge of Github Actions, Github Runners.


• Strong Observability Mindset: Understanding of observability pillars (logging, metrics, tracing) and hands-on experience with tools like Datadog, Prometheus, or ELK .


• Distributed Systems: Proficiency in running systems through concepts like Kafka or messaging queues.


• Independent Execution: Ability to operate with minimal guidance, take ownership of ambiguous projects, and follow a vision set by tech leads to execute independently.