Senior Site Reliability Engineer

About Us
Visa is a world leader in payments technology, facilitating transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories, dedicated to uplifting everyone, everywhere by being the best way to pay and be paid.

At Visa, you'll have the opportunity to create impact at scale - tackling meaningful challenges, growing your skills and seeing your contributions impact lives around the world.

Join Visa and do work that matters - to you, to your community, and to the world. Progress starts with you.

Job Description

We are looking for a talented Sr. Site Reliability Engineer (SRE) to join our Observability team. In this role, you will work alongside other engineers to help build, maintain, and improve reliable, secure, and scalable observability solutions for our global infrastructure.

As an Sr. SRE, you will help bridge the gap between development and operations, ensuring our systems and services are robust and well-monitored. You will assist in automating workflows, deploying and maintaining monitoring tools, and supporting the smooth operation of our technology platforms.

You will participate in troubleshooting issues, responding to incidents, and implementing fixes under the guidance of senior team members. You'll also contribute to building and enhancing automation, documentation, and reliability practices

Responsibilities

• Design, deploy, and enhance monitoring and logging instrumentation to ensure comprehensive observability.
• Develop and maintain automation to streamline operational workflows and improve system integration.
• Operate, support, and optimize observability platforms and tools, including Splunk, ClickHouse, Grafana, Prometheus, M3DB, OpenTelemetry, Fluent Bit, ElasticSearch, OpenSearch, and CloudWatch.
• Collaborate with development teams to architect monitoring solutions and assist in building development, staging, and production environments.
• Safeguard systems by proactively applying security hotfixes and OS patches to mitigate cybersecurity threats.
• Champion DevOps best practices across all platforms and teams.
• Design, implement, and maintain CI CD pipelines to automate build, test, and deployment processes.
• Manage and optimize cloud infrastructure (AWS, GCP) to ensure availability, scalability, performance, and security.
• Leverage infrastructure-as-code tools such as Terraform, Ansible, or CloudFormation to automate infrastructure provisioning and management.
• Monitor system health, diagnose issues, and deliver solutions to enhance reliability and efficiency.
• Implement and manage containerization and orchestration platforms, such as Docker and Kubernetes, for streamlined application deployment.
• Conduct root cause analysis for production incidents and implement preventative solutions.
• Maintain clear and up-to-date documentation for infrastructure, procedures, and processes.
• Provide technical support and troubleshooting for infrastructure and deployment-related issues.
• Continuously develop automation to boost operational efficiency and improve system integration.

This is a hybrid position. Expectation of days in office will be confirmed by your Hiring Manager.

Visa requires at least 3 days in office, expectations of these days will be confirmed by your Hiring Manager.

Qualifications

Basic Qualifications
• Bachelor's degree with 3+ years of relevant professional experience, OR Advanced degree (e.g., Master's, MBA, JD, MD) with at least 2 years of relevant experience, OR 6+ years of relevant industry experience

Preferred Qualifications
• Hands-on experience performing upgrades and maintenance for observability tools such as Splunk Enterprise and Universal Forwarder, Fluent Bit, Prometheus container images, Grafana, ElasticSearch, OpenSearch, Kibana, and ClickHouse.
• Experience configuring and deploying exporters (e.g., Node Exporter, Cert Exporter) for metrics collection.
• Proficient with containerization and orchestration platforms, including Docker and Kubernetes.
• Strong background working with observability solutions such as Splunk, ClickHouse, Grafana, Prometheus, M3DB, OpenTelemetry, Fluent Bit, ElasticSearch, OpenSearch, and CloudWatch.
• Experience managing CI CD pipelines using tools like GitHub and Ansible.
• Familiarity with Infrastructure as Code tools (e.g., Terraform) and configuration management approaches such as GitOps.
• Scripting abilities in Python and or Shell.
• Working knowledge of query languages such as PromQL, MS SQL, or Splunk SPL is a plus.
• Solid experience in Linux environments and Unix shell scripting.
• GCP or AWS cloud certifications are a plus.
• Strong analytical skills with the ability to assess problems and solutions at the appropriate level of detail to engage with various stakeholders.
• Excellent communication and leadership abilities.

Justification
Visa's Observability ecosystem includes over 2,000 platform nodes, utilizing approximately 15 different tools for logging, monitoring, and tracing, alongside 80,000 client agents. The system handles daily log ingestion exceeding 100TB and oversees hundreds of critical applications, supporting vital alerts, dashboards, and reports. To maintain this high level of performance and reliability, we need a Sr. Site Reliability Engineer (SRE) with comprehensive knowledge and practical experience. This position requires an I5-level engineer who can operate independently with minimal supervision.

About Visa's PRE Observability Team
Visa's Product Reliability Engineering (PRE) Observability team partners with Product Development as well as Operations & Infrastructure teams to build and manage innovative, reliable, scalable, secure, and cost-effective observability platform solutions. We are looking for talented Senior Site Reliability Engineers to join our driven team, with a focus on maximizing system availability, performance, security, and reliability. This dynamic role requires technical leadership, strong problem-solving skills, and expertise in coding, testing, and debugging.Information for US Applicants :For roles located in the US, the estimated salary range for this position is $110,700.00 to $ 171,800.00 USD per year, which may include potential sales incentive payments (if applicable). Salary may vary depending on job-related factors which may include knowledge, skills, experience, and location. In addition, this position may be eligible for bonus and equity.Visa has a comprehensive benefits package for which this position may be eligible that includes Medical, Dental, Vision, 401(k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.

Work Hours

Varies upon the needs of the department.

Travel Requirements

This position requires travel 5-10% of the time.

Mental/Physical Requirements

This position will be performed in an office setting. The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.

Visa is an EEO Employer
Qualified applicants will receive consideration for employment without regard to race, color religion, sex, national origin, sexual orientation, gender identity, disability or protect veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with the EEOC guidelines and applicable local law. Visa_Talent_Portal