Principal Site Reliability Engineer

At Early Warning, we've powered and protected the U.S. financial system for over thirty years with cutting-edge solutions like Zelle®, Paze®, and so much more. As a trusted name in payments, we partner with thousands of institutions to increase access to financial services and protect transactions for hundreds of millions of consumers and small businesses.

Early Warning follows a hybrid work model to allow for a more collaborative working environment.

Candidates responding to this posting must independently possess the eligibility to work in the United States, for any employer, at the date of hire. This position is ineligible for employment Visa sponsorship.

Overall Purpose


The Principal Site Reliability Engineer partners with development teams by designing availability and resiliency patterns in applications and infrastructure.


Essential Functions:

• Design and Implement software and tools to improve the performance - availability, scalability, and latency, while delivering end products to customer with the highest efficiency and meeting all security standards.

• Supports the company's commitment to risk management and protecting the integrity and confidentiality of systems and data.

• Build automation and tooling around application management, such as deployments, configuration changes and disaster recovery scenarios.

• Design, Implement and evangelize Observability and monitoring systems to proactively detect problems and identify cause.

• Evaluate capacity of the application on a continuous basis to provide stats to the Product/Business teams and recommend an efficient path to scale for future needs.

• Identify performance bottlenecks and work with cross-functional teams to troubleshoot and resolve issues.

• Serve as a technical liaison for the application and provide documents and runbooks to Level 1 and Level 2 teams.

• Participate in 24 X 7 on-call rotation.

• Be a champion of excellent processes; take the initiative in developing repeatable patterns and standard, re-usable work across teams.

• Work directly with application development teams to provide feedback and technical requirements to the software development lifecycle, implementing best-practice microservice design patterns and other modern software development approaches.

• Understand and support the adoption of best-practice microservice design patterns and other modern software reliability approaches and techniques.

• Be a thought leader: a senior point of expertise on site reliability engineering issues, industry trends and developing technologies. Be a role model to others on the team. Coach and mentor team members.

Minimum Qualifications

• Education and experience typically obtained through completion of a Bachelor's Degree in Business and/or Computer Science or related field.

• 12+ years of related experience managing large complex projects in a technical or software development environment inclusive of post-graduate degree

• Proven ability to lead a team through high priority Incidents and improve the RCA proces

• Excellent troubleshooting skills and proven experience resolving technical issues in complex environments

• Hands-on experience in designing and developing using the one or more of the following technologies
  - Python, Go, Java
  - Docker
  - Experience in Microservices Architecture.
  - Messaging frameworks such as Kafka, SQS or JMS
  - Database Technologies like Oracle, Dynamo DB, Aurora etc..
  - Caching layers such as Redis and memcached

• Strong understanding of Linux administration

• Experience with CI/CD pipeline implementation including GIT, Chef, Maven, Jenkins etc

• Strong understanding of networking fundamentals

• Experience in leading cross-functional teams to create technical solutions.

• Proven track record designing and building complex end-to-end systems (full stack developer)

• Background and drug screen

Preferred Qualifications

• Good programming skills in one or more of the following languages: Java, ruby, python, JavaScript and GO

• Hands-on experience in supporting applications in a 24X7 customer-facing production environment.

• Working knowledge of AWS, Docker, Kubernetes, Swarm

Physical Requirements


Early Warning works together in a highly collaborative office environment. Working conditions consist of a normal office environment. Work is primarily sedentary and requires extensive use of a computer and involves sitting for periods of approximately four hours. Work may require occasional standing, walking, kneeling, and reaching. Must be able to lift 10 pounds occasionally and/or negligible amount of force frequently. Requires visual acuity and dexterity to view, prepare, and manipulate documents and office equipment including personal computers. Requires the ability to communicate with internal and/or external customers.


Employee must be able to perform essential functions and physical requirements of position with or without reasonable accommodation.


Candidates responding to this posting must independently possess the eligibility to work in the United States at the date of hire.


The base pay scale for this position in:
Phoenix, AZ in USD per year is: $194,000 - $237,000.

Additionally, candidates are eligible for a discretionary incentive plan and benefits.

This pay scale is subject to change and is not necessarily reflective of actual compensation that may be earned, nor a promise of any specific pay for any specific candidate, which is always dependent on legitimate factors considered at the time of job offer. Early Warning Services takes into consideration a variety of factors when determining a competitive salary offer, including, but not limited to, the job scope, market rates and geographic location of a position, candidate's education, experience, training, and specialized skills or certification(s) in relation to the job requirements and compared with internal equity (peers). The business actively supports and reviews wage equity to ensure that pay decisions are not based on gender, race, national origin, or any other protected classes.

Some of the Ways We Prioritize Your Health and Happiness

• Healthcare Coverage - Competitive medical (PPO/HDHP), dental, and vision plans as well as company contributions to your Health Savings Account (HSA) or pre-tax savings through flexible spending accounts (FSA) for commuting, health & dependent care expenses.
• 401(k) Retirement Plan - Featuring a 100% Company Safe Harbor Match on your first 6% deferral immediately upon eligibility.
• Paid Time Off - Flexible Time Off for Exempt (salaried) employees, as well as generous PTO for Non-Exempt (hourly) employees, plus 11 paid company holidays and a paid volunteer day.
• 12 weeks of Paid Parental Leave
• Maven Family Planning - provides support through your Parenting journey including egg freezing, fertility, adoption, surrogacy, pregnancy, postpartum, early pediatrics, and returning to work.

And SO much more! We continue to enhance our program, so be sure to check our Benefits page here for the latest. Our team can share more during the interview process!

Early Warning Services, LLC ("Early Warning") considers for employment, hires, retains and promotes qualified candidates on the basis of ability, potential, and valid qualifications without regard to race, religious creed, religion, color, sex, sexual orientation, genetic information, gender, gender identity, gender expression, age, national origin, ancestry, citizenship, protected veteran or disability status or any factor prohibited by law, and as such affirms in policy and practice to support and promote equal employment opportunity and affirmative action, in accordance with all applicable federal, state, and municipal laws. The company also prohibits discrimination on other bases such as medical condition, marital status or any other factor that is irrelevant to the performance of our employees.


Early Warning Services LLC is a proud participant in E-Verify, a federal program to help ensure a legal and authorized workforce. As part of our hiring process, we electronically verify the employment eligibility of all new hires through E-Verify. For more information on your rights and responsibilities under E-Verify please visit Home | E-Verify.