Lead Site Reliability Engineer

Job Description

Innovation isn't just a talking point at GM Financial, it's how we operate. From generative AI and cloud-native technologies to peer-led learning and hackathons, our tech teams are building real solutions that make a difference. We're committed to AI-powered transformation, using advanced machine learning and automation to help us reimagine customer interactions and modernize operations, positioning GM Financial as a leader in digital innovation within a dynamic industry. Join us and discover a workplace where your ideas matter, your development is prioritized, and you can truly make a global impact.

Responsibilities

About the Role:

We are expanding our efforts into complementary data technologies for decision support in areas of ingesting and processing large data sets. Our interests are in enabling data science and search based applications on large and low latent data sets in both a batch and streaming context for processing. To that end, this role will incorporates aspects of software engineering and operations, combining SRE and DevOps skills to come up with efficient ways of managing and operating applications. The role will require a high level of responsibility and accountability to deliver technical solutions. The data sets we deal with support both off-line and in-line machine learning training and model execution. Other data sets support search engine based analytics. Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, selecting data solutions software, and defining hardware requirements based on business requirements. Responsibility also includes documentation of procedures for deployment, monitoring, managing and switching the environments in production and disaster recovery sites. This role participates along with team counterparts to architect an end-to-end framework developed on a group of core data technologies.

• Manage/Administer/Deploy Kubernetes and Spark cluster environments, on bare-metal and container infrastructure, including service allocation and configuration for the cluster, capacity planning, performance tuning, and ongoing monitoring.
• Define and refine processes and procedures for the site reliability engineering practice.
• Setup, manage and maintain Kubernetes based scalable environments for high-availability and work with vendors for smooth and continuous operations.
• Work closely with data scientists, data architects, data engineers, ETL developers, cybersecurity, network, Linux, other IT counterparts, and business partners to design and setup the environments to manage the ingested and processed datasets from the external sources, internal systems, and the data warehouse to extract features of interest.
• Evaluate, research, experiment with data processing, management and scalability technologies in a lab to keep pace with industry innovation while assessing business impact and viability for use cases associated with efforts in hand.
• Design, setup, test, deploy, monitor, document, and troubleshoot data processing and associated automation issues from the operations perspective.
• Work with IT Operations and Information Security Operations with monitoring and troubleshooting of incidents to maintain service levels.
• Work with Information Security Vulnerability Management and vendors to remediate known impacting vulnerabilities.
• Contribute to the evolving distributed systems architecture to meet changing requirements for scaling, reliability, performance, manageability, and cost.
• Report utilization and performance metrics to user communities.
• Contribute to planning and implementation of new/upgraded hardware and software releases.
• Responsible for monitoring the Linux, Kubernetes, Object Storage(MinIO), Feature Store, and Spark.
• Research and recommend innovative, and where possible, automated approaches for administration tasks.
• Identify approaches to efficiencies in resource utilization, provide economies of scale, and simplify support issues.
• Responsible for administration of Machine Learning platforms & Operations (MLOps) Such as Kubeflow/Jupyterhub/Python.
• This role will support GMF international operations and will closely align with our GMF IT NorthStar architecture and operating Principles.

Qualifications

What Makes You an Ideal Candidate?

• Excellent knowledge of Kubernetes Administration, Deployments & Upgrades.
• Excellent Knowledge on Apache Spark administration on various platforms.
• Strong working knowledge of Object Store(MinIO) and Spark cluster security, networking connectivity and IO throughput along with other factors that affect distributed system performance.
• Strong working knowledge of disaster recovery, incident management, and security best practices.
• Working knowledge of containers (e.g., docker) and major orchestrators (e.g., Mesos, Kubernetes, Docker Datacenter).
• Working knowledge of software defined networking.
• Working knowledge of hardening Data at Rest with key based encryption technologies.
• Working knowledge of setting up and customize interactive data analytics tools (e.g., Apache Zeppelin, Jupyter notebooks).
• Excellent knowledge on building the docker images to provide Containers-as-a-service.
• Working knowledge on Azure Administration, Azure DevOps & Azure Kubernetes Service (AKS).
• Working knowledge of Pipeline Automation: Azure DevOps (YAML, ARM), Terraform, Jenkins, Chef/Puppet, Ansible.
• Working knowledge of CICD methodologies like Artifactory/Git/Gitops/Jenkins.
• Working knowledge of Code Scanning tools: SonarQube, Checkmarx/Blackduck/Twistlock.
• Working knowledge of Object Storage like S3/MinIO, Bucket policies and administration.
• Working knowledge of Kubernetes Storage protocols.
• Experienced with networking infrastructure including VLAN and firewalls.
• Working knowledge of hardening Kubernetes clusters with network policies like Calico/Tigera, service meshes like Istio, Internal & external load balancers.
• Proven track record with Red Hat Enterprise Linux & Kubernetes administration.
• Proficiency in a high-level language like Python, Go, Ruby and/or Java.
• Solid experience in High Availability and distributed systems, Linux, Data and SAN Storage Networks, NAS and Networking, leveraging tools to instrument and automate proactively and eventually predictive availability solutions.
• Proven track record leading complex enterprise production support efforts adhering to a mix of DevOps & SRE frameworks.
• Experience transitioning platforms to the cloud, with knowledge of cloud frameworks & design patterns, micro-service architectures
• Extensive Knowledge of networking, including DNS, DHCP, firewalls, load balancers and IP routing.
• Experience in Monitoring tools - Splunk, Zenoss, Elastic, Appdynamics, Dynatrace, Grafana, Promotheus, Kiali etc.
• Ability to grasp difficult concepts, large architectures, and sophisticated designs quickly and troubleshoot with debugging skills across a variety of integrated platforms.
• Proven capability to provide operational visibility on environment health to Senior Leadership, Technology and Business partners.
• Receptive, approachable teammate, with the ability to positively interact with business partners, technology teams, offshore, and professional services.
• Strong customer advocate with excellent written and verbal communication skills.

Additional Knowledge and Skills

Working effectively within an AI enabled environment:

• Ability to use AI tools (e.g., Microsoft Copilot) to support daily work.
• Skills in evaluating AI outputs for accuracy, compliance, and bias.
• Experience integrating AI into workflows to improve efficiency or insights.
• Familiarity with AI assisted research, summarization, and content generation.
• Understanding of responsible AI use, including ethics and data protection.

Education and Experience:

• 5-7 years of hands-on experience with supporting Linux production environments required.
• 5-7 years of hands-on administration experience on Spark required.
• 3-5 years hands-on experience with scripting with bash, perl, ruby, or python required.
• 3-5 years experience with Docker Datacenter required.
• 2-4 years of hands-on administration experience on Machine learning platforms required.
• Minimum of 1 year of experience in Mesos, Kubernetes, OpenShift and/or Deis or other such container/platform-as-a-service orchestrator required.
• Minimum of 1 year of hands-on experience on CICD tools & Technologies required.
• Minimum of 1 year of lead experience of site reliability engineering team required.
• Hands-on experience in cloud technologies with Microsoft Azure required.
• High School Diploma or equivalent required.
• Bachelor's Degree in related field or equivalent experience required.
• Master's Degree preferred.

What We