Senior Kubernetes Platforms System Engineer (Remote Eligible)

Requisition Id 17120 Annual Salary Range:$$160,000.00 -$$175,000.00 Work-Site Type: Remote ORNL offers a flexible work environment that supports both the organization and the employee. In addition, this position is considered remote-eligible for ORNL in pre-approved locations. Remote-eligibility is not a continued guarantee and could be subject to change based on evolving organizational needs.Note about pay range:Please note that the pay range information is a general guideline only. Many factors are taken into consideration when setting starting pay. Salary for this position will be commensurate with the final candidate’s qualifications and experience, including skills, knowledge, relevant education, certifications, plus also aligned with the internal peer group. It is not typical for an individual to be offered a salary at or near the top of the range for a position. Link to benefits. Overview:The National Center for Computational Sciences (NCCS) at Oak Ridge National Lab (ORNL), which hosts several of the world’s most powerful computer systems, is seeking highly qualified individuals to play a key role in improving the security, performance, and reliability of the NCCS computing infrastructure which supports multiple highly ranked Top500 Supercomputers, including the world’s first exaflop system, Frontier.The Team:As a Senior Kubernetes Platform Engineer for the HPC Platform teams, you will work to support all activities of our supercomputer center. Our primary platform is the OLCF Slate Service, built on Kubernetes and RKE2, which provides a container orchestration service for running critical operation applications and user-managed persistent applications that run alongside our OLCF Supercomputer systems and other OLCF managed HPC clusters.The Role:As a Senior Kubernetes Platforms System Engineer, you will operate, implement, and maintain the infrastructure underpinning our on-premises Kubernetes clusters, with a strong focus on scalability, reliability, and maintainability. You will assist in leading the technical direction of our platform engineering initiatives, evaluate and integrate key technologies, provide daily task management for team members on large and medium sized projects, and mentor a team of engineers to deliver a robust internal platform that powers development across the organization.Major Duties/Responsibilities:Work with the team to define and implement best practices and standards within the organizationKeeping the Kubernetes platform reliable, available, and fastArchitecting solutions to problems that improve the reliability, scalability, performance, and efficiency of our servicesRespond to, investigate, and fix service issues all the way from bare metal through the OS to the application codeCoordinate with vendors to resolve hardware and software problemsParticipate in an on-call rotation providing 24-hour, 7-day support and off-hours maintenance windowsWork with users to help them use KubernetesBasic Qualifications:BS degree in a scientific field and a minimum of 8 years of relevant experience. An equivalent combination of education and experience will be considered.Experience with Kubernetes as a cluster administrator for on-premises deploymentsExcellent interpersonal/communications skills, and the ability to work as part of a teamStrong working knowledge of Linux systems fundamentals and networked computing environment conceptsExperience with code reviews, code quality, CI/CD tooling, GitOps, SCM (e.g. GitLab)Ability to identify requirements and to define, plan, and implement requisite solutions for small and medium projectsAbility to develop and maintain programs and scripts that aid in the operation and automation of tasks using various shell and scripting languages (primarily bash, Python, and Go)Experience with on-call rotationPreferred Qualifications:Subject matter expert in Kubernetes as a cluster administrator for bare metal, on-premises deploymentsExcellent interpersonal/communications skills, be able to effectively communicate with other teams and organizational leadership. Convey technical details to a non-or semi-technical audience.Ability to identify requirements and to define, plan, and implement requisite solutions for large, organizationally impactful projects.Self-driven with the ability to work in a dynamic, loosely structured research & development environment.Experience with RKE2 (nice to haves: Red Hat OpenShift and Talos). Multi-cluster management tools for Kubernetes (e.g. Fleet), and container security tools (Neuvector, SCC, pod admission control)Experiencing with managing image registries such as Quay or HarborExperience using tools such as Prometheus, Nagios, and Grafana to monitor systems, metrics and create dashboardsExperience designing and implementing highly-available systems/servicesExperience with Infrastructure-as-Code tooling such as Terraform, Helm, and PuppetExperience implementing systems-level security technologies (e.g. SELinux, Seccomp, linux capabilities), experience with DevSecOps, and general security best practices.Experience with AIOps and MLOps tooling – e.g. KServe, Kubeflow, vLLM, NVidia Enterprise AI, AMD Silo AI, ClearML, MLFlowExperience using HPC hardware for Kubernetes – e.g. RDMA, DPUs, Infiniband, many-core CPUsExperience with declarative CI/CD tools such as ArgoCDExperience with workflow engines such as Apache Airflow or Argo WorkflowsExperience with infrastructure automationCloud engineering experience with at least one cloud service providerExperience with reusable, automated workflows such as PagerDuty playbooksSpecial Requirements:Visa sponsorship is not available for this position.This position requires the ability to obtain and maintain a clearance from the Department of Energy. As such, this position is a Workplace Substance Abuse (WSAP) testing designated position. WSAP positions require passing a pre-placement drug test and participation in an ongoing random drug testing program.This position is located in Oak Ridge, Tennessee and requires onsite presence. We offer a flexible work environment that supports both the organization and the employee. A hybrid/onsite working arrangement may be available with this position, which provides flexibility to work periodically from your home, while reporting onsite to the Oak Ridge, Tennessee location on a weekly and regular basis.About ORNL:As a U.S. Department of Energy (DOE) Office of Science national laboratory, ORNL has an impressive 80-year legacy of addressing the nation’s most pressing challenges. Our team is made up of over 7,000 dedicated and innovative individuals! Our goal is to create an environment where a variety of perspectives and backgrounds are valued, ensuring ORNL is known as a top choice for employment. These principles are essential for supporting our broader mission to drive scientific breakthroughs and translate them into solutions for energy, environmental, and security challenges facing the nation.ORNL offers competitive pay and benefits programs to attract and retain individuals who demonstrate exceptional work behaviors. The laboratory provides a range of employee benefits, including medical and retirement plans and flexible work hours, to support the well-being of you and your family. Employee amenities such as on-site fitness, banking, and cafeteria facilities are also available for added convenience.Please note that the pay range information is a general guideline only. Many factors are taken into consideration when setting starting pay. Salary for this position will be commensurate with the final candidate’s qualifications and experience, including skills, knowledge, relevant education, certifications, plus also aligned with the internal peer group. It is not typical for an individual to be offered a salary at or near the top of the range for a position. Link to benefits. Other benefits include the following: Prescription Drug Plan, Dental Plan, Vision Plan, 401(k) Retirement Plan, Contributory Pension Plan, Life Insurance, Disability Benefits, Generous Vacation and Holidays, Parental Leave, Legal Insurance with Identity Theft Protection, Employee Assistance Plan, Flexible Spending Accounts, Health Savings Accounts, Wellness Programs, Educational Assistance, Relocation Assistance, and Employee Discounts.This position will remain open for a minimum of 5 days after which it will close when a qualified candidate is identified and/or hired.We accept Word (.doc, .docx), Adobe (unsecured .pdf), Rich Text Format (.rtf), and HTML (.htm, .html) up to 5MB in size. Resumes from third party vendors will not be accepted; these resumes will be deleted and the candidates submitted will not be considered for employment.If you have trouble applying for a position, please email View email address on click.appcast.io is an equal opportunity employer. All qualified applicants, including individuals with disabilities and protected veterans, are encouraged to apply. UT-Battelle is an E-Verify employer.