Systems Engineering, Design & Development

Union: ID: #132811 HPC Systems Engineer

The Mission of the San Diego Supercomputer Center is to translate innovation into practice. SDSC adopts and partners on innovations in industry and academia in the areas of software, hardware, computational and data sciences, and related areas, and translates them into cyberinfrastructure that solves practical problems across any and all scientific domains and societal endeavors. Cyberinfrastructure refers to an accessible, integrated network of high‑performance computing, data, and networking resources and expertise, focused on accelerating scientific inquiry and discovery. With more than 250 employees and $30‑50M of revenue a year, SDSC is a global leader in the design, development, and operations of cyberinfrastructure.

SDSC supports hundreds of multidisciplinary programs spanning a wide variety of domains, from earth sciences and biology to astrophysics, bioinformatics, and health IT. SDSC presently operates multiple large HPC systems ranging from a 120k x86 CPU core general purpose system to a system explicitly designed for Artificial Intelligence and Machine Learning, and a nationally distributed system open for all of academia to integrate with. SDSC offers research data services across the entire vertical stack from universally scalable storage to consulting services on FAIR, Big Data, and AI. SDSC offers a rich set of cloud services both on‑premise, in the commercial cloud, and as hybrid services across both.

SDSC has three geographic scopes, a national scope supporting cyberinfrastructure for the entire US research and education community, a California scope with a special focus on convergence research that addresses the three dominant threats to CA: Drought, Fire, Earthquakes, and a campus scope focusing on advancing the global impact of SDSC by advancing the research objectives of the UC San Diego faculty, researchers, and students. SDSC annually trains thousands of researchers in cyberinfrastructure tools and software, and supports thousands of individual researchers via Unix accounts on its large HPC systems. SDSC was a leader developing the Science Gateway concept, and continues to be a global leader in its evolution. SDSC operates multiple major such gateways with user communities ranging from the tens of thousands to the millions. SDSC’s educational programs includes online courses that have been attended by more than a million students.

The HPC Systems Engineer is responsible for overseeing the management of national and campus‑level high‑performance computing (HPC) clusters and their associated storage systems, including large parallel file systems, NFS file servers, and underlying storage technologies. This role involves Linux systems administration, on‑call duties, managing hardware, operating systems, I/O, and installation and maintenance of the software environment. Responsibilities include in‑depth analysis, testing, scripting, and benchmarking of advanced systems, data, and networks in a research and performance evaluation setting. The engineer contributes to the design, installation, and upgrade of large‑scale HPC clusters and storage resources. Familiarity with high‑performance file systems such as Lustre, Ceph, and GPFS is required, along with deep knowledge of system internals, storage, networking, operating systems, and emerging technologies.

The engineer works closely with other teams to integrate HPC systems into SDSC’s network, cloud, and user environments, and develops security procedures to safeguard these systems. The role involves managing performance testing at CPU, memory, GPU, interconnect bandwidth, and file system IOPS levels. Performance testing is crucial for troubleshooting and optimizing system functionality. The engineer deploys monitoring tools with appropriate alert mechanisms for rapid incident detection and collaboration with SDSC Operations to streamline incident evaluation and resolution.

The engineer also serves as a liaison between the Operations team and computational scientists, providing advanced technical guidance and supporting ongoing training efforts. They contribute to national projects like ACCESS and may present their work at national meetings as necessary. Expertise spans system internals, data management, storage, network infrastructure, security, and the interrelationships between these critical components.

Bachelor’s degree in a related area and/or equivalent experience/training. Professional technical engineering or technical programming experience or Master’s Degree preferred.

Strong knowledge of administering Linux systems, primarily Red Hat and its derivatives, including services, networking, and file systems.

Ability to install, maintain, upgrade, and troubleshoot large (petabyte‑scale) high‑performance parallel and distributed file systems such as Lustre, GPFS, and Ceph.

Strong demonstrated experience with a major configuration management software, including application packaging and installation.

Experience programming and scripting, with Bash and Python preferred.

This position has recently been accreted by TX union and will be a part of that union moving forward.

Occasional evenings and weekends may be required.

Overtime and weekends may be required.

The University of California is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, age, protected veteran status, or other protected status under state or federal law.

As a condition of employment, the final candidate who accepts an offer of employment will be required to disclose any final administrative or judicial decisions within the last seven years determining that they committed any misconduct; Misconduct" means any violation of the policies governing employee conduct at the applicant’s previous place of employment, including, but not limited to, violations of policies prohibiting sexual harassment, sexual assault, or other forms of harassment, or discrimination, as defined by the employer. For reference, below are UC’s policies addressing some forms of misconduct:

UC Sexual Violence and Sexual Harassment Policy

UC Anti-Discrimination Policy