ZTF Pipeline Operator

Pipeline Operator for Zwicky Transient Facility

Caltech is a world-renowned science and engineering institute that marshals some of the world's brightest minds and most innovative tools to address fundamental scientific questions. We thrive on finding and cultivating talented people who are passionate about what they do. Join us and be a part of the diverse Caltech community.

Job Summary

The Physics, Math and Astronomy Division (PMA) at Caltech is looking for a pipeline operator to support operations of a survey telescope. As a Pipeline Operator, you will be part of the Science Data System team for the Zwicky Transient Facility (ZTF). We operate a computing cluster and real-time automated pipelines that process images of the night sky taken with a 640-megapixel camera on a telescope at Palomar Observatory. Our system delivers alerts to astronomers about supernovae, variable stars, asteroids, and other transient phenomenatypically within minutes of each exposure. The Pipeline Operator plays a key role in ensuring that this automated system runs smoothly, performing routine monitoring, diagnostics, and maintenance in collaboration with scientists and engineers on the Science Data System team. They also participate in searching for Near-Earth Asteroids in ZTF data.

This is anticipated to be a one-year assignment.

Essential Job Duties

• Regular tasks:
  • Monitor observing conditions and Science Data System status
  • Alert observatory staff of real-time data issues
  • Monitor Solar System candidate asteroid detections and assess their validity
  • Receive, diagnose, and resolve user-reported problems across Tier 1 (T1) and Tier 2 (T2) support levels
  • Analyze system performance in finding candidate asteroid detections and characterizing them
  • Perform preliminary nightly data quality assurance
  • Liaise with scientists to develop and curate data products, both on an ad hoc and regular basis
• Additional duties (with support from T2/T3 staff):
  • Diagnose technical issues across operating systems, software, and network layers
  • Apply fixes when possible
  • Correct errors in the science database and check its integrity
  • Prepare valid asteroid data for submission to the Minor Planet Center as ZTF discoveries
  • Use the database to help troubleshoot system issues
  • Identify recurring problems in data products
  • Assist in creating and sharing data products with partners
  • Support metadata requests and data releases
  • Help ensure data quality
  • We work a regular work week; however, support of off-hours activities may be required. We encourage continuing career education, and this position offers occasional opportunities to attend industry conferences and related training.

Basic Qualifications

• Bachelor's degree or equivalent experience in Computer Science, Computer Engineering, Astronomy, Physics, Math, or a related technical field.
• Experience with Linux/Unix operating systems, including writing basic shell scripts.
• Experience writing and executing SQL queries.
• Strong written and interpersonal communication skills, with an emphasis on the ability to share ideas in a collaborative setting.

Preferred Qualifications

• In addition to the basic requirements, we value a range of skills and experiences that can enhance your contributions as a pipeline operator. While none of these are mandatory, they may give you a strong start in the role:
• Proficiency in scripting with Perl or Bash.
• Familiarity with PostgreSQL.
• Experience submitting batch jobs on a computing cluster.
• Comfortable using tools like Zoom, Slack, Jira, GitHub, or Confluence.

Required Documents

• Resume.
• Cover letter.