Senior Data Engineer (1043) - DataSF - Office of City Administrator

Overview Company Description DataSF is seeking a Senior Data Engineer with 3+ years of experience to join our growing team. Reporting to the Principal Data Engineer, you will be instrumental in designing, building, and maintaining the City's data infrastructure, enabling robust data pipelines and reliable data access for analytical and operational needs. Department: DataSF Job class: 1043 Senior Data Engineer Role type: Permanent Exempt Hours: Full-time. The department has a hybrid work schedule. About Permanent Exempt: This is an Exempt position excluded by the Charter from the competitive Civil Service examination process, pursuant to the City and County of San Francisco, Charter Section 10.104. It is considered "at will" and shall serve at the discretion of the Appointing Officer. Responsibilities Platform Administration: Manage the central Snowflake data warehouse, including access control, security policies, resource monitoring, performance tuning, and cost optimization. Administer the platform with a focus on data democratization and accessibility, while protecting privacy and security. Pipeline Development: Build and maintain scalable and resilient pipelines to ingest and structure data from diverse sources. Design infrastructure to support both streaming and batch processes, and both structured and unstructured data sources. Infrastructure as Code (IaC): Use Terraform to define, deploy, and manage data infrastructure, ensuring pipelines are reproducible, version-controlled, and production-ready. Best Practices & Innovation: Champion and implement best practices for documentation, data modeling, warehouse architecture, SQL optimization, and testing. Provide guidance to department partners on data engineering best practices. Collaboration: Work with data scientists, analysts, product managers, software engineers, and nontechnical stakeholders to understand data requirements and build solutions that meet their needs. Monitoring & Support: Proactively monitor the health of the data platform and pipelines, troubleshoot issues, and ensure high standards of data quality and availability. Qualifications Minimum Qualifications: Education : An associate degree in computer science, computer engineering, information systems, or a closely related field from an accredited college or university OR equivalent in total course credits/units (as specified). Experience : Three (3) years of experience analyzing, installing, configuring, enhancing, and/or maintaining the components of an enterprise network. Substitution : Additional experience may be substituted for the required degree on a year-for-year basis (up to two years). One year is equivalent to 30 semester units / 45 quarter units with minimums as noted. Completion of the 1010 Information Systems Trainee Program may be substituted for the required degree. Verification : Please clearly indicate in your application how you meet the minimum qualifications. Applicants must submit verification of qualifying education and experience. See Verification of Experience and/or Education for details. Note: Falsifying education, training, or work experience may result in disqualification. One-year full-time employment is equivalent to 2000 hours. Overtime hours are not included in this calculation. Applicants must meet minimum qualification requirements by the final filing date unless otherwise noted. Desirable Qualifications Technical Knowledge Hands-on experience with Snowflake, BigQuery, or Databricks. Advanced SQL and data transformation using dbt. Python (pandas, PySpark) for data processing. Terraform or similar IaC tools. Experience with data pipelines using Airflow, Dagster, or similar tools. Data warehousing concepts, data modeling, and ELT principles. Data governance, security, and privacy principles. Real-time streaming technologies (Kafka, Kinesis, Snowpipe). Deploying data pipelines for machine learning models. Collaboration and Communication Skills: Problem-solving, clear communication to non-technical stakeholders, and cross-functional teamwork. Mission Alignment: Commitment to equity, transparency, and ethical data use; interest in public sector data and social impact work. Selection Procedures The selection process will evaluate applications against minimum requirements and assess knowledge, skills, and abilities. Additional screening, such as an oral interview or written/performance exercise, may be used. Not all applicants will be invited to continue in the process. Additional Information Information about the hiring process, equal employment opportunity, disability accommodations, veterans’ preference, and other topics is provided as part of the application process. See the official City and County of San Francisco employment pages for details. How to Apply Applications are accepted online. Visit to begin the application process. Selected candidates should apply as soon as possible; the announcement will close when filled or on the listed deadline (June 21, 2026 at 11:59 PM PST). A Supplemental Questionnaire will be required via this link: HR Analyst Information: Connie Poon at View email address on click.appcast.io Condition of Employment: The City and County of San Francisco encourages women, minorities and persons with disabilities to apply. Applicants will be considered regardless of sex, race, age, religion, color, national origin, ancestry, disability, HIV status, sexual orientation, gender identity, or other protected status. #J-18808-Ljbffr City and County of San Francisco