Senior Data Engineer

Description

The salary above reflects the entire range for this classification. Starting salary is dependent on experience and may be offered up to the midpoint of the salary range. Any starting salary above the midpoint requires authorization by the PRISM Executive Committee.

PRISM is a joint powers authority (JPA) that provides risk management and coverage programs for over 1,700 public entities throughout California. Our organization is member driven meaning that our public entity members have numerous opportunities available to participate in the governance of our JPA.


One of PRISM's greatest assets is its staff. PRISM employees are smart, creative, hard-working, and passionate individuals working in areas ranging from member services, risk control, claims administration, information technology, accounting, and risk pool administration. Working here requires energy, commitment, and teamwork. At the same time, we offer a great work environment built upon our Core Values of People, Families, Trust, Integrity, and Growth . We are looking for an individual who shares these values to join the PRISM team.

We are seeking a Senior Data Engineer to play a central role in our data platform modernization. This is a hands-on technical position focused on migrating our on-premises MS SQL Server environment and Qlik reporting layer into Microsoft Fabric (Fabric Lakehouse, Data Factory, and Data Warehouse) with Power BI as our reporting layer. You will design and build robust data pipelines, establish solid data modeling standards, and help set the engineering foundation that the team will scale on for years to come.

This role also carries mentorship responsibility - you will collaborate with others in the Data & Analytics team, conduct code reviews, and help grow the team's collective capability on the new platform.

Job Responsibilities

Migration & Platform Engineering

• Contribute to the technical migration from MS SQL Server (on-prem) and Qlik to Microsoft Fabric and Power BI
• Design and implement data pipelines using Fabric Data Factory (pipelines and dataflows) and Fabric Notebooks, leveraging Python and Spark SQL for complex transformations and business logic
• Build and maintain Lakehouse architecture on OneLake, including medallion layer design (Bronze / Silver / Gold)
• Develop and optimize data models in Fabric Data Warehouse and/or Fabric Lakehouse
• Write performant transformation logic using Python, Spark SQL, and T-SQL

Data Modeling & Architecture

• Design scalable, maintainable data models aligned to business and reporting needs
• Establish and enforce naming conventions, schema standards, and documentation practices
• Collaborate with analysts and Power BI developers to ensure the semantic layer is well-structured and performs at scale

DevOps & Engineering Excellence

• Own and maintain CI/CD pipelines for data assets using Azure DevOps.
• Implement version control practices, testing frameworks, and deployment automation for data pipelines
• Design and manage data platform orchestration across pipelines, notebooks, and warehouse workloads - including scheduling, dependency management, alerting, and failure handling
• Proactively identify and resolve performance bottlenecks across pipelines, notebooks, SQL code, and overall platform architecture
• Participate in architecture discussions and contribute to platform roadmap decisions

Compliance & Data Governance

• Ensure all data handling practices comply with regulatory requirements and organizational PII/PHI policies
• Implement appropriate data masking, access controls, and audit logging within the platform
• Partner with security and compliance teams to maintain data governance standards

Mentorship

• Partner with and actively uplift team members who are growing their Fabric skills
• Conduct code reviews with constructive, solutions-oriented feedback
• Contribute to internal documentation, runbooks, and knowledge sharing
• Approach technical challenges with a growth mindset by modeling curiosity, continuous learning, and a bias toward practical, solutions-focused problem solving

Requirements

We are seeking candidates who can fulfill this position through a hybrid work model consisting of mostly remote work and in-person work in our Folsom office several times throughout year.

Minimum Required Skills & Experience

• Proven experience as a data engineer, with a strong track record in data pipeline development and data modeling
• Strong proficiency in Python for data engineering tasks
• Working knowledge of Spark SQL and distributed compute concepts
• Deep experience with MS SQL Server and T-SQL
• Understanding of CI/CD practices applied to data engineering (Azure DevOps)
• Ability to communicate technical concepts clearly to non-technical stakeholders

Desirable Skills & Experience

• Hands-on experience with Microsoft Fabric - specifically Lakehouse/OneLake, Data Factory, and Data Warehouse
• Familiarity with Power BI semantic models and dataset design (Direct Lake)
• Experience in a government or public sector environment
• Experience working with compliance-regulated data and implementing appropriate controls for PII/PHI
• Hands on experience with Microsoft Fabric and/or Microsoft certifications (DP-700 Fabric Data Engineer, DP-600 Fabric Analytics Engineer, DP-203 Data Engineer, or similar)

Physical Requirements

Physical requirements of this position typically include: reaching, grasping, talking, hearing, seeing, repetitive motions, exerting up to 20 pounds of force occasionally and/or up to 10 pounds of force frequently, and/or negligible amount of force constantly to move. Some travel may be required.