Senior Data Engineer

About this Project:

This initiative involves modernizing and migrating 10-15 years of accumulated meeting house data to Microsoft Azure, upgrading data technologies, and improving the consistency, security, and global availability of the data. The successful candidate will be instrumental in moving away from legacy tools like SSRS and SQL Server on-prem to cloud-based solutions like Databricks and PowerBI. This role will support ongoing reporting and automation efforts for the meeting house facilities department and potentially other organizational units.

Key Responsibilities:

• Lead the migration of data from legacy systems (SQL Server, SSRS, SSAS, SAP Data Services) to Azure-based infrastructure using Databricks, PowerBI, and related cloud tools.
• Design and implement dimensional modeling and data warehousing solutions using DBT (Data Build Tool), Python, and other technologies.
• Manage ETL pipelines, ensuring secure, reliable, and scalable data transfer using DBT, Databricks notebooks, and REST APIs.
• Support reporting and analytics efforts using PowerBI, developing semantic layers, SQL endpoints, and lakehouse structures in Microsoft PowerBI Fabric.
• Collaborate closely with outsourced vendors and internal team members, ensuring smooth project delivery and knowledge transfer.
• Lead efforts to modernize data workflows and implement CI/CD pipelines using Microsoft DevOps, Terraform, and Python automation.
• Enable data accessibility and insights for global data stewards and other departments by establishing robust reporting structures.
• Participate in Agile/sprint planning, ensuring alignment with two-week sprint cycles, making commitments, and delivering on them.
• Communicate effectively with contractors and internal teams, maintaining high standards of technical English.

Required Skills & Experience:

• Cloud Technologies: Strong experience with Microsoft Azure (Data Lake, Azure Synapse, etc.), Databricks, and Azure DevOps for CI/CD.
• Data Warehousing: Proven expertise in data warehousing methodologies, dimensional modeling, and ETL processes. Experience with DBT for managing ETL workloads.
• Programming & Automation: Proficiency in Python for data extraction and automation tasks, particularly in conjunction with REST APIs and CI/CD workflows.
• Reporting & Visualization: Advanced experience with PowerBI, including building semantic models, working with PowerBI Fabric, SQL endpoints, and migrating from on-prem SSRS to PowerBI Service.
• Collaboration & Communication: Excellent English communication skills and proven ability to work in Agile environments. Experience working with outsourced vendors and internal stakeholders in a collaborative manner.
• Senior-Level Experience: A seasoned professional with a track record of successfully delivering complex cloud migration projects, ideally without needing extensive onboarding or guidance.

Preferred Skills:

• ML Document Processing: Familiarity with machine learning document processing tools is a plus.
• Terraform: Hands-on experience using Terraform for infrastructure management in the cloud.