Senior Data Engineer

About us Every day, we deliver safe and secure energy to homes, communities, and businesses, connecting people to the energy they need for their lives. Our expertise and track record position us uniquely to shape the sustainable future of our industry as the pace of change accelerates. To succeed, we must anticipate customer needs, reduce energy delivery costs, and pioneer flexible energy systems. This requires delivering on our promises and seeking opportunities for growth. In IT and Digital, we collaborate closely with the diverse energy businesses within the National Grid group, revolutionizing operations through technology. Embracing modern methodologies and digital mindsets, we drive efficiency and bring new capabilities to internal and external customers as we lead the charge toward a carbon-free future. Our work is critical, as National Grid powers millions of homes and businesses across the UK and US, and the technology we employ is vital to this task. The successful applicant for this position will play a crucial role in our mission, supported by our multicultural, customer-centric global team, with opportunities for professional development. This role operates under a hybrid working model and is based out of our Waltham, MA or Brooklyn, NY office. Hybrid work expectations are subject to business needs and company policy. Job Purpose As a Senior Data Engineer, you will be part of a Data Engineering team that is focused on making critical data at National Grid available to our business teams.  Using the agile framework, you will build end-to-end pipelines based on rigorous engineering standards and coding practices to deliver data that is accessible and of the highest quality.  A Senior Data Engineer will also contribute to the modernization of our architecture and tools to help increase our output, scalability, and speed. A Senior Data Engineer will design and develop highly scalable and extensible data pipelines which enable collection, storage, distribution, modeling, and analysis of large data sets from many channels.  This position requires an innovative software engineer who is passionate about data & data quality. The ideal candidate will have strong data warehousing and API integration experience and the ability to develop scalable data pipelines that make data management and analytics/reporting faster, more insightful, and more efficient. Key Accountabilities

• Develop, test, document and support scalable data pipelines.
• Build out new data integrations including APIs to support continuing increases in data volume and complexity.
• Establish and follow data governance processes and guidelines to ensure data availability, usability, consistency, integrity, and security.
• Build and implement scalable solutions that align to our data governance standards and architectural road maps for data integrations, data storage, reporting, and analytic solutions.
• Collaborate with analytics and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making across the organization.
• Design and develop data integrations and a data quality framework. Write unit/integration/functional tests and document work.
• Design, implement, and automate deployment of our distributed system for collecting and processing streaming events from multiple sources
• Perform data analysis needed to troubleshoot data-related issues and aid in the resolution of data issues.
• Guide and mentor junior engineers on coding best practices and optimization.  Qualifications

Education: 4-year college degree or equivalent combination of education and experience.  Prefer an academic background in Computer Science, Mathematics, Statistics, or related technical field. 6 years of relevant work experience in analytics, data engineering, business intelligence or related field. Skilled in object-oriented programming (Python in particular). Strong Experience in Python, PySpark and SQL. Strong Experience in Databricks. Experience in Snowflake. Experience developing integrations across multiple systems and APIs. Experience with or knowledge of Agile software development methodologies. Experience with cloud-based databases, specifically Azure technologies (e.g., Azure data lake, ADF, Azure DevOps and Azure Functions). Experience using SQL queries as well as writing and perfecting SQL queries in a business environment with large-scale, complex datasets. Experience with data warehouse technologies. Experience creating ETL and/or ELT jobs. Excellent problem solving and troubleshooting skills. Process oriented with great documentation skills. Experience designing data schemas and operating SQL/NoSQL database systems is a plus. Experience in Kafka, Flink ,Fivetran and Matillion is nice to have. Experience in Data Science and Machine Learning is nice to have. Salary Waltham: $114k - $134k a year Brooklyn: $122k - $143k a year National Grid utilizes an assessment that evaluates the job qualifications/characteristics using AI or statistically based scoring. For more information, please view NYC Local Law 144. This position has a career path which provides for advancement opportunities within and across bands as you develop and evolve in the position; gaining experience, expertise and acquiring and applying technical skills. Candidates will be assessed and provided offers against the minimum qualifications of this role and their individual experience. National Grid is committed to providing equal employment opportunities to all employees and applicants for employment regardless of protected class. We do not discriminate on the basis of race, color, religion, sex (including pregnancy, gender identity, and sexual orientation), national origin, age, disability, genetic information, or any other protected status in accordance with applicable federal, state, and local laws. National Grid maintains affirmative action programs for individuals with disabilities and protected veterans. Our employment practices are designed to ensure that all individuals are treated fairly and with respect throughout the hiring process and during employment. National Grid complies with all applicable federal, state, and local anti-discrimination laws. We are dedicated to fostering a workplace that is free from unlawful discrimination and harassment, and we encourage a culture of respect for all.