Senior Data Engineer / Lead Data Engineer

Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer (or Lead Data Engineer) to design, build, and optimize scalable data pipelines and ETL solutions across enterprise environments. The ideal candidate will bring deep expertise in Big Data technologies, Azure cloud platforms, and data engineering best practices, while also providing technical leadership and mentorship to the broader data team. This role will play a key part in ensuring high-quality, secure, and reliable data delivery that supports analytics, AI, and business intelligence initiatives.Key ResponsibilitiesLead the design, development, and implementation of high-performance and reusable data pipelines and ETL processes to support enterprise data initiatives.Establish and enforce best practices for data modeling, data integration, and data product development.Provide thought leadership and hands-on execution in data architecture design, domain selection, and data platform enhancement.Partner with the Enterprise Data Management team to define data standards, governance policies, and procedures ensuring data integrity, quality, and compliance.Collaborate with business stakeholders to understand requirements and translate them into efficient technical solutions.Mentor and guide junior engineers on data architecture, development practices, and performance optimization.Communicate effectively with cross-functional teams and leadership to present technical insights and recommendations.Ensure adherence to CleanEarth internal control standards, addressing internal and external audit findings promptly.Perform additional duties and contribute to strategic data initiatives as assigned.Required Skills & Qualifications12+ years of experience in data engineering, including ETL development and integration.Proven ability to design scalable, reusable, and high-performance data pipelines using modern architecture patterns.7+ years of hands-on experience with Big Data technologies such as Apache Spark and Kafka.Strong experience working with Azure Data Lake, Azure Data Factory, Azure DevOps, and Databricks.Expertise in Python and PySpark for data transformation, automation, and analytics workloads.Experience implementing data quality frameworks and fuzzy matching algorithms for customer data management.Working knowledge of SQL, SQL Server, and Vector Databases.Familiarity with Databricks AI foundational models and Genie is highly desirable.Strong analytical, communication, and problem-solving skills.Ability to thrive in a fast-paced, collaborative, and evolving data environment.Preferred QualificationsExperience with AI/ML-driven data pipelines or DataOps frameworks.Exposure to data governance or metadata management tools.Prior experience in regulated industries or environments with strong audit/compliance standards.