Lead Data Engineer

Lead Data Engineer

Location: Remote Duration: Fulltime – Visa independents only or C2H

Responsibilities

• Lead data wrangling activities utilizing Databricks, Python, pyspark, snowflake, kafka and AWS services to process, cleanse, and transform large volumes of data for analytical and operational purposes.
• Design and implement scalable data pipelines and workflows to efficiently ingest, process, and store data from various sources into our data lake and data warehouse environments.
• Act as a solution provider, collaborating with cross-functional teams including data scientists, analysts, and business stakeholders to understand data requirements, identify data sources, and develop solutions to address business needs.
• Define data architecture standards, best practices, and guidelines to ensure consistency, reliability, and performance across data engineering initiatives.
• Establish and maintain data governance policies and procedures to ensure data quality, integrity, and security throughout the data lifecycle.
• Lead efforts to identify, evaluate, and implement data management tools and technologies to optimize data processing and analytics capabilities.
• Manage and prioritize data assets and initiatives based on business priorities, ensuring alignment with organizational goals and objectives.
• Lead the data engineering team, providing technical guidance, mentoring, and coaching to team members, fostering a culture of collaboration, innovation, and continuous learning.
• Serve as the primary point of contact and subject matter expert for data engineering solutions, providing guidance, support, and mentorship to team members and stakeholders.
• Act as a liaison between technical teams and business stakeholders, effectively communicating technical concepts and solutions to non-technical audiences.