Senior / Lead Big Data Engineer

The Lead Data Engineer will be responsible for designing, developing, testing, implementing, and supporting scalable data engineering solutions. The role will focus on building data pipelines, data movement solutions, data modeling, data quality, governance, and developer productivity metrics. As part of the CODE Metrics team, the Lead Data Engineer will help develop a developer productivity metrics platform used by engineers across the enterprise to measure developer effectiveness and experience. The role requires strong expertise in Python, SQL, cloud technologies, distributed computing, real-time data processing, data warehouses, and modern data engineering technologies. Key Responsibilities Collaborate with Agile teams to design, develop, test, implement, and support technical solutions using data movement tools and technologies. Lead development efforts involving data movement, distributed computing, and full-stack data systems. Design and develop scalable data pipelines and data engineering solutions using Python and SQL. Work with open-source relational and NoSQL databases and cloud-based data warehousing services. Develop and maintain solutions using Snowflake, Databricks, and other modern data platforms. Optimize information systems for end users and downstream application consumers using sound data design practices. Design and implement data models for data warehousing environments. Develop and support real-time data and streaming applications. Work with distributed data and computing technologies such as MapReduce, Hadoop, EMR, Kafka, and Spark. Apply data quality, governance, and permissible-use principles throughout data engineering activities. Leverage interactive AI development tools such as Claude Code, Windsurf, and GitHub Copilot to accelerate software delivery. Stay current with emerging technologies and modern data engineering practices. Experiment with and evaluate new technologies to improve engineering solutions. Participate in internal and external technology communities. Mentor other members of the engineering team and contribute to the broader engineering community. Required Qualifications 10+ years of application development experience using Python and SQL. 5+ years of experience working with public cloud platforms such as AWS, GCP, or Azure. 5+ years of experience with distributed data and computing technologies such as MapReduce, Hadoop, EMR, Kafka, or Spark. 5+ years of experience working with real-time data and streaming applications. 5+ years of experience with data warehouses and services such as Snowflake, Databricks, or Redshift. 5+ years of experience with data modeling for data warehousing. Strong hands-on experience with Python and SQL. Strong understanding of data engineering, data movement, and data pipeline development. Experience working with relational and NoSQL databases. Experience working in Agile development environments. Experience leveraging interactive AI development tools such as Claude Code, Windsurf, or GitHub Copilot. Strong problem-solving, collaboration, and communication skills. Preferred Qualifications Experience with Cube for semantic data layers. Experience with Tableau or Quick for business intelligence and analytics. #J-18808-Ljbffr Compunnel, Inc.