Data Engineering Lead

Job title Data Engineering Lead Work arrangement Telework. This role is 100% remote. Base pay $175,000.00/yr - $180,000.00/yr Job Description Seeking an experienced and highly skilled Data Engineering Technical Lead to spearhead the design, development, and optimization of robust and scalable data pipelines and data infrastructure. This role requires a deep understanding of data engineering principles, advanced proficiency in distributed data processing technologies, and a proven track record of leading technical teams in an agile environment. The Technical Lead will play a critical role in shaping our data architecture, ensuring data quality and governance, and enabling data-driven initiatives across the organization, particularly within the healthcare and supply chain logistics domains. This role is 100% remote. Key Responsibilities Lead, mentor, and guide a team of data engineers in best practices for data pipeline development, data modeling, and infrastructure management. Provide technical oversight and ensure adherence to architectural standards. Drive the design and implementation of highly scalable, reliable, and efficient data architectures, including data lakes, data warehouses, and streaming platforms. Define technical roadmaps and strategies for data engineering initiatives. Design, develop, and maintain complex ETL/ELT processes and data pipelines using modern data engineering tools and frameworks (e.g., Apache Spark, Kafka, Airflow). Ensure data ingestion, transformation, and loading processes are optimized for performance and reliability. Establish and enforce data governance policies, data quality standards, and data security protocols. Implement robust monitoring and alerting for data pipelines and data quality issues. Leverage expertise in cloud-based data platforms (e.g., AWS, Azure, GCP) to design and implement cloud-native data solutions, optimizing for cost, performance, and scalability. Collaborate closely with product owners, data scientists, data analysts, and other engineering teams to translate business requirements into technical specifications and deliver data solutions that align with strategic objectives. Act as a primary interface with customers, business stakeholders, and cross-functional teams to gather requirements, provide updates, and ensure alignment on data initiatives. Identify and resolve complex data-related performance bottlenecks and architectural challenges. Implement strategies for query optimization and data storage efficiency. Actively participate in agile ceremonies, including sprint planning, backlog refinement, daily stand-ups, and sprint reviews, contributing to a culture of continuous delivery and improvement. Lead and own the full Software Development Life Cycle (SDLC) for data solutions, from conceptualization and design through deployment, testing, and operational support. Create and maintain comprehensive technical documentation, including data flow diagrams, architectural designs, data dictionaries, and operational runbooks. Minimum Qualifications Bachelor’s Degree in a related field; Master’s Degree preferred Industry leading certification for technical expertise preferred 12+ years of experience in a technical field with at least 2 years in a technical leadership or lead engineer role. Other Job Specific Skills Expert-level proficiency in SQL and experience with relational and NoSQL databases. Extensive experience with distributed data processing frameworks (e.g., Apache Spark, Hadoop) and stream processing technologies (e.g., Apache Kafka, Flink). Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures. Proven experience with cloud-based data platforms (e.g., AWS Redshift, S3, Glue, EMR; Azure Data Lake, Synapse, Databricks; GCP BigQuery, Dataflow). Proficiency in at least one programming language commonly used in data engineering (e.g., Python, Scala, Java). Demonstrated experience with CI/CD practices for data pipelines and infrastructure as code (IaC). Solid understanding of data governance, data security, and compliance best practices (e.g., HIPAA, GDPR). Seniority level Mid-Senior level Employment type Full-time Job function Information Technology and Project Management Industries IT Services and IT Consulting #J-18808-Ljbffr