Data Engineering Manager

Responsibilities

We are seeking an experienced Data Engineering Manager to lead the design, development, and delivery of scalable data platforms and data products that power personalized customer experiences across digital retail channels. This leader will manage and develop a team of Data Engineers responsible for building reliable, secure, and high-performance data pipelines, machine learning data infrastructure, and customer data solutions that enable personalized product search, search ranking, recommendations, customer segmentation, behavioral analytics, and omnichannel personalization.

As a people leader, you will be responsible for hiring, onboarding, coaching, performance management, succession planning, and career development while fostering a culture of innovation, operational excellence, and continuous improvement. You will partner closely with Product Management, Data Science, Machine Learning Engineering, Search Engineering, Customer Experience, and senior technology leaders to deliver strategic initiatives that drive measurable business outcomes.

The ideal candidate combines deep expertise in modern data engineering and large-scale data platforms with proven leadership experience and a strong understanding of customer behavior data, personalization systems, recommendation engines, and cloud-based technologies.

Key Responsibilities & Essential Functions

Leadership & Team Management

• Lead, mentor, and develop a high-performing team of Data Engineers across one or more engineering squads.
• Foster an environment of accountability, collaboration, innovation, and customer-centric thinking.
• Manage all people leadership responsibilities, including hiring, onboarding, performance reviews, career development, promotions, succession planning, compensation planning, and employee engagement.
• Coach and mentor engineers in engineering best practices, technologies, processes, and career growth.
• Empower team members to be autonomous, highly effective, and capable of delivering scalable solutions.
• Establish engineering standards, coding practices, operational excellence frameworks, and delivery processes.
• Drive Agile planning, sprint execution, prioritization, and delivery of strategic initiatives.

Data Platform & Engineering

• Lead the design, development, and operation of scalable batch, streaming, and real-time data platforms.
• Develop and maintain data products supporting:
  • Personalized product search
  • Search relevance and ranking optimization
  • Product recommendations
  • Customer segmentation
  • Customer identity and householding
  • Behavioral analytics
  • Omnichannel personalization
• Design scalable data architectures utilizing modern lakehouse, data lake, and cloud-native patterns.
• Build and support feature stores, APIs, and data services used by machine learning and personalization systems.
• Ensure high levels of data quality, reliability, observability, governance, security, and compliance.
• Optimize platform performance, scalability, availability, and cost efficiency.
• Implement monitoring, alerting, SLA management, and incident response procedures for production data platforms.

Technical Strategy & Architecture

• Develop technical roadmaps aligned with business priorities and long-term organizational objectives.
• Lead the technical design and delivery of complex initiatives across multiple systems and platforms.
• Recommend improvements to architecture, scalability, reliability, security, performance, and operational processes.
• Evaluate emerging technologies and industry best practices to enhance platform capabilities.
• Guide engineering teams on architectural decisions, code quality, design reviews, and technical standards.
• Assist in diagnosing and resolving highly complex technical and operational issues.

Customer Personalization & Machine Learning Enablement

• Build foundational data capabilities that support:
  • Product recommendation engines
  • Customer 360 platforms
  • Customer identity resolution
  • Clickstream analytics
  • Purchase behavior analysis
  • Real-time personalization
  • Search relevance optimization
  • Behavioral event processing
• Partner with Data Scientists and Machine Learning Engineers to operationalize and scale personalization models.
• Enable experimentation, A/B testing, feature engineering, and measurement frameworks that improve customer experiences.

Cross-Functional Collaboration

• Collaborate closely with Product Management, Data Science, Machine Learning Engineering, Search Engineering, Customer Experience teams, and business stakeholders.
• Translate business objectives into scalable technical solutions and execution plans.
• Communicate technical strategy, progress, risks, recommendations, and outcomes to leaders and stakeholders.
• Lead cross-functional initiatives with significant business impact and organizational visibility.

Operational Excellence

• Establish operational objectives, work plans, staffing strategies, and resource allocations.
• Ensure adherence to budgets, timelines, and performance requirements.
• Implement strategic policies, processes, and standards that support departmental and organizational objectives.
• Drive continuous improvement through modern engineering practices, automation, observability, and operational excellence.

Qualifications & Key Requirements

Work Experience

• 8+ years of experience in software engineering, data engineering, or related technical disciplines.
• 3+ years of experience leading and developing engineering teams.
• Proven experience delivering large-scale data platform, analytics, or machine learning infrastructure initiatives.
• Experience managing technical roadmaps, cross-functional projects, and engineering delivery.

Knowledge, Skills & Abilities

• Strong leadership skills with demonstrated success building and managing high-performing engineering teams.
• Expert knowledge of data architecture, distributed systems, software design patterns, and engineering best practices.
• Deep understanding of data modeling, ETL/ELT, streaming architectures, and event-driven systems.
• Strong expertise with:
  • Python
  • SQL
  • Apache Spark
  • Kafka
  • Data orchestration frameworks
• Experience with cloud platforms such as AWS and/or Google Cloud Platform.
• Experience with modern data lake and lakehouse architectures.
• Experience building APIs, data products, and services supporting machine learning applications.
• Strong understanding of scalability, reliability, security, observability, and performance engineering.
• Ability to lead technical strategy while balancing business priorities and organizational goals.
• Strong communication and stakeholder management skills.

Preferred Qualifications

• Experience in retail, e-commerce, digital commerce, or customer-facing digital products.
• Experience supporting:
  • Personalized product search
  • Search ranking and relevance systems
  • Recommendation engines
  • Customer personalization platforms
  • Customer 360 initiatives
• Experience working with clickstream, behavioral, transactional, and customer identity data.
• Familiarity with:
  • Recommendation systems
  • Collaborative filtering
  • Embeddings and feature engineering
  • Vector search and semantic search technologies
  • MLOps platforms
  • Feature stores
  • Experimentation frameworks and A/B testing
• Experience supporting machine learning platforms and production AI/ML workloads.

Education

• Bachelor's degree in Computer Science, Engineering, Information Systems, Data Science, or a related field, or equivalent combination of education and professional experience.

Physical Demands & Working Conditions

• Ability to function in a fast-paced, multi-priority environment.
• Ability to travel as needed.
• May require occasional extended hours to support critical business initiatives and production events.

The responsibilities and qualifications outlined above describe the general nature and level of work assigned to this position and are not intended to be an exhaustive list of all duties, responsibilities, or skills required. Duties may be modified at any time based on business needs.

Last revised: 11/01/2024