Data Engineering Manager

Data Engineer ManagerAs a Data Engineer Manager, you will design and maintain data platform road maps and data structures that support business and technology objectives. Naturally inquisitive and open to the deep exploration of underlying data, finding actionable insights, and working with functional competencies to drive identified actions. You also enjoy working both freely and as part of a team and have the confidence to influence and communicate with stakeholders at all levels, and to work in a fast-paced complex environment with conflicting priorities.Reporting into the delivery leader, you will deliver consumable, contemporary, and immediate data and AI solutions to support and drive business decisions. As a Manager Data Engineer, you will be a critical leader in our technology team, responsible for shaping the architectural vision and technical roadmap for our data and AI initiatives. You will work closely with business stakeholders, data scientists, engineers, and product teams to translate complex business requirements into elegant, high-performance technical solutions. This role demands a hands-on architect who can navigate complex environments, mentor teams, and drive the adoption of best practices in solution design and delivery.ResponsibilitiesBrief Description of Role:We are looking for 6+ years of experience in architecture design, data engineering, and AI based initiatives in a customer or business facing capacity and experience in the following:Solution Architecture Design: Participate in the end-to-end design of complex, scalable, and fault-tolerant data and AI solutions, ensuring alignment with business strategy, enterprise architecture standards, and industry guidelines. Knowledge and experience of delivering CI/CD and DevOps capabilities in a data environment.Solution Assessments: Conduct thorough solution assessments of existing systems and proposed architectures, identifying gaps, risks, and opportunities for improvement and modernization.Data Mesh & Data Products: Champion and design a data mesh architectural approach, enabling the creation and management of high-quality, discoverable, and reusable data products for decentralized data ownership and consumption.AI/ML Integration: Define pipelines that seamlessly integrate Artificial Intelligence (AI) and Machine Learning (ML) models into operational systems, ensuring robust data for model training, inference, and monitoring.Cloud Ecosystem Mastery: Design and optimize solutions primarily within AWS and Azure cloud environments, making strategic choices for platform services, infrastructure, security, and cost efficiency.Modern Data Stack Expertise: Drive the adoption and optimal utilization of platforms like Databricks for data engineering, machine learning, and analytics, and/or Snowflake for scalable cloud data warehousing.Data Pipeline Development: Design and oversee the implementation of robust, automated, and observable data pipelines for batch, streaming, and real-time data integration from diverse sources.Solution Integration: Architect and guide the integration of new and existing systems, applications, and third-party services, ensuring data flow, API design, and security protocols are meticulously defined.Technical Leadership & Mentorship: Provide technical leadership, guidance, and mentorship to engineering teams, fostering a culture of innovation, excellence, and continuous improvement.Stakeholder Collaboration: Collaborate extensively with product owners, business analysts, data scientists, and senior leadership to gather requirements, present architectural options, and gain consensus.Technology Evaluation: Continuously evaluate new technologies, tools, and methodologies in data, AI, and cloud spaces, recommending strategic investments and fostering innovation.Security & Compliance: Ensure all architectural designs incorporate robust security measures, data governance, and compliance with relevant regulations (e.g., GDPR, HIPAA, industry-specific standards).QualificationsRequired Qualifications:6+ years of progressive experience in IT with at least 5+ years in a dedicated engineering role, focusing on data, analytics, and AI/ML initiatives.Extensive experience designing and developing identity spines/graphs in both the paid and owned audience spaces.Demonstrable expertise in Solution Architecture Design, including creating architectural diagrams, design documents, and technical specifications for complex systems.Deep hands-on experience with modern data platforms, specifically Databricks and Snowflake, including data lakehouse patterns, SQL warehousing, and MLflow.Extensive experience designing and implementing solutions on major public cloud platforms, with strong proficiency in AWS (e.g., Q, S3, EC2, Lambda, Glue, SageMaker, RDS) and/or Azure (e.g., Data Lake, Data Factory, Synapse Analytics, Azure ML, Cosmos DB).Proven track record in designing and optimizing robust data pipelines using various integration patterns (ETL/ELT, streaming) and tools.Strong programming skills in Python for data engineering, scripting, and automation.Expert-level proficiency in SQL for data manipulation, querying, and optimization.Experience with Java is highly desirable, particularly in enterprise-level application integration or backend services.Solid understanding of Artificial Intelligence (AI) and Machine Learning (ML) concepts, including model development lifecycle, MLOps principles, and integrating ML models into production systems.Experience with various integration patterns (API, message queues, event-driven architectures) and technologies.Excellent communication, presentation, and interpersonal skills with the ability to articulate complex technical concepts to both technical and non-technical audiences.Preferred Qualifications:Platform certifications (e.g., Data Engineer (Professional) – Databricks, Gen AI Engineer (Professional) – Databricks, etc.)Experience with other data technologies such as Kafka, Spark, Flink.Familiarity with containerization (Docker, Kubernetes) and CI/CD practices.Experience building reusable feature pipelinesExperience ensuring training/serving parityManagement of embedded pipelinesHands-on experience with Vector DB integrationsThe following skills are nice to have, and expertise is not required:Adobe (AEP, AJO, CJA, Campaign, Audience Manager, Analytics)Salesforce (Marketing Cloud, Data Cloud)Microsoft Power BI/TableauAdditional InformationWhen You Join Us, We'll Create Something EPIC Together: Epsilon is a global data, technology and services company that powers the marketing and advertising ecosystem. For decades, we've provided marketers from the world's leading brands the data, technology and services they need to engage consumers with 1 View, 1 Vision, and 1 Voice. 1 View of their universe of potential buyers. 1 Vision for engaging each individual. And 1 Voice to harmonize engagement across paid, owned and earned channels.Epsilon's comprehensive portfolio of capabilities across our suite of digital media, messaging and loyalty solutions bridge the divide between marketing and advertising technology. We process 400+ billion consumer actions each day using advanced AI and hold many patents of proprietary technology, including real-time modeling languages and consumer privacy advancements. Thanks to the work of every employee, Epsilon has been consistently recognized as industry-leading by Forrester, Adweek and the MRC. Epsilon is a global company with more than 9,000 employees around the world.Our pillars aren't just words. They're how we show up every day.People centricity: We focus on employee well-being in an environment where colleagues truly care about each other.Collaboration: We work together, support one another, and collectively achieve goals.Growth: There are endless opportunities for growth through learning, development and career advancement.Innovation: We drive progress through cutting-edge solutions and forward-thinking approaches.Flexibility: We've created a balance between work and personal life, and we encourage adaptability to solve problems creatively.Our values guide us to create value for our clients, our people and consumers.Act with integrityWork together to win togetherInnovate with purposeRespect all voicesEmpower with accountabilityThese pillars and values are our foundation—shaping our culture, guiding our decisions, and uniting us in common purpose.Because You Matter:As an Epsilon employee, you deserve