Senior Big Data Engineer

Senior Software EngineerEpsilon powers connected marketing by unlocking the full potential of data and identity. For this role, we are seeking a qualified Senior Software Engineer to join Epsilon's Development Team in our Data Practice. In this role with our company, you will be focused on project tasks for Epsilon's TotalSource Plus (TSP) demographic and lifestyle products. As a member of the Epsilon Data Team, you'll be a critical part of a fast-moving, innovative team where you'll work independently in some situations as well as working with a greater team in tasks that require full collaboration.Responsibilities include:Design, develop, and maintain scalable data processing solutions across on-premises and cloud environments using Python and Apache Spark on Databricks.Optimize and fine-tune Spark jobs for performance, including resource utilization, shuffling, partitioning, and caching, to ensure maximum efficiency in large-scale big data environments.Design and implement scalable, fault-tolerant data pipelines with end-to-end monitoring, alerting, and logging.Leverage AWS cloud services to build and manage data pipelines and distributed processing workloads.Develop and optimize SQL queries across relational and data warehouse systems.Apply design patterns and best practices for efficient data modeling, partitioning, and distributed system performance.Use Git for source control and maintain strong unit and integration testing practices.Collaborate with Product Owners, partners, and cross-functional teams to translate business requirements into technical solutions.Demonstrate strong analytical skills with the ability to extract actionable insights from large datasets and support data-driven decision-making.Mentor junior engineers, conduct code reviews, and contribute to establishing engineering best practices and standards.Qualifications include:5+ years of software development experience in a scalable, distributed, or multi-node environment.Proficient in programming with Scala, Python, or Java; comfortable building data-driven solutions at scale, with Databricks preferred.Significant experience with Apache Spark and exposure to Hadoop, Hive, and related big data technologies.Demonstrated experience with cloud platforms (AWS, preferred) and an interest in cloud migration projects.Exposure to modern data tools and frameworks such as Kubernetes, Docker, and MWAA/Airflow.Strong problem-solving skills with the ability to own problems end-to-end and deliver results.Collaborative team player with an eagerness to learn from peers and mentors while contributing to a culture of growth.Motivated to grow your career within a dynamic, innovative company.Epsilon operates in 30+ countries across EMEA, the Americas, and APAC, with more than 8,000 employees. Epsilon helps brands deliver personalized experiences at scale and build lasting customer relationships by unlocking the full potential of data and identity.Epsilon's policy is not to discriminate against any applicant or employee based on actual or perceived race, age, sex or gender (including pregnancy), marital status, national origin, ancestry, citizenship status, mental or physical disability, religion, creed, color, sexual orientation, gender identity or expression (including transgender status), veteran status, genetic information or any other characteristic protected by applicable federal, state or local law. Epsilon also prohibits harassment of applicants and employees based on any of these protected categories.Compensation Range: USD $88,900.00 - USD $165,100.00/Annually.