Lead Data Engineer

Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets. An ecosystem where human imagination is at the heart of every decision. Where no possibility is written off, only challenged to get better. We believe that a true Fractalite is the one who empowers imagination with intelligence. Fractal has been featured as a Great Place to Work by The Economic Times in partnership with the Great Place to Work® Institute and recognized as a 'Cool Vendor' and a 'Vendor to Watch' by Gartner. Please visit Fractal | Intelligence for Imagination for more information about Fractal Note: This position is not eligible for Immigration Sponsorship at this time Role Overview: The Lead Data Engineer is responsible for integrating and harmonizing data from multiple internal and external sources to build scalable, enterprise-grade data pipelines on Azure. The role involves collaborating with clients, vendors, and cross-functional teams to understand data sources, define business metrics and KPIs, and develop trusted datasets that support multiple downstream analytics, reporting, and business initiatives. Strong expertise in Azure data services, Python, stakeholder management, and data modelling is essential. Key Responsibilities: Translate business requirements into technical specifications; evaluate and select optimal tools, technologies, and architectural patterns based on scalability and cost. Lead and coordinate with multiple teams to understand data sources, design the data model and build in the pipelines. Design and develop production-optimized pipelines using Azure Data Factory, Databricks, and other Azure services. Build reusable, modular Python scripts and configure Azure Storage (Blob, Data Lake) with appropriate security and partitioning. Develop serverless, event-driven workflows using Azure Functions and Logic Apps. Implement error handling, logging, monitoring, and testing (unit, integration, performance) to ensure pipeline reliability. Optimize performance by tuning Databricks clusters, ADF activities, and Python code. Document solutions, enforce engineering standards and code reviews, mentor junior engineers, and troubleshoot production pipelines. Required Qualifications: 7+ years in data engineering or related fields. 5+ years hands-on with Azure Data Factory building enterprise data pipelines. 5+ years with Databricks, including cluster management, jobs, and performance tuning. Advanced Python proficiency and strong SQL across relational and distributed data systems. Strong experience with Azure Storage (Blob, Data Lake, Table) and hands‑on experience with Azure Functions and/or Logic Apps. Proficiency in ETL/ELT design; experience with Git and CI/CD for data engineering. Agile/Scrum experience with strong problem‑solving, communication, and documentation skills. Preferred Qualifications: Apache Spark and big‑data frameworks; streaming platforms (Azure Event Hubs, Kafka). Data quality and governance frameworks; cloud security, compliance, and best practices. Azure certifications (Azure Data Engineer Associate or higher) MLOps concepts; Docker, Kubernetes, and CI/CD. Technical Skills: Pay: The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Fractal, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is $120,000 - $165,000 with the potential for a higher base depending on experience, skills, and overall fit for the position. In addition, for the current performance period, you may be eligible for a discretionary bonus. As a full-time employee of the company or as an hourly employee working more than 30 hours per week, you will be eligible to participate in the health, dental, vision, life insurance, and disability plans in accordance with the plan documents, which may be amended from time to time. You will be eligible for benefits on the first day of employment with the Company. In addition, you are eligible to participate in the Company 401(k) Plan after 30 days of employment, in accordance with the applicable plan terms. The Company provides for 11 paid holidays and 12 weeks of Parental Leave. We also follow a "free time" PTO policy, allowing you the flexibility to take the time needed for either sick time or vacation. Fractal provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. #J-18808-Ljbffr Fractal, Inc.