Senior Data Engineer

Overview Are you passionate about transforming legacy data systems and driving enterprise‑scale platform modernizations? Do you thrive in modern cloud environments and want to help shape our strategic migration to Microsoft Fabric? Join our innovative Data & Analytics team as a Senior Data Engineer , where you'll play a pivotal role in refactoring, optimizing, and scaling our core enterprise data platform. You will build robust, scalable PySpark pipelines and modern Lakehouse structures within our Azure Synapse environment, while also helping lay the foundation for our upcoming Microsoft Fabric migration in collaboration with external architecture partners and mentoring junior team members. This is your opportunity to make a lasting impact at the intersection of hands‑on data engineering and next‑generation modernization – solving complex pipeline bottlenecks today while architecting the ecosystem of tomorrow. We embrace modern development tools and encourage the use of AI companions (like GitHub Copilot or Google Gemini) to speed up workflows. However, we believe that true senior‑level engineering requires the foundational depth to question, validate, and ruthlessly scrutinize AI‑generated solutions before they ever hit production. Code accountability ultimately rests with the engineer. Responsibilities Data Platform Modernization: Lead the technical execution of refactoring legacy logic (predominantly SQL Stored Procedures) into modern, optimized PySpark pipelines within a Lakehouse architecture. Architecture Collaboration: Partner alongside a third‑party architecture vendor to help influence, design, and implement foundational frameworks, CI/CD pipelines, and governance standards for our upcoming Microsoft Fabric environment. Hands‑on Engineering: Actively design, build, and optimize robust ETL/ELT pipelines, ensuring high performance, scalability, and data integrity across the Azure ecosystem. Operational Excellence & Optimization: Take ownership of data platform health by refactoring fragile processes, minimizing technical debt, and building highly stable frameworks that reduce production bottlenecks. Mentorship & Leadership: Act as a technical guide and escalation point for associate data engineers, establishing engineering best practices and setting up the team to successfully manage ongoing platform maintenance. Data Quality & Governance: Implement frameworks that ensure high data quality, strict security standards, and comprehensive lineage tracking across all data ingestion and migration paths. Qualifications Experience: 5+ years of dedicated data engineering or related experience, with a proven track record of designing, building, and deploying pipelines in enterprise cloud environments. SQL Mastery: Deep expertise in advanced T‑SQL, including extensive experience writing, debugging, and reverse‑engineering complex Stored Procedures. Advanced PySpark/Python: Strong proficiency in Python and PySpark for data processing, with the capacity to translate complex, legacy SQL‑driven logic into efficient notebook‑based pipelines. Azure Expertise: Strong hands‑on experience with core cloud services, specifically: Azure Synapse Analytics Azure Pipeline Azure Data Lake Storage (ADLS Gen2) Critical Code Evaluation: Possess the deep, foundational understanding of SQL and PySpark internals required to critically evaluate, debug, and optimize complex code—whether written by peers or generated by AI systems. ETL/ELT Pipeline Management: Proven track record of building, running, and troubleshooting robust data pipelines in high‑availability production environments. Problem Solver: A passion for performance tuning, troubleshooting bottlenecked processes, and re‑engineering legacy code for maximum resilience. Preferred Qualifications (Nice‑to‑Have) Conceptual understanding of, or early hands‑on exposure to, Microsoft Fabric tools (OneLake, Lakehouses, Warehouses). Experience collaborating with external systems integrators, consultants, or architecture vendors to deploy enterprise‑level frameworks. Strong foundation in Git‑based workflows, DevOps principles, and CI/CD concepts in a data engineering ecosystem. Compensation & Benefits We are excited to share that the base salary range for this position is $104,626.08 - 139,501.44 USD Annual. This position is also eligible for an annual discretionary bonus, targeted at 3%. Employees are also eligible for a robust benefit program, which includes Medical, Dental, Vision, Prescription Drug Coverage, 401k Plan, Wellness Program, Life Insurance, Paid Time Off, and Paid Parental Leave, among other benefit plan options. Equal Opportunity Employer/Protected Veterans/Individuals With Disabilities The contractor will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by the employer, or (c) consistent with the contractor’s legal duty to furnish information. 41 CFR 60‑1.35(c) LA County Applicants: The Company will consider qualified applicants, including those with criminal histories, in a manner consistent with applicable state and local laws. #J-18808-Ljbffr