Senior Data Engineer

Job Details Job Location: MBI TN - Loudon, TN 37774 The Senior Data Engineer designs, builds, and supports the data pipelines, integrations, and platform capabilities that power Malibu Boats, Inc.’s business applications, manufacturing operations, dealer ecosystem, analytics, and enterprise reporting. This is a hands‑on senior engineering role that bridges MBI’s current SQL‑based environment with its modern Microsoft Fabric data platform. The Senior Data Engineer will maintain the reliability of business‑critical production integrations while progressively modernising legacy ETL, stored procedures, linked‑server processes, and middleware workflows. The ideal candidate combines strong SQL and production‑support experience with modern cloud data engineering skills, including Microsoft Fabric, lakehouse architecture, Python, PySpark, Delta Lake, APIs, and automated deployment practices. Success requires technical depth, practical judgment, end‑to‑end ownership, and the ability to collaborate effectively across a fast‑moving organization. Essential Duties and Responsibilities Modern Data Platform Engineering Design, develop, test, deploy, and operate scalable ETL/ELT pipelines within Microsoft Fabric or a comparable cloud data platform. Build and maintain Fabric lakehouses, warehouses, Data Factory pipelines, notebooks, SQL analytics endpoints, and related platform components. Develop PySpark and Delta Lake solutions supporting full loads, incremental processing, merge/upsert patterns, partitioning, and schema evolution. Apply medallion architecture principles, preserving source fidelity in Bronze, creating validated and conformed data in Silver, and delivering business‑ready datasets through Gold. Build pipelines using reusable, version‑controlled Python components rather than embedding complex business logic entirely within notebooks. Implement watermark‑based incremental loading, write‑back‑on‑success controls, checkpointing, and idempotent processing so pipelines can be safely restarted or rerun. Design data models and transformation patterns that balance source‑system fidelity, enterprise consistency, performance, and business usability. Enterprise and Operational Integration Build and support bidirectional integrations between the enterprise data platform and operational systems, including ERP, CPQ, CRM, dealer portals, internal applications, vendor platforms, and third‑party SaaS solutions. Develop integrations using REST APIs, webhooks, SFTP, JSON, flat files, scheduled exports, middleware, and database‑based interfaces. Support operational write‑back scenarios such as ERP transactions, CRM updates, dealer‑system exchanges, and downstream application feeds. Design integrations with appropriate transactional boundaries, correlation identifiers, retry logic, reconciliation, auditability, and delivery confirmation. Account for the different performance, latency, validation, and recovery requirements of analytical pipelines and operational integrations. Implement secure connectivity using service principals, managed identities, Azure Key Vault, on‑premises data gateways, and other approved security patterns. Current‑State Production Support and Modernisation Develop, optimise, and troubleshoot complex SQL queries, stored procedures, views, database objects, SQL Agent jobs, and production ETL processes. Maintain and safely modify existing data solutions, including unfamiliar or insufficiently documented code. Support linked servers and cross‑system queries while identifying their performance, security, and reliability limitations. Operate and troubleshoot existing middleware and iPaaS workflows, such as Workato, including error resolution, record reprocessing, and changes required by source or target systems. Support batch‑processing solutions and file‑based integrations using SFTP, CSV, Excel, and other standard enterprise formats. Plan data extraction around production OLTP workloads, considering locking, resource utilisation, operational schedules, and system performance. Apply a modernisation mindset to legacy support: stabilise the process, document its business purpose and dependencies, and prepare it for migration rather than unnecessarily extending technical debt. Reliability, Quality, and Operational Excellence Build data solutions with validation gates, zero‑row protections, schema‑drift detection, error handling, structured logging, monitoring, and actionable alerting. Design pipelines to fail visibly and safely instead of silently producing incomplete, duplicated, or inaccurate data. Investigate complex data and integration incidents, perform root‑cause analysis, and implement sustainable corrective and preventive solutions. Improve the performance, resiliency, observability, scalability, and maintainability of existing data processes. Protect data quality and completeness by reconciling delivered records, preserving unresolved records when appropriate, and preventing silent data loss. Support critical production issues and participate in scheduled after‑hours support when necessary. Engineering Practices and Collaboration Use Git‑based engineering practices, including feature branches, pull requests, peer reviews, automated testing, and controlled promotion across development, test, and production environments. Contribute to CI/CD pipelines and repeatable deployment processes for database, integration, and Microsoft Fabric solutions. Apply professional Python development practices, including modular design, dependency management, unit testing, linting, and pre‑commit quality checks. Create and maintain clear technical documentation covering data flows, source‑to‑target mappings, rename rules, watermark logic, architecture, dependencies, operational procedures, and known source‑system behaviours. Partner closely with Application Development, Database Administration, Infrastructure, Security, Analytics, business teams, and external vendors to deliver complete solutions. Participate in architecture discussions, technical design reviews, code reviews, and the continued development of MBI’s data engineering standards. Provide technical guidance, share knowledge, and help strengthen engineering practices across the Data Services team. QualificationsPreferred Qualifications Hands‑on experience with Microsoft Fabric, including Data Factory pipelines, lakehouses, warehouses, notebooks, OneLake, SQL analytics endpoints, or Materialised Lake Views. Strong experience with Python, PySpark, Delta Lake, and scalable incremental‑processing patterns. Experience with Azure Data Factory, Azure Functions, Logic Apps, Workato, or another middleware/iPaaS platform. Experience implementing secure cloud‑to‑on‑premises connectivity using gateways, service principals, managed identities, or Azure Key Vault. Experience supporting ERP, CPQ, CRM, manufacturing, dealer, supply‑chain, or order‑to‑cash systems. Familiarity with Python testing and quality tools such as pytest, Ruff, pre‑commit, uv, or Poetry. Experience modernising legacy SQL, SSIS, linked‑server, or middleware‑based integrations. Experience operating data solutions in environments with formal security, privacy, governance, or audit requirements. Success in This Role The Senior Data Engineer is expected to operate with a high degree of technical independence while remaining collaborative, practical, and responsive to the needs of the business. This individual will take ownership beyond writing code—asking questions early, understanding the business process behind the data, identifying risks, documenting decisions, and ensuring solutions work reliably in production. MBI operates with a hands‑on, team‑oriented culture. The successful candidate will be comfortable working across technical and business boundaries, adapting as priorities evolve, and balancing immediate operational needs with the long‑term modernisation of MBI’s data platform. #MBICareers #MalibuBoats #J-18808-Ljbffr