Senior Data Engineer

Overview We are seeking a seasoned Databricks Data Engineer with deep expertise in Azure cloud services and modern data platforms. This role will focus on designing and optimizing large-scale data pipelines, modernizing cloud-based data ecosystems, and enabling secure, governed data solutions in a highly collaborative environment. This is an opportunity to work on cutting‑edge data architecture while partnering closely with cross‑functional teams to deliver scalable, high‑performance data solutions. Key Responsibilities Design, build, and optimize large-scale data pipelines on the Databricks Lakehouse platform, ensuring reliability, scalability, and governance Modernize cloud-based data ecosystems, contributing to architecture, distributed data engineering, modeling, security, and CI/CD automation Develop and maintain ETL/ELT pipelines using Python, PySpark, and Spark SQL Leverage orchestration tools such as Apache Airflow for workflow automation Design and optimize Delta Lake data models for performance and scalability Implement and manage Unity Catalog for RBAC, lineage, and secure data sharing Build reusable frameworks using Databricks Workflows, Repos, and Delta Live Tables Develop scalable ingestion pipelines for APIs, databases, files, streaming sources, and MDM systems Automate API integrations and workflows using Python and REST APIs Support data governance, cataloging, lineage, and metadata management initiatives Enable downstream consumption for BI, analytics, and application use cases Write and optimize SQL/T‑SQL queries, stored procedures, and curated datasets Automate deployments, testing pipelines, and DevOps workflows Required Qualifications 8+ years of experience designing and developing scalable data pipelines in modern data environments Strong expertise in data engineering and data warehousing concepts Advanced proficiency in SQL, Python, and ETL/ELT frameworks, including performance tuning Hands‑on experience with Azure and Databricks (Lakehouse architecture, Delta Lake, Unity Catalog) Experience working with enterprise‑scale data systems and integrations Proven ability to lead and collaborate across cross‑functional teams Preferred Experience Experience with Snowflake or other modern cloud data platforms Background working with regulated or highly governed datasets Familiarity with CI/CD, DevOps practices, and data pipeline automation What’s Offered Competitive compensation and comprehensive benefits package 401(k) with employer contribution Health, dental, and vision insurance Collaborative, high‑performing team environment Opportunities for professional development and career growth Tuition reimbursement for eligible programs Collaborative, product‑focused team Emphasis on clean architecture, scalability, and continuous improvement Opportunity to make a direct impact on modern data platform initiatives If you're passionate about building scalable data systems and working with modern cloud technologies, we’d love to connect. #J-18808-Ljbffr Interactive Resources LLC