Senior Data Engineer Azure & Microsoft Fabric

Job DescriptionJob DescriptionTitle: Sr. Data Engineer(Azure/Fabric) Location: Deerfield Beach, FL (Hybrid 3 days onsite, 2 days remote) Duration: Long term contract with the potential to goperm in the future, but no guarantee, Compensation: $75/hr -$80/hr Work Requirements: U.S. Citizen, GC Holders, or Authorized to Work in the U.S. Senior Data Engineer – Azure & Microsoft FabricThe Senior Data Engineer designs, builds, tests, and supports the data pipelines and data productsbehind our analytics, reporting, and machine learning work on Azure and Microsoft Fabric. The rolefocuses on data engineering: ingesting, transforming, and modeling structured and unstructured datafrom sources such as Azure Synapse, Azure SQL, SQL Server, and ADLS Gen2. That data lands inFabric Lakehouse and Warehouse on OneLake through Azure Data Factory, Fabric Data Factorypipelines, Dataflow Gen2, and PySpark notebooks. The engineer will also design and run machinelearning (ML) pipelines with the Data Science team, automate builds and releases through Azure DevOpspipelines, and set up supporting Azure resources with Bicep templates. They will apply secure datapractices (Managed Identity, Microsoft Entra ID, Key Vault, OneLake security, and Microsoft Purview) andbuild data quality, lineage, and governance into every pipeline. This role works closely with DataScientists, Solution Architects, Software Engineers, DevOps Engineers, QA Engineers, Project Managers,Information Security Engineers, and Delivery Managers.What You Will Be DoingTech Breakdown (Coding and Collaboration) 10% Data Exploration and Design (Source Analysis, Data Modeling, Security, Quality,Performance) 40% ETL/ELT Pipeline Development (Azure Data Factory, Fabric Data Factory, Dataflow Gen2,Fabric Notebooks, Unit Testing) 20% ML Pipeline Design and Development (Data Preparation, Feature Engineering, Training andBatch Scoring Pipelines, MLflow) 20% DevOps and Infrastructure as Code (Azure DevOps CI/CD Pipelines, Bicep Templates,Fabric Workspace Promotion, Release Management) 10% Data Operations and Governance (Monitoring, Purview Lineage, Access Control, Cost andPerformance Optimization)Key Responsibilities:• Design, build, and maintain scalable ETL/ELT pipelines using Azure Data Factory and MicrosoftFabric Data Factory (pipelines, Copy activity, Dataflow Gen2). These pipelines pull data fromAzure Synapse, Azure SQL, SQL Server, APIs, and file-based sources.• Build a medallion architecture (bronze, silver, gold) in Fabric Lakehouse and Warehouse onOneLake using Delta Parquet tables, with shortcuts and Mirroring where they fit.• Write data transformations in PySpark, Spark SQL, and T-SQL in Fabric Notebooks, and buildreusable, parameterized, metadata-driven pipeline frameworks.• Replace legacy SAS and file-based ETL processes (CSV exports, file shares) with governed ADFand Fabric pipelines that include alerting, retries, and the ability to re-run.• Design and develop ML pipelines with the Data Science team: data preparation, featureengineering, model training, and batch scoring (Fabric PREDICT). Publish model outputs toLakehouse tables for Power BI (Direct Lake) reporting.• Manage the model lifecycle with MLflow experiment tracking and the model registry, and helpmove Azure Machine Learning workloads to Fabric Data Science.• Build and maintain Azure DevOps CI/CD pipelines (YAML) for ADF, Fabric items, notebooks,SQL objects, and ML assets. This includes Git integration, branching strategy, pull-requestvalidation, and promotion between Dev and Prod workspaces.• Write and maintain Bicep templates that set up supporting Azure resources the same way inevery environment (Data Factory, ADLS Gen2, Key Vault, managed identities, private endpoints,and networking).• Build dimensional (star schema) data models and Fabric semantic models for analytics andreporting.• Apply data security and compliance controls: Managed Identity and Entra ID authentication, KeyVault secrets, OneLake and workspace roles, row- and column-level security, data masking, andsensitivity labels.• Build data quality checks, lineage, and cataloging (Microsoft Purview) into every data-movementprocess.• Monitor, troubleshoot, and tune pipelines, Spark jobs, and queries for performance and capacitycost using the Fabric Monitoring Hub and Azure Monitor.• Work with stakeholders to turn business requirements into technical specifications, and keepdesign specifications and component documents up to date.• Keep up with the Azure and Microsoft Fabric roadmap and recommend generally availablefeatures and best practices to adopt.Qualifications And Requirements:• Must be a team player with a get-it-done attitude• 15+ years of software development with relational databases, including 5+ years of dataengineering on Azure• Must have developed and deployed at least 15 Azure Data Factory or Fabric Data Factorypipelines to production following SDLC processes• Expert SQL/T-SQL skills and strong Python and PySpark skills; demonstrable debugging skillsin MS SQL• Hands-on experience with Microsoft Fabric: Lakehouse, Warehouse, Data Factory pipelines,Dataflow Gen2, Notebooks, OneLake, and semantic models• Demonstrable experience with Azure data services: Azure Data Factory, Azure Synapse, AzureSQL, and ADLS Gen2• Experience building ML pipelines and supporting data scientists (feature engineering,training and scoring pipelines, MLflow, Azure Machine Learning, or Fabric Data Science)• Experience with Azure DevOps (Repos, YAML Pipelines, Boards) and CI/CD for data solutions• Experience writing infrastructure as code with Bicep (ARM experience; Terraform is a plus)• Strong knowledge of data modeling, medallion architecture, Delta Lake, and relational andnon-relational databases• Knowledge of data security and governance: Entra ID, Managed Identity, Key Vault, MicrosoftPurview, and private networking• Follow best practices and coding standards (e.g., code reviews, logging and instrumentation,static/dynamic code analysis, code coverage, unit and integration tests)• Work with Product Managers, Solution Architects, DevOps, and Information Security teams tounderstand data requirements in an Agile environment• Eager to share knowledge with teammates in all forums and eager to learn from them• Strong initiative to find ways to improve solutions, systems, and processes• Bachelor's degree in computer science/engineering or a related technical field, or equivalentexperienceMust have skills:• Microsoft Fabric (Lakehouse, Warehouse, Data Factory, Notebooks, OneLake)• Azure Data Factory and ETL/ELT design• SQL/T-SQL, Python, PySpark• ML pipeline concepts (feature engineering, training, batch scoring, MLflow)• Azure DevOps CI/CD pipelines• Bicep / Infrastructure as Code• Data Lake / Lakehouse and medallion architectureNice to have skills:• Azure Synapse Analytics and Azure Databricks• Microsoft Purview• Power BI semantic models and Direct Lake• Fabric Real-Time Intelligence (Eventstream, KQL)• GitHub Enterprise and fabric-cicd• Experience migrating SAS workloads1 or more certifications:• Microsoft Certified: Fabric Data Engineer Associate (DP-700)• Microsoft Certified: Fabric Analytics Engineer Associate (DP-600)• Azure Data Scientist Associate (DP-100)• Azure DevOps Engineer Expert (AZ-400)• Azure Data Engineer Associate (DP-203, retired but still recognized About INSPYR Solutions Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, project, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com.