Senior Data Engineer

About the Company We're looking for a Senior Data Engineer to design, build, and operate modern data platforms at scale. You'll work across Azure, Microsoft Fabric, and AWS — building reliable pipelines, performant data models, and self-service analytics that drive real business decisions. A core part of this role is containerized workload execution: designing and deploying pipeline jobs using Azure Container Apps Jobs for scheduled and event-driven processing. This is a hands-on role for an engineer who thrives in ambiguity, values clean architecture, and moves comfortably between strategy and implementation. About the Role Data Platform & Architecture Design and build scalable data pipelines using Python and cloud-native orchestration tools (Azure Data Factory, Azure Container Apps Jobs, Fabric Data Pipelines) Architect solutions across Microsoft Fabric Warehouses, Azure SQL, and AWS (S3, Redshift), selecting the right tool for the workload Implement Medallion/layered architecture (Bronze → Silver → Gold) for structured, governed data delivery Manage and optimize large-scale data warehouse environments for performance, cost, and maintainability Responsibilities Develop Python-based ETL/ELT pipelines ingesting data from APIs, flat files, databases, and SaaS platforms Build and deploy containerized jobs using Azure Container Apps Jobs, including scheduling, scaling rules, secrets management (Key Vault), and Container Registry integration Maintain data movement between on-premises SQL Server and cloud targets Design idempotent, fault-tolerant pipeline patterns with robust logging, alerting, and retry logic Partner with analytics teams to deliver clean, well-documented data models for Power BI and similar BI tools Qualifications 5–8 years of hands-on data engineering in production environments Proven track record delivering data platforms on Azure and/or AWS Experience migrating legacy on-premises infrastructure to cloud-native solutions Hands-on experience with Azure Container Apps Jobs or a comparable containerized job execution platform Required Skills Python — production-grade pipelines, transformations, and automation SQL — advanced T-SQL and ANSI SQL; SQL Server, Azure SQL, Redshift, and Fabric query engines Microsoft Fabric — Warehouses, Lakehouses, Data Pipelines, OneLake Azure — ADF, Azure SQL, ADLS, Key Vault, Container Apps Jobs, Container Registry Containerization — Docker image development, container registry management, Container Apps Jobs (scheduling, scaling, secret injection) AWS — S3 for data lake storage, Redshift for cloud warehousing Data Modeling — dimensional modeling, star/snowflake schema, ER modeling for OLTP and OLAP DevOps — Git, GitHub, PR workflows, CI/CD pipelines Visualization — Power BI, Tableau Preferred Skills Microsoft Fabric in production (Fabric Warehouses, OneLake integration) dbt or similar transformation frameworks Power BI semantic model development, dataset optimization, DirectQuery/Import tradeoffs Agile/Scrum experience Cloud certifications: DP-203 (Azure Data Engineer), AWS Certified Data Analytics, or equivalent Bachelor's degree in Computer Science, Information Systems, Data Science, or a related field — or equivalent professional experience