Senior Data Engineer

Senior Data EngineerWe are seeking a highly skilled Senior Data Engineer with 8–12+ years of experience in designing and building enterprise-scale data platforms. The ideal candidate will have deep expertise in Snowflake, DBT, Python, PySpark, Qlik Replicate, Astronomer Airflow, and CI/CD to develop scalable, secure, and high-performance data pipelines supporting modern analytics and business intelligence initiatives.The successful candidate will play a key role in architecting data ingestion frameworks, implementing data governance and security, managing schema evolution, and optimizing cloud-based data platforms.Key ResponsibilitiesDesign, develop, and maintain scalable, resilient data pipelines using Snowflake features, including Snowpipe, Streams, Tasks, Dynamic Tables, and advanced SQL.Build and maintain DBT models with testing, documentation, lineage, and reusable transformations.Develop Python and PySpark-based ingestion frameworks for files, APIs, and enterprise data sources.Build ingestion pipelines for structured and semi-structured data formats, including CSV, JSON, XML, Excel, fixed-width files, and multi-record layouts.Design ingestion solutions for Mainframe VSAM/EBCDIC data sources.Implement metadata-driven schema evolution and schema drift detection strategies.Configure and manage Qlik Replicate for Change Data Capture (CDC) and database replication from Oracle, SQL Server, and DB2.Develop robust, auditable, and recoverable data replication pipelines.Implement Snowflake data governance using RBAC, masking policies, and security best practices.Apply Protegrity (or similar tokenization platforms) for sensitive data protection.Develop and orchestrate workflows using Astronomer Airflow with dependency management, scheduling, retries, and monitoring.Integrate data pipelines with GitLab and Azure DevOps CI/CD pipelines.Optimize Snowflake performance, compute resources, storage utilization, and query execution.Implement monitoring, alerting, logging, and operational dashboards for data pipelines.Collaborate with Data Architects, Analysts, Data Scientists, and business stakeholders to deliver reliable data solutions.Required Skills8–12+ years of experience in Data Engineering.Strong expertise in Snowflake architecture and development.Hands-on experience with: Snowpipe, Streams, Tasks, Dynamic Tables, Advanced SQL, Data Masking, RBAC, Performance Tuning.Strong experience with DBT (Data Build Tool).Proficiency in Python and PySpark.Experience with Qlik Replicate for CDC and database replication.Hands-on experience with Astronomer Airflow workflow orchestration.Experience implementing schema drift detection and schema evolution.Knowledge of GitLab, Git workflows, and CI/CD automation.Experience with REST APIs and enterprise data integration.Strong understanding of data governance, metadata management, and data quality.Excellent analytical, troubleshooting, and problem-solving skills.Strong communication and collaboration skills.Preferred QualificationsExperience with Protegrity or similar data security/tokenization platforms.Experience working with Oracle, SQL Server, and DB2 databases.Knowledge of Azure DevOps and cloud-native data engineering practices.Experience with enterprise-scale data lake and data warehouse implementations.Exposure to Agile/Scrum development methodologies.Experience Required8–12+ years of hands-on Data Engineering experience.