Senior Data Engineer

Senior Data Engineer

We are looking for a Senior Data Engineer with 12+ years of experience in designing and building enterprise-scale data platforms. The ideal candidate will have deep expertise in Snowflake, DBT, Python, PySpark, Qlik Replicate, and Airflow, with strong experience in data ingestion, CDC, schema evolution, data governance, and CI/CD automation.

Must Have Technical/Functional Skills:

• 12+ years of hands-on Data Engineering experience.
• Expert knowledge of Snowflake, including Snowpipe, Tasks, Streams, Dynamic Tables, RBAC, Data Masking, performance tuning, and advanced SQL.
• Strong experience with DBT Cloud for data transformation, testing, lineage, and documentation.
• Excellent proficiency in Python and PySpark for ingestion frameworks and automation.
• Hands-on experience with Qlik Replicate for CDC and full-load replication.
• Experience with Astronomer Airflow for workflow orchestration and scheduling.
• Strong understanding of schema drift detection and schema evolution strategies.
• Experience with GitLab, branching strategies, merge requests, and CI/CD pipelines.
• Familiarity with Protegrity or similar data tokenization/security platforms.
• Experience working with structured and semi-structured data formats including CSV, JSON, XML, Excel, and fixed-width files.

Roles & Responsibilities:

• Design and develop scalable data pipelines using Snowflake technologies such as Snowpipe, Tasks, Streams, and Dynamic Tables.
• Build and maintain DBT models with testing, documentation, and lineage.
• Develop Python/PySpark ingestion frameworks for APIs and file-based data sources.
• Design ingestion solutions for CSV, JSON, XML, Excel, fixed-width, and semi-structured data.
• Implement Mainframe VSAM ingestion for EBCDIC data formats.
• Detect and manage schema drift using metadata-driven schema evolution techniques.
• Configure and manage Qlik Replicate for CDC and full-load replication from Oracle, SQL Server, and DB2.
• Implement Snowflake data masking, RBAC, and governance policies.
• Apply Protegrity tokenization to secure sensitive enterprise data.
• Build and schedule workflows using Astronomer Airflow.
• Integrate data pipelines with GitLab and Azure DevOps CI/CD processes.
• Monitor pipeline health, optimize Snowflake performance, and improve scalability and reliability.
• Collaborate with cross-functional teams to deliver secure, high-quality data solutions.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.