Data Integration Engineer

Responsibilities and Skills Design and implement enterprise‑grade data pipelines using Snowflake, including ingestion and transformation. Strong in both Core and Semantic aspects. Develop complex SQL transformations, stored procedures and dynamic tables inside Snowflake to enable near real‑time and batch processing. Implement Snowflake data sharing and data marketplace integrations. Engineer Snowpipe and Kafka‑to‑Snowflake streaming ingestion pipelines to handle high throughput event data at scale. Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys. Architecture, design aspects, performance tuning, time travel, warehouse concepts – scaling, clustering, micro‑partitioning. Experience with SnowSQL and Snowpipe. Design and maintain end‑to‑end ETL/ELT pipelines using Apache Airflow. Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial. Thorough on data quality checks. Integrate AI/LLMs with data pipelines via Python UDFs or API callouts – enabling text analytics, semantic search, and GEN‑AI augmented workflows. Experience with Python‑based frameworks – scikit‑learn, PyTorch, TensorFlow. Experience with NLP and text‑mining techniques on unstructured data to identify actionable information. Time‑series forecasting, anomaly detection and propensity modeling. Experience with data visualization aspects. Hands‑on experience with writing complex queries using joins, self‑joins, views, materialized views, cursors, recursive queries, GROUP BY, PARTITION BY functions and SQL performance tuning. Hands‑on experience with ETL and dimensional data modelling – Slowly Changing Dimensions (SCD – Type 1, 2, 3). Good understanding of concepts like schema types, table types – fact‑dimension, dimension vs fact design considerations. Proficiency in Python scripting/programming – using Pandas, PyParsing, Airflow. Pandas, Tableau server modules, Numpy, Datetime, Apache Airflow related modules, APIs. Actively participating in discussions with business to understand requirements, perform thorough impact analysis, and provide suitable solutions. Salary Range: $80,000-$140,000 Per Year + Benefits #J-18808-Ljbffr