Data Integration Engineer

Key Qualifications Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation Expertise in both core and semantic aspects of Snowflake Develop complex SQL transformations, stored procedures and dynamic tables to enable near real-time and batch processing Implement Snowflake data sharing and data marketplace integrations Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines, handling high-throughput event data at scale Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys Architectural design, performance tuning, time travel, warehouse concepts – scaling, clustering, micro-partitioning Experience with SnowSQL and Snowpipe Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow Build reusable, parameterized data ingestion pipelines/frameworks Perform thorough data quality checks Integrate AI/LLMs with data pipelines via Python UDFs or API callouts, enabling text analytics, semantic search and GEN‑AI augmented workflows Experience with Python machine learning frameworks (scikit‑learn, PyTorch, TensorFlow) Apply NLP and text‑mining techniques on unstructured data to identify actionable information Time‑series forecasting, anomaly detection and propensity modeling Data visualization using relevant tools Complex SQL query development – joins, self joins, views, materialized views, cursors (also recursive), GROUP BY, PARTITION BY, and SQL performance tuning ETL and dimensional data modeling – slowly changing dimensions (SCD Types 1, 2, 3) Strong understanding of schema types, table types (fact/dimension), and design considerations for dimensions versus facts Proficiency in Python scripting and programming – Pandas, Numpy, PyParsing, Apache Airflow, and related APIs Experience with Tableau Server modules and related data visualization platforms Automate data pipelines and maintain high-quality code Strong Python programming skills Active participation in business discussions to understand requirements, perform impact analysis, and provide suitable solutions Location: Sunnyvale, CA Salary range: $80,000-$140,000 a Year. #J-18808-Ljbffr