Snowflake Data Engineer

Senior Data Engineer- Responsibilities:

• Design, develop, and optimize scalable data pipelines and ETL processes utilizing Snowflake and related cloud technologies.
• Develop and implement efficient data ingestion methods, integrating various structured and semi-structured data sources.
• Create and maintain data warehouse schemas, tables, views, and materialized views in Snowflake.
• Ensure data quality, data integrity, and data governance standards are maintained throughout ETL processes.
• Collaborate closely with data analysts, data architect, and business stakeholders to understand and fulfill data requirements.
• Perform regular performance tuning and optimization of Snowflake queries and warehouse operations.
• Develop and enforce best practices for data modeling, ETL architecture, data warehousing, and data security.
• Proactively identify, troubleshoot, and resolve data issues and pipeline failures.
• Document technical processes, system architecture, and data lineage clearly and comprehensively.

Charter Data Engineer: Overview: Project is related to customer intelligence, has data coming in from different vendors (Google Analytics, HootSuite, Known), different file formats (API, CSV), transforming and ingesting into S3 buckets, from there into Snowflake. Different layers in Snowflake: staging, cleansed, reporting; need to move and curate data with business logic in the cleansing layer. Leveraging different technologies: Python, Snowflake, Airflow orchestration and DAG creation, Gitlab source control, Flyway for version control (don't really need expertise in it, but developers with experience in source/version control and CI/CD) Python to ingest data and build out Airflow-related code, performing transformations by passing parameters. Creating tasks within DAG, DAGs trigger Gitlab, Flyway runs version control from Gitlab, implements schema changes. Using Snowflake procedures, SQL stored procedures for smaller volumes of data. Still solutioning large volumes of data--may end up being a Python/Spark approach. Required Skills : Requirements: • Python • SQL • ETL • Stored procedures • Snowflake (can be basic + upskilling, acquisition of badges ahead of joining) • Job orchestration i.e. Airflow (Airflow itself is nice to have) • Version and Source Control experience Nice to Have: • Airflow • Gitlab • Flyway Basic Qualification : Additional Skills : Background Check : Yes Drug Screen : Yes Notes : Selling points for candidate : Project Verification Info :"The information provided below is for Apex Systems AV use only and is not to be distributed publicly, or to any third party. Any distribution of the below information will result in corrective action from Apex Systems Vendor Management. MSA: Blanket Approval Received Client Letter: Will Provide" Exclusive to Apex :No Face to face interview required :No Candidate must be local :No Candidate must be authorized to work without sponsorship ::No Interview times set : :No Type of project : Master Job Title : Branch Code : My3Tech Inc