Data Engineer (Snowflake, AWS Glue Specialist)

Data Engineer (Snowflake, AWS Glue Specialist)

Location: Fort Mill, SC - 100% ONSITE

Job Description: Local Grade C2 Location Fortmill SC Must Have Skills Fivetran HVR AWS Glue Lambda Airflow Snowflake DBT Hiring Manager Johari Abhishek ajohari JD Responsible for designing developing and maintaining scalable data pipelines and ETL processes using AWS Glue. This role involves working closely with data engineers analysts and other IT professionals to ensure data is efficiently integrated transformed and made available for business use.

1. Snowflake Snowpark: The candidate should have a deep understanding of Snowflake data warehousing platform and be proficient in using Snowpark for data processing and analytics.

2. DBT: Experience with DBT Data Build Tool for modeling data and creating data transformation pipelines is a plus.

3. AWS services Airflow: The candidate should have hands-on experience with AWS services particularly Apache Airflow for orchestrating complex data workflows and pipelines.

4. AWS services Lambda: Proficiency in AWS Lambda for serverless computing and event-driven architecture is essential for this role.

5. AWS services Glue: The candidate should be well-versed in AWS Glue for ETL Extract Transform Load processes and data integration.

6. Fivetran HVR: Working knowledge and hands-on experience on Fivetran HVR.

7. Python: Strong programming skills in Python are required for developing data pipelines data transformations and automation tasks.

Responsibilities:

• Design develop and maintain data pipelines and ETL processes using Snowflake AWS services Python and DBT.
• Collaborate with data scientists and analysts to understand data requirements and implement solutions.
• Optimize data workflows for performance scalability and reliability.
• Troubleshoot and resolve data-related issues in a timely manner.
• Stay updated on the latest technologies and best practices in data engineering.
• Design and implement scalable ETL solutions using AWS Glue.
• Collaborate with data engineers and analysts to understand data requirements.
• Develop and manage data extraction transformation and loading processes.
• Optimize and improve existing data pipelines and ETL workflows.
• Ensure data quality and integrity during the transformation process.
• Monitor and troubleshoot issues with ETL jobs.
• Maintain detailed documentation of data workflows and processes.
• Stay updated with the latest AWS services and tools.

Qualifications:

• Bachelor’s degree in Computer Science Engineering or related field.
• Proven experience in data engineering roles with a focus on Snowflake AWS services Python and DBT.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork abilities.
• Technical certifications are a plus.
• Bachelor’s degree in Computer Science Information Technology or a related field.
• 3 years of experience working with AWS Glue and other AWS data services.
• Proven experience in designing and maintaining ETL processes.
• Strong knowledge of SQL and database management.
• Familiarity with data warehousing concepts and tools.
• Experience with Python or other scripting languages.
• Excellent problem-solving and analytical skills.
• Strong communication and teamwork abilities.

If you meet the above requirements and are passionate about data engineering and analytics we encourage you to apply for this exciting opportunity.