Data Engineer

Job Description Experience 2-4 Years We are seeking a talented Job Summary We are seeking a talented Data Engineer with 2-4 years of experience in building and maintaining scalable data pipelines on the AWS Cloud using Snowflake. The ideal candidate should have hands-on experience with AWS data services, ETL/ELT development, cloud data warehousing, and data integration. The candidate will be responsible for designing, developing, and optimizing data pipelines that support business intelligence, analytics, and reporting initiatives. Job Title: Data Engineer (Snowflake & AWS) Key Responsibilities Design, develop, and maintain scalable ETL/ELT data pipelines using AWS services and Snowflake. Build robust data ingestion frameworks to load data from multiple sources, including relational databases, APIs, flat files, and cloud storage. Develop and optimize data models, schemas, and database objects in Snowflake. Implement data transformation logic using SQL, Snowflake Stored Procedures, and Snowpark (preferred). Load structured and semi-structured data (CSV, JSON, XML, Parquet, Avro) into Snowflake. Develop batch and incremental data processing pipelines using AWS services. Optimize Snowflake performance through clustering, query tuning, partitioning, and warehouse optimization. Integrate data from AWS S3 into Snowflake using Snowpipe, COPY INTO, or external stages. Monitor, troubleshoot, and resolve ETL failures and performance bottlenecks. Implement data quality checks, validation, and reconciliation processes. Collaborate with business analysts, data architects, and application teams to deliver high-quality data solutions. Follow coding standards, documentation practices, and data governance policies. Participate in Agile ceremonies, code reviews, and production support activities. Required Technical Skills Snowflake Strong hands-on experience with Snowflake Cloud Data Platform. Experience with Snowflake architecture, virtual warehouses, databases, schemas, stages, and file formats. Strong SQL development skills with query optimization techniques. Experience with Snowflake Tasks, Streams, Stored Procedures, Secure Views, and Time Travel. Experience using Snowpipe and COPY INTO commands for automated data ingestion. Knowledge of data sharing and role-based access control (RBAC). AWS Data Engineering Amazon S3 AWS Glue AWS Lambda Amazon Redshift (good to have) AWS IAM Amazon CloudWatch AWS Secrets Manager Amazon EventBridge or Amazon SNS/SQS (preferred) ETL/ELT Design and development of ETL/ELT workflows. Data migration from on-premises systems to AWS Cloud. Experience with incremental loads and Change Data Capture (CDC). Experience scheduling and orchestrating workflows using AWS Glue Workflows or Apache Airflow (preferred). Databases SQL Server Oracle PostgreSQL MySQL Programming Languages SQL (Mandatory) Python (Mandatory) PySpark (Preferred) Version Control & CI/CD Git Jenkins, GitHub Actions, or AWS CodePipeline AWS CodeCommit (preferred) Data Formats CSV JSON XML Parquet Avro Required Qualifications Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. 2-4 years of experience in Data Engineering. Hands-on experience with Snowflake and AWS Cloud services. Strong SQL and Python programming skills. Experience developing cloud-based ETL pipelines. Good understanding of data warehousing concepts and dimensional modeling. Familiarity with Agile/Scrum development methodologies. Preferred Skills Experience with Apache Airflow or Managed Workflows for Apache Airflow (MWAA). Experience with dbt (Data Build Tool). Exposure to Apache Spark or PySpark. Knowledge of Amazon Redshift. Experience integrating REST APIs. Familiarity with Power BI, Tableau, or Amazon QuickSight. Experience implementing CI/CD pipelines for data engineering solutions. Soft Skills Strong analytical and problem-solving skills. Excellent verbal and written communication skills. Ability to work independently and within cross-functional teams. Strong attention to detail and commitment to delivering high-quality solutions. Ability to manage multiple priorities in a fast-paced environment. Nice to Have SnowPro Core Certification. AWS Certified Data Engineer – Associate. AWS Certified Solutions Architect – Associate. Experience with Terraform or AWS CloudFormation. Exposure to Kafka or Amazon Kinesis for streaming data. Knowledge of DataOps and data governance best practices. Key Competencies Snowflake AWS Glue Amazon S3 AWS Lambda SQL Python ETL/ELT Development Snowpipe Data Warehousing Data Modeling Performance Optimization Git Jenkins AWS IAM CI/CD Cloud Data Engineering Data Quality Agile Development Qualifications Bachelor or equivalent Range Of Year Experience-Min Year 2 Range Of Year Experience-Max Year 4 #J-18808-Ljbffr