ETL AWS Data engineer

AWS Data Engineer

AWS Data Engineer who specializes in building and optimizing data pipelines and integration solutions within the AWS ecosystem, with particular expertise in AWS Glue and Workato. Deployment is immediate. We are looking for some good offshore resources who can start in a week or 2. Should clear client evaluations too.

Location: Offshore.

Rate: 8000

1. Design, develop, and maintain robust and scalable ETL/ELT data pipelines using AWS Glue (including Glue Studio, Crawlers, Jobs - Spark/Python Shell, Workflows). Should be able to Ingest, transform, and load data from diverse sources.
2. Optimize AWS Glue jobs for performance and cost-efficiency, leveraging best practices for Spark, partitioning, and data formats (Parquet, ORC). Manage and extend the AWS Glue Data Catalog for metadata management and schema discovery.
3. Design, build, and maintain automated workflows and integrations using the Workato iPaaS platform.
4. Connect Workato recipes with a wide array of AWS services and Implement real-time data synchronization and event-driven automation between systems via Workato.
5. Utilize other relevant AWS services to support data pipelines and integration patterns, including: Amazon S3, Lambda, CloudWatch, API Gateway, SQS, SNS, and IAM.
6. Should be able to Implement data quality checks, validation rules, and error handling mechanisms within data pipelines.
7. Should be excellent in analytical and troubleshooting skills to resolve complex data and integration challenges.
8. Strong verbal and written communication skills, with the ability to explain technical concepts clearly to diverse audiences.
9. Should be a standalone contributor willing to work collaboratively with client in some overlap timing.
10. Some understanding in Snowflake is added advantage