AWS Data Engineer - Open Data Platform

Title: AWS Data Engineer - Open Data Platform Location: New York, NY Work Arrangement: Remote Ok, preferred NY Locals Duration: 6-Month Contract Experience Range: 8-10 years Our client is seeking an experienced AWS Data Engineer to design and build cloud-native data pipelines on an open data platform, architecting Apache Iceberg tables and optimizing large-scale data processing across AWS and Snowflake environments. Responsibilities & Qualifications Design and develop end-to-end data ingestion, transformation, and processing pipelines using AWS Glue, Apache Spark, and Amazon S3 Build and manage Apache Iceberg tables to enable open, interoperable access across multiple compute engines Configure and integrate Apache Polaris with Iceberg REST Catalog for AWS and Snowflake environments Implement table maintenance, schema evolution, and partitioning strategies to optimize performance and cost Support platform security, access controls, and AWS networking infrastructure Conduct workload benchmarking and performance analysis comparing AWS Glue versus Snowflake for cost and efficiency Provide operational monitoring, troubleshooting, and optimization of cloud-based data platform infrastructure Requirements 8-10 years of experience in data engineering and cloud-based data pipeline development Proven expertise with Amazon S3, AWS Glue, Apache Spark, and large-scale data processing Strong hands-on experience with Apache Iceberg architecture, including table design and maintenance Demonstrated proficiency with Apache Iceberg schema evolution and partitioning optimization Experience configuring and deploying Apache Polaris and Iceberg REST Catalog Solid understanding of AWS IAM, networking, and security best practices in cloud environments Experience with performance tuning, cost optimization, and cross-platform data tool evaluation RIT Solutions, Inc.