AWS Data Engineer

AWS Data EngineerDuration-Contract Location- Charlotte, NC /HybridMust Have:Strong experience with AWS services on big data platforms.Strong Python experience in building Dash application in AWSSparkShell scripting and Linux knowledgePrimary SkillsAWS Cloud Formation, ETL, PythonThe person on this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.Data Ingestion: Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.Data Transformation: Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.Data Storage: Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.Data Processing: Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.Data Monitoring and Optimization: Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.Integration: Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.Documentation: Maintain thorough documentation of data engineering processes, data flows, and system configurations.Scalability: Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.Cost Management: Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.Troubleshooting: Diagnose and resolve AWS-related issues to minimize downtime and disruptions.