AWS Data Engineer

AWS Data Engineer

Duration-Contract Location- Charlotte, NC /Hybrid

Must Have:

• Strong experience with AWS services on big data platforms.
• Strong Python experience in building Dash application in AWS
• Spark
• Shell scripting and Linux knowledge

Primary Skills

• AWS Cloud Formation, ETL, Python
• The person on this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics systems.
• Data Ingestion: Implement data ingestion processes to collect data from various sources, including databases, streaming data, and external APIs.
• Data Transformation: Develop ETL (Extract, Transform, Load) processes to transform and cleanse raw data into a structured and usable format for analysis.
• Data Storage: Manage and optimize data storage solutions, including Amazon S3, Redshift, and other AWS storage services.
• Data Processing: Utilize AWS services like AWS Glue, Amazon EMR, and AWS Lambda to process and analyze large datasets.
• Data Monitoring and Optimization: Continuously monitor and optimize data pipelines and infrastructure for performance, cost-efficiency, and scalability.
• Integration: Collaborate with data scientists, analysts, and other stakeholders to integrate AWS-based solutions into data analytics and reporting platforms.
• Documentation: Maintain thorough documentation of data engineering processes, data flows, and system configurations.
• Scalability: Design AWS-based solutions that can scale to accommodate growing data volumes and changing business requirements.
• Cost Management: Implement cost-effective solutions by optimizing resource usage and recommending cost-saving measures.
• Troubleshooting: Diagnose and resolve AWS-related issues to minimize downtime and disruptions.