Data Engineer

Overview : We are seeking an experienced Engineer with strong programming and cloud platform skills. The ideal candidate will have hands-on experience with a variety of AWS services, a solid background in Python development, SQL, and data modeling, and the ability to leverage AI tools such as GitHub Copilot to enhance productivity. Key Responsibilities: Write efficient, maintainable code in Python/Pyspark, leveraging frameworks like PySpark for data processing. Investigate and troubleshoot data pipelines using PySpark, with a strong understanding of data structures and processing. Develop and optimize SQL queries; understand and implement Slowly Changing Dimensions (SCD) Type 1 and Type 2 in data warehousing. Design, develop, and deploy APIs on AWS, ensuring scalability and reliability. Work extensively with AWS services such as ECS, Lambda, SNS, and DynamoDB. Build, test, and deploy applications using CI/CD pipelines, ensuring smooth and automated delivery. Utilize AI-powered development tools (e.g., GitHub Copilot) to accelerate coding and improve code quality. Collaborate with cross-functional teams to deliver robust cloud solutions. Troubleshoot and resolve issues related to cloud infrastructure and application performance. Required Skills: Strong knowledge of the AWS platform and its core services (ECS, Lambda, SNS, DynamoDB). Proficient in Python programming; experience with PySpark is highly desirable. Solid understanding of SQL and data modeling concepts, including SCD Type 1 and Type 2. Experience in building and deploying APIs (FastAPI/REST/RESTful/Flask) on AWS /(ECS). Good understanding of CI/CD processes and tools. Ability to work with AI-powered development tools such as GitHub Copilot. #J-18808-Ljbffr