Data Engineer

Optomi in partnership with a leading professional services firm, is seeking a highly skilled Senior Data Engineer with deep expertise in AWS, Python, PySpark, and SQL. The ideal candidate will have experience building scalable ETL/ELT solutions, designing data models utilizing SCD Type 1 and Type 2 methodologies, and developing API-driven solutions using Python frameworks such as Fast API or Flask. This engineer will work closely with cross-functional teams to design, deploy, and maintain cloud-native applications and data platforms while ensuring reliability, scalability, and operational excellence. What the Right Candidate Will Enjoy: Building scalable cloud-native solutions within AWS. Working extensively with Python, PySpark, and modern data engineering technologies. Designing and deploying APIs that support enterprise applications. Leveraging AI-powered development tools such as GitHub Copilot. Collaborating with highly technical engineering and data teams. Solving complex data challenges within a modern cloud environment. What Type of Experience Does the Right Candidate Have: 6+ years of Data Engineering experience. Strong experience with AWS services including ECS, Lambda, SNS, and DynamoDB. Advanced Python programming skills. Hands-on experience with PySpark and distributed data processing. Strong SQL development and optimization experience. Deep understanding of data modeling concepts, including SCD Type 1 and Type 2. Experience building and deploying APIs using FastAPI, Flask, or similar Python frameworks. Experience implementing CI/CD processes and deployment automation. Experience working with AI-assisted development tools such as GitHub Copilot. Strong troubleshooting, analytical, and problem-solving abilities. What the Responsibilities Are of the Right Candidate: Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark. Investigate, troubleshoot, and optimize complex data pipelines. Develop and optimize SQL queries supporting enterprise reporting and analytics. Design and implement data models utilizing SCD Type 1 and Type 2 methodologies. Build, deploy, and support APIs within AWS environments. Develop solutions utilizing AWS services such as ECS, Lambda, SNS, and DynamoDB. Implement and maintain CI/CD pipelines to support automated deployments. Utilize AI-powered development tools to enhance development efficiency and code quality. Collaborate with cross-functional engineering teams to deliver scalable cloud-based solutions. Monitor, support, and continuously improve platform performance and reliability. #J-18808-Ljbffr Optomi