Data Engineer

Job Description

Job Description

Job Summary We are looking for an experienced Data Engineer with strong expertise in Python, PySpark, SQL, and AWS cloud technologies. The ideal candidate will have hands-on experience building and maintaining scalable data pipelines, processing large datasets, and developing reliable ETL solutions. Experience in the healthcare insurance domain, along with strong problem-solving and collaboration skills, is highly desirable. Key Responsibilities Design, develop, and maintain scalable data pipelines using Python and PySpark. Process and transform large datasets using SQL and distributed data processing technologies. Build, maintain, and optimize ETL workflows in AWS. Work with Amazon S3, Redshift, Lambda, EMR, AWS Glue, and Airflow. Develop and maintain Bash/Shell scripts for data processing and automation. Implement data transformation, cleansing, and validation logic. Troubleshoot pipeline failures and resolve data processing issues. Optimize Spark jobs and SQL queries for performance and scalability. Support production data workflows, monitoring, and issue resolution. Collaborate with data engineering teams and business stakeholders. Participate in Agile development processes and contribute to continuous improvement. Required Skills and Qualifications 8 years of overall IT experience, including at least 5 years of relevant experience. Strong hands-on experience in Python scripting and PySpark. Proficiency in SQL and experience working with large datasets. Experience with AWS services, including: Amazon S3 Amazon Redshift AWS Lambda Amazon EMR AWS Glue Apache Airflow Strong expertise in Bash/Shell scripting. Experience developing and supporting ETL and data processing pipelines. Understanding of the healthcare insurance domain. Experience working in Agile environments. Excellent analytical, problem-solving, communication, and collaboration skills. Preferred Qualifications Experience working with healthcare insurance data, such as claims, member eligibility, and provider data. Experience optimizing PySpark jobs and large-scale data processing workflows. Experience with production support, pipeline monitoring, and data quality checks. Candidate Submission Details Interested candidates can share their updated resume along with the following information: Total IT experience: Python experience: PySpark experience: SQL experience: AWS experience: Bash/Shell scripting experience: Healthcare domain experience: