Data Engineer - Big Data (Spark/Scala, Hadoop, SQL, Python)

Job Description

Job Description

Overview

CTG is seeking to fill a Data Engineer – Big Data (Spark/Scala, Hadoop, SQL, Python) position for our client.

 

Location: Piscataway, NJ

Duration: 12 months

 

Duties:

• Design, develop, test, and maintain scalable data-processing applications using Apache Spark and Scala.

• Develop and support Big Data applications and data pipelines utilizing Hadoop, Hive, and Impala.

• Write complex SQL queries for data extraction, transformation, reconciliation, analysis, and performance optimization.

• Develop Python scripts and utilities to support data processing, automation, and data engineering activities.

• Design, build, and maintain scalable ETL/data pipelines for large-volume datasets.

• Analyze, troubleshoot, and debug complex application and data-processing code.

• Support data quality, validation, reconciliation, and integrity across enterprise data platforms.

• Collaborate with application developers, data engineers, analysts, and business stakeholders in an Agile environment.

• Contribute to API development and application solutions supporting Big Data platforms and analytics.

• Apply knowledge of analytics libraries, open-source technologies, statistical computing, and Big Data processing frameworks as appropriate.

Skills:

• Strong hands-on experience with Apache Spark and Scala for distributed data processing.

• Strong experience with Hadoop, Hive, and Impala and enterprise Big Data platforms.

• Advanced SQL skills, including complex queries, joins, transformations, reconciliation, and performance tuning.

• Strong Python scripting and data-processing experience.

• Strong application development, coding, debugging, and problem-solving skills.

• Knowledge of ETL/data pipelines, data quality, and data reconciliation.

• Understanding of analytics libraries, statistical computing, and open-source data-processing technologies.

• Experience with Agile development methodologies and CI/CD practices is preferred.

Experience:

• Demonstrated experience developing and supporting applications in enterprise Big Data environments.

• Hands-on experience working with large datasets and distributed data-processing technologies.

• Experience designing, developing, testing, and debugging complex code.

• Experience with large-volume transactional or financial datasets is preferred.

• Banking or financial services experience is preferred.

• Experience supporting production data applications and resolving technical issues is a plus.

Education:

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent combination of education and experience.

Excellent verbal and written English communication skills and the ability to interact professionally with a diverse group are required.

 

CTG does not accept unsolicited resumes from headhunters, recruitment agencies, or fee based recruitment services for this role.

 

To Apply:

To be considered, please apply directly to this requisition using the link provided. Kindly forward this to any other interested parties. Thank you!

 

The expected base salary for this position ranges from $50.00 to $60.00/hour. Salary offers are based on a wide range of factors including relevant skills, training, experience, education, market factors, and where applicable, licensure or certifications obtained. In addition to salary, a competitive benefit package is also offered.