Data Engineer

Hello,

Job title: Data Engineer







Contract Duration: 4 months, Contract W2 Only Potential Extension



Primary Locations: Wilmington, DE Hybrid



Other Locations Considered: McLean, VA; Richmond, VA; New York City, NY; Plano, TX







Please submit the candidate who can take the assessment today for this particular position.

Position Overview

Client is seeking experienced Data Engineers to support a high-priority customer migration initiative involving the transition of Discover customers onto the platform. The selected candidate will design, develop, maintain, and support scalable data pipelines and data-processing solutions in a high-volume banking environment. This is primarily a Java-focused Data Engineering role . Candidates must have strong hands-on experience with Java, Spring Boot, Apache Spark, PySpark, AWS, and CI/CD .

The ideal candidate will have experience across data ingestion, transformation, integration, testing, deployment, monitoring, and production support.

Key Responsibilities

• Design, develop, and maintain scalable batch and streaming data pipelines.
• Build data ingestion, transformation, validation, and delivery processes.
• Develop production-quality applications and services using Java and Spring Boot .
• Use Apache Spark and PySpark to process and transform large datasets.
• Develop AWS-based data solutions using AWS Glue, AWS Lambda, and Amazon S3 .
• Build reusable frameworks and components to improve pipeline performance and reliability.
• Implement data quality checks, validation, reconciliation, and error-handling processes.
• Support customer and account data migration while ensuring data accuracy and completeness.
• Integrate data pipelines with APIs, databases, messaging systems, and downstream platforms.
• Develop RESTful APIs and supporting services when required.
• Create automated unit, integration, and functional tests.
• Participate in code reviews, technical design discussions, and documentation.
• Build and maintain CI/CD pipelines for automated build, testing, and deployment.
• Monitor pipeline execution and troubleshoot production failures.
• Perform root-cause analysis and resolve production issues.
• Optimize Spark jobs, data-processing workflows, and cloud resources.
• Collaborate with architects, developers, QA, DevOps, product managers, and business stakeholders.
• Support deployments across development, testing, staging, and production environments.
• Follow Client's security, data governance, risk-management, and software-development standards.

Required Qualifications / Must-Have Skills

• Strong professional experience in Data Engineering and Data Pipeline Development .
• Strong hands-on Java development experience.
• Hands-on experience with Spring Boot .
• Strong experience with Apache Spark .
• Strong experience with PySpark .
• Hands-on experience with AWS Glue .
• Experience with AWS Lambda .
• Strong experience with Amazon S3 and data lake solutions.
• Experience designing, developing, testing, and supporting scalable data pipelines.
• Experience with CI/CD pipelines and automated software delivery.
• Strong understanding of data ingestion, transformation, validation, reconciliation, and error handling.
• Production troubleshooting and support experience.
• Experience working in large-scale enterprise environments.
• Strong communication and problem-solving skills.

Preferred / Nice-to-Have Skills

• Python development experience.
• RESTful API and microservices development.
• Kafka or other event-streaming technologies.
• Customer data migration experience.
• Banking, financial services, payments, credit card, or related domain experience.
• AI/ML or AI-enabled data platform exposure.
• Cloud-native application development.
• Distributed systems experience.
• Data governance, security, privacy, and regulatory experience.
• Infrastructure automation.
• Docker/Kubernetes experience.

Technology Stack

Core:

• Java
• Spring Boot
• Apache Spark
• PySpark
• AWS Glue
• AWS Lambda
• Amazon S3
• CI/CD

Additional:

• Python
• REST APIs
• Microservices
• Kafka
• Kubernetes
• Cloud-native technologies

Ideal Candidate Profile

The ideal candidate is a hands-on Data Engineer with strong Java development experience who has built and supported enterprise-scale data pipelines. The candidate should be comfortable working across Java/Spring Boot, Spark/PySpark, AWS Glue, Lambda, S3, CI/CD, data migration, and production support .

Candidates with banking/financial services or customer data migration experience , particularly previous Client experience, would be highly relevant.

Screening Must Have

1. Strong Data Engineering experience.
2. Strong hands-on Java development.
3. Spring Boot experience.
4. Apache Spark experience.
5. PySpark experience.
6. AWS Glue experience.
7. AWS Lambda experience.
8. Amazon S3 experience.
9. Data pipeline development and production support.
10. CI/CD experience.
11. Data ingestion, transformation, validation, and reconciliation experience.
12. Strong troubleshooting and problem-solving skills.

Best Regards,

Ishika

SR. IT Technical/Engineer Recruiter

Ph No- View phone number on us.fitly.work

Email: View email address on us.fitly.work

Address: 505 Knolle Court, Saint Augustine, FL 32092