AWS Data Lead

Job Title: AWS Data Pipeline Engineer / Technical Lead /AWS Data Engineer

Location: Newark, NJ (Hybrid – 2 to 3 days onsite per week), Local candidates

Duration: 12 Months

Domain: Investment Banking

Position Summary

We are seeking a highly experienced AWS Data Pipeline Engineer/Technical Lead to design, build, optimize, and support scalable, enterprise-grade data pipelines within a cloud-native AWS environment.

Key Responsibilities

• Design, develop, deploy, and maintain batch and real-time data pipelines using AWS services.
• Build scalable and resilient solutions for processing multi-terabyte datasets and high-throughput event streams.
• Implement end-to-end data ingestion, transformation, validation, enrichment, and delivery processes.
• Design and implement serverless data processing solutions using AWS Lambda and Step Functions.
• Develop distributed data processing solutions leveraging AWS Glue, EMR, and PySpark.
• Build and optimize data warehouse solutions using Amazon Redshift.
• Develop high-performance analytical solutions using Amazon Athena and S3-based Data Lakes.
• Optimize data pipelines through partitioning strategies, file layout design, indexing, caching, and workload tuning.

Required Skills

• Bachelor's degree in Computer Science, Information Systems, Engineering, or related field.
• 13+ years of experience in Data Engineering, ETL/ELT, Data Integration, or Data Pipeline Development.
• Strong expertise in AWS Cloud Data Engineering Services .
• Hands-on implementation experience with:

AWS Lambda

AWS Step Functions

Amazon S3

AWS Glue

Amazon Redshift

Amazon Athena

• Strong proficiency in Python, PySpark, and SQL .
• Experience building and supporting enterprise-scale Data Lakes and Data Warehouses.
• Expertise in data modeling, partitioning strategies, file formats, and performance optimization.
• Strong understanding of query tuning, workload management, and bottleneck analysis.
• Experience implementing logging, monitoring, alerting, retry mechanisms, fault tolerance, and disaster recovery.
• Proven ability to explain architecture decisions, technical implementations, and performance considerations in detail.
• Strong analytical, troubleshooting, and technical leadership skills.
• Excellent verbal and written communication skills.

Preferred Qualifications

• Experience working in Investment Banking, Capital Markets, or Financial Services environments.
• Knowledge of Data Governance, Metadata Management, and Master Data Management (MDM).
• Experience with Docker and Kubernetes.
• Experience architecting cloud-native, event-driven data platforms.
• Prior experience leading or mentoring data engineering teams.