Data Engineer

V-Soft Consulting Group

Technical Stack & Requirements:
• Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks, Sagemaker.
• Infrastructure: CloudFormation/Terraform, CI/CD with GitHub Actions.
• Core Responsibilities: Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-level production support to ensure system reliability.

Qualifications:
• Bachelor's degree in Computer Science or related field.
• Strong hands-on experience with Apache Spark and Glue.
• Proven ability to work with Python-based data pipelines.
• Experience with Git and GitHub Actions is mandatory.

Required

Qualifications
• Bachelor's degree in Computer Science, Engineering, Information

Systems, or related discipline.
• 5-7+ years of enterprise Data Engineering experience.
• Strong Python development experience.
• dvanced SQL programming skills.
• Extensive experience with Apache Spark and PySpark.
• Strong experience with Databricks application.
• Hands-on experience with AWS data services including:

o S3

o WS Glue

o thena

o Lambda

o Step

Functions

o EventBridge
• Experience building enterprise ETL and ELT pipelines.
• Experience in EMR-based Query Cluster workloads, that includes Jupyter notebooks, Park Application and ETL Pipelines.
• Strong understanding of distributed data processing.
• Experience working with large-scale cloud data platforms.
• Strong knowledge of dimensional data modeling.
• Experience implementing CI/CD pipelines.
• Familiarity with Agile/Scrum methodologies.
• Excellent communication and stakeholder management skills.

Preferred

Qualifications
• Financial Services or Asset Management industry experience.
• Knowledge of Apache Iceberg, Delta Lake, or Hudi.
• Experience with Airflow.
• Knowledge of Kafka or Kinesis streaming.
• Experience supporting AI/ML and advanced analytics platforms.
• WS Professional or Specialty Certifications.

Apply

Vacancy posted 3 days ago

Similar jobs that could be interesting for youBased on the Data Engineer in Malvern, PA vacancy

Data Engineer
Find a JobJoin IDR Internal TeamStaffing ServicesConsultantsAbout UsBlog
Suggested
IDR Internal Data Resources
Malvern, PA
7 hours ago
Data Engineer/Developer
...Data Engineer Troubleshoots software and processes for data consistency and integrity. Integrates complex data from a variety of sources for business partners to generate insight and make decisions. Translates business specifications into design specifications and...
Suggested
Samprasoft
Malvern, PA
2 days ago
Data Engineer
...Job Title: Data Engineer Duration- Fulltime Permanent Location: Malvern, PA, 19355 / Charlotte, NC, 28262 (Hybrid /requiring 3 days onsite Mandate (Tuesday, Wednesday, and Thursday) (Local Only) Job Description: Data Engineer Python, Glue ETL, Attunity...
Suggested
Permanent employment
Full time
Local area
Q1 Technologies
Malvern, PA
3 days ago
Data Engineer
...Data Engineer As a Data Engineer, you’ll build sophisticated solutions that support a wide range of advanced data driven capabilities. Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data...
Suggested
Samprasoft
Malvern, PA
2 days ago
Data Engineer
...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr....
Suggested
Contract work
Local area
Immediate start
Syntricate Technologies
Malvern, PA
4 days ago
Data Engineer
$60 - $67 per hour
...Akkodis is seeking a Data Engineer for a Contract with a client in Malvern, PA. The ideal candidate is responsible for designing, building, and optimizing scalable data platforms, real-time streaming pipelines, and cloud-native data solutions . Rate Range: $6...
Hourly pay
Contract work
Temporary work
Local area
Akkodis
Malvern, PA
3 days ago
Data Engineer
...Senior Data Engineer We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources...
RIT Solutions
Malvern, PA
1 day ago
Data Engineer
...Pyhton, Pyspark Strong experience in designing and developing solutions inAWS Redshift, Loading, organizing and querying tables data into AWS S3 using AWS Glue. Create Glue catalog configurations Develop reliable Glue ETL Jobs that effectively extract,...
Diverse Lynx
Malvern, PA
2 days ago
Data engineer
...High-Level Data Engineer Position Data engineer Malvern PA hybrid assessment required. Minimum 8-10 years of experience. Location – preferred Charlotte (but can be in Malvern). Role – level 4 big data engineer, Python developer. Required skillset – Python, SQL, AWS...
RIT Solutions
Malvern, PA
1 day ago
Data Engineer
...Job Description Overview We are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal...
InterSources
Malvern, PA
5 days ago
Data Engineer
...Data Engineer Address:301 Lindenwood Dr Ste 330, Malvern, Pennsylvania Remote: 3 days onsite Hours: 8-5 Position Type: FULL-TIME KEY RESPONSIBILITIES Data Architecture & Pipeline Development Design, construct, and maintain robust data...
Full time
Remote work
MHK TECH INC
Malvern, PA
3 days ago
Big Data Engineer
$49.46 - $54.46 per hour
...Data EngineerResponsibilities Data Engineer with experience designing scalable data pipelines, ETL workflows, and cloud-based analytics solutions. Robust expertise in Python, SQL, PySpark, Databricks and AWS services. Location: Malvern, Pennsylvania. Job type: Contract...
Hourly pay
Contract work
Randstad
Malvern, PA
3 days ago
Big Data Engineer
...Job TitleResponsibilitiesAbility to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data filesQualificationsExpert in SQLAWS S3 and Glue experienceAdvanced Data Modeling principlesRequired Skills : Data WarehouseBasic Qualification...
Kaav Inc.
Malvern, PA
3 days ago
Big Data Engineer
...Big Data EngineerHybrid - Malvern, PAResponsibilitiesWrites ETL (Extract / Transform / Load) processes, designs database systems,... ...performs other duties as assigned.Qualifications1-3 years Data Engineering Experience with SQL, Python & AWSExperience with CI/CD and Jira...
RIT Solutions
Malvern, PA
3 days ago
Senior Data Engineer
...Job Title: Senior Data Engineer Location : Malvern, PA Duration:6 Months Experience Required: 8-10 Request ID:82529-1 Description : Please prioritize candidates with Vanguard experience. ROLE_DESCRIPTION - Build and maintain event-driven...
I-Flow
Malvern, PA
1 day ago
Senior Data Engineer (Hybrid)
...Senior Data Engineer (Hybrid) Location: Wilmington, DE / Berwyn, PA Duration: 18 Months, quarterly increments Job Description: Senior Data Engineer to support a large-scale enterprise data modernization initiative focused on migrating critical banking and enterprise...
Serigor Inc
Berwyn, PA
2 days ago
Senior Data Engineer
...Senior Data Engineer – 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing, building, and optimizing robust data solutions. The ideal candidate must possess top-tier skills in Python, AWS services,...
Samprasoft
Malvern, PA
2 days ago
SENIOR DATA ENGINEER
...ROLE SENIOR DATA ENGINEER INDUSTRY Pharma, Development & Manufacturing DURATION CONTRACT, probable extension LOCATION REMOTE (off-shore or near-shore), Malvern, PA 19355 INTERVIEW MS TEAMS conversation the candidate will be required...
Contract work
Immediate start
Remote work
3B Staffing LLC
Malvern, PA
1 day ago
Senior Lead Data & AI Engineer
$190k
...Senior Lead Data & AI Engineer Malvern, PA, USA Up to $190,000/ year Machine Learning Engineer Artificial Intelligence Engineer Platform Engineer Data Engineer Vanguard is seeking an expert Lead Data & AI Engineer with 9 to 12+ years of experience to architect...
Hackajob
Malvern, PA
3 days ago
Data Engineer
Data Engineer Assessment required Hybird Malvern, pa Responsibilities Building and maintaining ETL pipelines, Data Analysis, Reimaging Technical Architecture for Analytics Qualifications Python, SQL, AWS Services
Akaasa Technologies
Malvern, PA
1 day ago
Data Engineers
...Opening ! Hybrid onsite ROPES TEST / BROWN- I will send. Description- Design, develop, and maintain high-volume data transformation logic primarily using AWS Glue jobs written in Python . Develop custom code and potentially AWS Lambda...
Relocation
VSG Business Solutions LLC
Malvern, PA
1 day ago
Data Engineer
$53 - $56 per hour
...IDR is seeking a Data Engineer to join one of our top clients for an opportunity in Malvern, PA. This role involves designing scalable data pipelines, ETL workflows, and cloud-based analytics solutions within the financial services industry. The position offers a chance...
Temporary work
IDR
Malvern, PA
1 day ago
AWS Data Engineer
...Job TitleWe are looking for strong AWS Data Engineers who are passionate about Cloud technology.ResponsibilitiesDesign and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.Implement...
Syntricate Technologies
Malvern, PA
3 days ago
Senior Manager, AI and Data Engineering
The Senior Manager, AI & Data Engineering leads teams responsible for building and operating reusable, governed data products and AI-ready semantic intelligence that enable enterprise data integration, activation, analytics, personalization, and decision-making across...
Full time
For contractors
Work at office
Vanguard
Malvern, PA
2 days ago
Data Engineer, Specialist
Core ResponsibilitiesProvides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure. Utilizes a deep dive analytical skillset...
Full time
Work experience placement
Vanguard
Malvern, PA
3 days ago
Head of Data Engineering & Analytics
...VSH). Learn more at . Do you want to help us build the DNA of tech.? Vishay Italy is currently seeking applicants for Head of Data Engineering & Analytics. In this position, you will be accountable for defining, governing, and evolving the enterprise‑wide data architecture...
Full time
Vishay
Malvern, PA
4 days ago
Data Engineer
We are looking for a Data Engineer to join a growing team in Conshohocken, Pennsylvania within the financial services industry. In this role, you will build and enhance modern data pipelines and warehouse structures that support reporting, analytics, and business decision...
Robert Half
Conshohocken, PA
2 days ago
Senior AWS Data Engineer / Software Engineer
Senior AWS Data Engineer / Software Engineer Location: Malvern, PA Work Model: Hybrid Duration: 12+ Months Contract Interview: Video Experience: 7+ Years Primary Technology Stack The core technology stack we're targeting is: AWS...
Contract work
Akaasa Technologies
Malvern, PA
1 day ago
Big Data Engineer
...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...
Permanent employment
Full time
H1b
Sonsoft
King of Prussia, PA
2 days ago
Data Senior Engineer
Data Engineer - AIDay to Day job Duties: (what this person will do on a daily/weekly basis)Design and build scalable ingestion, transformation, and processing pipelines for AI/ML.Develop and maintain data storage, such as vector databases, that support Retrieval-Augmented...
NTT DATA
King of Prussia, PA
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!