Senior Data Engineer
I-Flow
Job Title: Senior Data Engineer
Location : Malvern, PA
Duration:6 Months
Experience Required: 8-10 Request ID:82529-1 Description :
Please prioritize candidates with Vanguard experience. ROLE_DESCRIPTION -
Build and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency.
Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks.
Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift.
Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.
Ensure high standards of data quality, observability, monitoring, and alerting (CloudWatch, Datadog, etc.).
Implement data security best practices including IAM, encryption (KMS), networking, and governance.
Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments.
Collaborate with data scientists, analysts, and business teams to deliver well-modeled, reliable datasets.
Lead design reviews, mentor junior engineers, and contribute to engineering best practices. Required Qualifications
Overall 8+ yrs of experience
5+ years of professional experience in Data Engineering
Must have an experience on Gule jobs and Lambda,
Experience of working on Java is an advantage
Strong expertise in Python and PySpark for large-scale data processing.
Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, Redshift).
Deep experience building event-driven and streaming data pipelines.
Strong SQL experience for analytical and ETL workloads.
Hands-on experience with workflow orchestration tools such as Airflow or Step Functions.
Experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).
Strong understanding of distributed systems, Spark performance tuning, data modeling, and cloud cost optimization.
Knowledge of data security, encryption, networking, and compliance best practices in cloud environments
Skills : Digital : Spring Boot
Location : Malvern, PA
Duration:6 Months
Experience Required: 8-10 Request ID:82529-1 Description :
Please prioritize candidates with Vanguard experience. ROLE_DESCRIPTION -
Build and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR.
Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency.
Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks.
Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift.
Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution.
Ensure high standards of data quality, observability, monitoring, and alerting (CloudWatch, Datadog, etc.).
Implement data security best practices including IAM, encryption (KMS), networking, and governance.
Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments.
Collaborate with data scientists, analysts, and business teams to deliver well-modeled, reliable datasets.
Lead design reviews, mentor junior engineers, and contribute to engineering best practices. Required Qualifications
Overall 8+ yrs of experience
5+ years of professional experience in Data Engineering
Must have an experience on Gule jobs and Lambda,
Experience of working on Java is an advantage
Strong expertise in Python and PySpark for large-scale data processing.
Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, Redshift).
Deep experience building event-driven and streaming data pipelines.
Strong SQL experience for analytical and ETL workloads.
Hands-on experience with workflow orchestration tools such as Airflow or Step Functions.
Experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation).
Strong understanding of distributed systems, Spark performance tuning, data modeling, and cloud cost optimization.
Knowledge of data security, encryption, networking, and compliance best practices in cloud environments
Skills : Digital : Spring Boot
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer in Malvern, PA vacancy
- ...Sr. Data EngineeringThe Sr. Data Engineering will manage the data pipeline development from data ingestion to transformation, implementing the Architect’s data models and ensuring high-quality data flow through the bronze, silver, and gold layers. This role involves leading...Senior
Kaav Inc.
Malvern, PA3 days ago - ...Senior Data Engineer – 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing, building, and optimizing robust data solutions. The ideal candidate must possess top-tier skills in Python, AWS services,...Senior
Samprasoft
Malvern, PA2 days ago - ...We are seeking a Senior Data Engineer to design, develop, and maintain scalable data pipelines and data ingestion processes using modern Big Data technologies. The ideal candidate will have strong experience in data aggregation, transformation, validation, quality management...Senior
Techvilla Solutions
Berwyn, PA1 day ago - ...Senior Data Engineer Assessment required hybrid - Malvern, PA Technical Stack & Requirements: • Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks, Sagemaker. • Infrastructure: CloudFormation...Senior
RIT Solutions, Inc.
Malvern, PA2 days ago - ...ROLE SENIOR DATA ENGINEER INDUSTRY Pharma, Development & Manufacturing DURATION CONTRACT, probable extension LOCATION REMOTE (off-shore or near-shore), Malvern, PA 19355 INTERVIEW MS TEAMS conversation the candidate will be required...SeniorContract workImmediate startRemote work
3B Staffing LLC
Malvern, PA1 day ago - ...Senior Data Engineer (Hybrid)Location: Wilmington, DE / Berwyn, PADuration: 18 Months, quarterly incrementsJob Description:Senior Data Engineer to support a large-scale enterprise data modernization initiative focused on migrating critical banking and enterprise data...Senior
Serigor Inc
Berwyn, PA1 day ago $190k
...Senior Lead Data & AI Engineer Malvern, PA, USA Up to $190,000/ year Machine Learning Engineer Artificial Intelligence Engineer Platform Engineer Data Engineer Vanguard is seeking an expert Lead Data & AI Engineer with 9 to 12+ years of experience to architect...SeniorHackajob
Malvern, PA3 days ago- The Senior Manager, AI & Data Engineering leads teams responsible for building and operating reusable, governed data products and AI-ready semantic intelligence that enable enterprise data integration, activation, analytics, personalization, and decision-making across Vanguard...SeniorFull timeFor contractorsWork at office
- Senior AWS Data Engineer / Software Engineer Location: Malvern, PA Work Model: Hybrid Duration: 12+ Months Contract Interview: Video Experience: 7+ Years Primary Technology Stack The core technology stack we're targeting is: AWS...SeniorContract work
Akaasa Technologies
Malvern, PA2 days ago - Data Engineer - AIDay to Day job Duties: (what this person will do on a daily/weekly basis)Design and build scalable ingestion, transformation, and processing pipelines for AI/ML.Develop and maintain data storage, such as vector databases, that support Retrieval-Augmented...Senior
- Coforge is seeking a Senior Data Engineer (Agentic Retrieval & Memory) in Oaks, PA for a full-time role. You will design agent memory interfaces across session state, short-term history, and long-term memory, and select optimal stores for each tier. You will integrate...SeniorFull timeTemporary work
- Role: Senior Data Engineer (Agentic Retrieval & Memory) Mode of Hire: Full time RESPONSIBILITIES Design and build the agent memory interface across the tiers an agent actually needs: working or session state within a run, short-term conversation history, and long-term...SeniorFull timeTemporary work
- Vish Consulting Services (VISHUSA) in Oaks, PA, is seeking a Senior Data Engineer specializing in Agentic Retrieval, memory systems, and enterprise AI platforms. The role focuses on building intelligent data retrieval systems, memory architectures, and LLM-driven enterprise...Senior
- Find a JobJoin IDR Internal TeamStaffing ServicesConsultantsAbout UsBlog
- Job Title: Senior Data Engineer (Agentic Retrieval & Memory Systems) | Vish Consulting Services (VISHUSA) | Oaks, Pennsylvania, USA Recruiting Company: Vish Consulting Services (VISHUSA) Job Location: Oaks, Pennsylvania, USA (Hybrid) Job Type: Contract (C2C) Application...SeniorContract workH1bImmediate start
- A leading insurance company is seeking a Data Operations & Database Engineer to design and monitor ETL/ELT pipelines, manage databases, and partner with other data professionals. Ideal candidates will have over 7 years of experience in data engineering, with strong SQL...Senior
$120k - $130k
...Attunity, Pyspark, AWS (S3, IAM, Lambda, Cloud Formation) Roles & Responsibilities • Design, develop, and maintain scalable ETL/data integration pipelines using AWS Glue, PySpark, and Python. • Develop and optimize data transformation workflows to process large...Tata Consultancy Services
Malvern, PA2 days ago$60 - $67 per hour
...Akkodis is seeking a Data Engineer for a Contract with a client in Malvern, PA. The ideal candidate is responsible for designing, building, and optimizing scalable data platforms, real-time streaming pipelines, and cloud-native data solutions . Rate Range: $6...Hourly payContract workTemporary workLocal areaAkkodis
Malvern, PA3 days ago- ...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr....Contract workLocal areaImmediate start
Syntricate Technologies
Malvern, PA4 days ago - ...Job TitleWe are looking for strong AWS Data Engineers who are passionate about Cloud technology.ResponsibilitiesDesign and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.Implement...
Syntricate Technologies
Malvern, PA3 days ago - ...Responsibilities ~ Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required...
Procyon TS
Malvern, PA18 hours ago - ...analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates complex and large-scale... ...consultant for the client. Educates and develops junior data engineers on the team while applying quality control to their work....
Kaav Inc.
Malvern, PA5 days ago - ...Data Engineer Address:301 Lindenwood Dr Ste 330, Malvern, Pennsylvania Remote: 3 days onsite Hours: 8-5 Position Type: FULL-TIME KEY RESPONSIBILITIES Data Architecture & Pipeline Development Design, construct, and maintain robust data...Full timeRemote work
MHK TECH INC
Malvern, PA3 days ago - Data EngineerDuration: ContractLocation: Malvern PA (Day 1 onsite) Local OnlyJob Description:AWS Redshift is mandatoryWith SQL, PythonIf someone worked with Apache Iceberg would be greatRegards,Pallavi VermaSr. Technical Recruiter | Syntricate Technologies Inc.Local area
Syntricate Technologies
Malvern, PA3 days ago - ...Data Engineer As a Data Engineer, you’ll build sophisticated solutions that support a wide range of advanced data driven capabilities. Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data...
Samprasoft
Malvern, PA2 days ago - ...Data engineer Malvern pa hybrid Assessment required Minimum 8-10 years of exp n * Location - Preferred Charlotte (but can be in Malvern) * pprox desired start - asap * Role - Level 4 Big Data Engineer, Python developer * required skillset - Python...Immediate start
RIT Solutions, Inc.
Malvern, PA3 days ago - ...Data EngineerClient: Vanguard Location: Malvern, PA or Charlotte, NC or Phoenix, AZ (3 days onsite) - need to be onsite day one Duration: 1 year Video Interview is okayResponsibilitiesData Engineers will be data mining, writing code, and creating tables/environments in...
RIT Solutions
Malvern, PA3 days ago - ...Pyhton, Pyspark Strong experience in designing and developing solutions inAWS Redshift, Loading, organizing and querying tables data into AWS S3 using AWS Glue. Create Glue catalog configurations Develop reliable Glue ETL Jobs that effectively extract,...
Diverse Lynx
Malvern, PA2 days ago - ...Job Description Overview We are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal...
InterSources
Malvern, PA5 days ago $49.46 - $54.46 per hour
...Data EngineerResponsibilities Data Engineer with experience designing scalable data pipelines, ETL workflows, and cloud-based analytics solutions. Robust expertise in Python, SQL, PySpark, Databricks and AWS services. Location: Malvern, Pennsylvania. Job type: Contract...Hourly payContract workRandstad
Malvern, PA3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
Related searches
- aws data engineer Malvern, PA
- data science developer Malvern, PA
- data engineer machine learning Malvern, PA
- data center engineer Malvern, PA
- data developer Malvern, PA
- senior data center engineer Malvern, PA
- finance data engineer Malvern, PA
- etl data engineer Malvern, PA
- data engineer Malvern, PA
- data engineer analytics Malvern, PA