AWS Data Engineer
Programmers.io
Overview:
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
Expertise in building and optimizing ETL pipelines in a cloud environment.
In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
Strong problem-solving and troubleshooting skills.
Familiarity with infrastructure as code tools like AWS CloudFormation.
Experience with version control (e.g., Git) and CI/CD pipelines.
Solid understanding of data security, data privacy, and compliance frameworks.
Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Core Java| GUI & Rich Clients~Core Java
Experience Required: 10 & Above
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
Expertise in building and optimizing ETL pipelines in a cloud environment.
In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
Strong problem-solving and troubleshooting skills.
Familiarity with infrastructure as code tools like AWS CloudFormation.
Experience with version control (e.g., Git) and CI/CD pipelines.
Solid understanding of data security, data privacy, and compliance frameworks.
Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Core Java| GUI & Rich Clients~Core Java
Experience Required: 10 & Above
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AWS Data Engineer in Malvern, PA vacancy
- ...High-Level Data Engineer Position Data engineer Malvern PA hybrid assessment required. Minimum 8-10 years of experience. Location – preferred... ...(but can be in Malvern). Role – level 4 big data engineer, Python developer. Required skillset – Python, SQL, AWS services....Amazon Web Service
RIT Solutions
Malvern, PA2 days ago - Job Title Required Skills: Python, Spark, AWS services, Glue, IAC Responsibilities ~ Work with Analytics and Business - derive requirements, build Engineering pipelines aligned to standards, trouble shoot data discrepancies and ensure quality dataAmazon Web Service
Syntricate Technologies
Malvern, PA2 days ago - ...Description Overview We are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting... ...will be a hands-on developer with deep expertise in the AWS data stack, focusing primarily on data engineering and pipeline...Amazon Web Service
InterSources
Malvern, PA15 hours ago - ...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr....Amazon Web ServiceContract workLocal areaImmediate start
Syntricate Technologies
Malvern, PA5 days ago - ...Role Summary: The Sr. Data Engineering will manage the data pipeline development from data ingestion to transformation, implementing the... ...Responsibilities: Lead the development of data pipelines using AWS Glue and PySpark, transforming data from raw to analytics-...Amazon Web Service
Kaav Inc.
Malvern, PA15 hours ago - ...Cloud Data Engineer This role requires an understanding of mapping legacy or home-grown data platforms and modernization or migration to a cloud-native data platform. Understanding of AWS cloud systems is critical. A minimum of 10 years of hands-on expertise in AWS...Amazon Web Service
Diverse Lynx
Malvern, PA3 days ago - ...Big Data Engineer - Malvern, PA Duration - 12+ months Interview Process - Video interview Must Have - Expert in Python... ...pipelines Proficiency in developing serverless architectures using AWS services Experience with both relational and NoSQL databases...Amazon Web Service
Veracity
Malvern, PA2 days ago - ...Senior Data Engineer We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources...Amazon Web Service
RIT Solutions
Malvern, PA2 days ago - ...Data engineer 5-8 years of exp hybird - malvern, pa or plano, tx, or charlotte, nc assessment required Description Details... ...skills in data transformation using Python, PySpark, SQL, and AWS services, including experience with creating data products to...Amazon Web Service
RIT Solutions, Inc.
Malvern, PA2 days ago - ...Requirements: • Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks,... ...Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-...Amazon Web Service
V-Soft Consulting Group
Malvern, PA4 days ago - ...Trading Analytics And Strategy Engineer Client's Trading Analytics and Strategy (TAS) team... ...passive as well as active mandates, to conduct data-driven analyses and build tools to help... ...Extensive experience with key AWS services/components including EMR, Lambda,...Amazon Web Service
Samprasoft
Malvern, PA3 days ago - ...Job Title: Data Engineer Location: Malvern, PA Duration:6 Months Experience Required: 10 & Above Job Description: Technical skill sets : Python, Pyspark, AWS Lambda, S3,AWS glue, Kinesis,SQL, Apache Flink, Confluent Kafka, AI-LLM, Gen-AI , Java is optional...Amazon Web Service
I-Flow
Malvern, PA4 days ago - ...Data Analytical Engineer SEEKING A SENIOR CANDIDATE hybrid - Malvern. PA Job Description Writes ETL (Extract / Transform... ...generate insight and make decisions. Experience building AWS cloud architecture and supporting services and technologies...Amazon Web Service
RIT Solutions
Malvern, PA3 days ago - Responsibilities Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required Skills : Data Warehouse Basic Qualification :...Amazon Web Service
- ...Data Engineer - Assessment required Hybird Malvern, pa Job Description Duties: Build out an extensive transaction stitching... ...over the lifetime of the funds. Skills: Advanced Python, SQL skills and AWS Services (e.g. Glue, S3, Lambda, Redshift...Amazon Web Service
Akaasa Technologies
Malvern, PA1 day ago - Role Description: We are looking for strong AWS Data Engineers who are passionate about Cloud technology. Your work will be to: Design and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting...Amazon Web Service
- ...Hi , This is Praveen from Pronixinc Role: AWS Big Data Engineer with Databricks (Mid, Sr and SME level role)- W2 Only Location: Malvern, PA (Hybrid 3 days in office and 2 days remote) Duration: long term Contract 1+ year (likely to be extended multiple...Amazon Web ServiceLong term contractWork at officeRemote work
Pronix - US
Malvern, PA1 day ago - Core ResponsibilitiesProvides advanced data solutions by using software to process, store, and serve data to others. Tests data quality... ...and batch processing.Experience in cloud data platforms including AWS Redshift, Google Big Query, or Azure Synapse, with proficiency in...Amazon Web ServiceFull timeWork experience placement
- Overview: Job Description - AWS Data Engineer (Python, AWS Glue, PySpark & Attunity) Job Title AWS Data Engineer - Python, AWS Glue, PySpark & Attunity Job Summary We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS Glue, PySpark, Attunity...Amazon Web Service
- The Senior Manager, AI & Data Engineering leads teams responsible for building and operating reusable, governed data products and AI-ready semantic... ..., and AI workloads.Drive modernization initiatives through AWS-based platforms, automation, DataOps, MLOps, platform...Amazon Web ServiceFull timeFor contractorsWork at office
- Role Overview The Data Engineer, Specialist is responsible for designing, developing, and maintaining scalable data pipelines and infrastructure... ...retrieval strategies in cloud-based data warehouses such as AWS Redshift, Google Big Query, or Azure Synapse, ensuring...Amazon Web Service
- ...Data Engineer Client: Vanguard Location: Malvern, PA or Charlotte, NC or Phoenix, AZ (3 days onsite) - need to be onsite day one... ...environments in S3 Qualifications SQL, Python, Pyspark, S3, AWS, environment creation Preferable for both - retail...Amazon Web Service
RIT Solutions, Inc.
Wayne, PA4 days ago - ...the office for the remote option.)Job SummaryWe are seeking a Data Engineer to join our Data Product Engineering team responsible for managing... ...of interaction data, leveraging modern technologies including AWS, PySpark, Databricks, Kafka, Kubernetes, and Airflow.Job...Amazon Web ServiceFull timeWork at officeRemote workWorldwide
- Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. • Deep technical knowledge - including...Amazon Web Service
- Data Engineer (TS/SCI with Full Scope Polygraph) {S} Job Category : ENG Requisition Number : DATAE003244 Apply now Posted : February 23,... ...containers, (e.g. Docker, Docker Compose), cloud services (e.g. AWS), tools for distributed computing (e.g. Spark, Pyspark) Active...Amazon Web ServiceFull timeTemporary workWork experience placementWork at officeVisa sponsorshipRelocation packageFlexible hours
$100k - $180k
...Data Center Network Engineer Location: Multiple locations (Wayne, PA; Plano, TX; Scottsdale, AZ; Charlotte, NC) Schedule: Hybrid 3 days a... ...supporting global data center infrastructure. Exposure to AWS and/or GCP networking environments. Experience supporting...Amazon Web ServicePermanent employmentContract workRemote work3 days per weekApex Systems
Wayne, PA5 days ago- ...Role at a Glance Lincoln Financial's AI, Data and Analytics (AIDA) organization is... ...deliver with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams,... ...grounding, and citations. Design and build AWS source-to-product pipelines and agent-...Amazon Web ServiceFor contractorsWork experience placementWork at officeRelocation package3 days per week
Lincoln Financial
Radnor, PA1 day ago - Job Title: Data Center Network Engineer Location: Multiple locations (Wayne, PA; Plano, TX; Scottsdale, AZ; Charlotte, NC) Schedule: Hybrid 3 days... ...supporting global data center infrastructure. Exposure to AWS and/or GCP networking environments. Experience supporting migrations...Amazon Web ServicePermanent employmentRemote work3 days per week
- ...Role at a Glance Lincoln Financial's AI, Data and Analytics (AIDA) organization is... ...deliver with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams,... ...data models using SQL, Python, and approved AWS services. Build and maintain FastAPI...Amazon Web ServiceContract workWork experience placementWork at officeRelocation package3 days per week
Lincoln Financial
Radnor, PA1 day ago $120.38k - $192.6k
....Requisition #: 76603The Role at a GlanceAs a Senior Lead Risk Data Engineer and Architect, you will architect and lead the design and implementation... ...management. Working on a centralized cloud data platform (AWS, Databricks or Dataiku), you'll design scalable data pipelines,...Amazon Web ServiceWork experience placementWork at office3 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AWS Data Engineer. Be the first to apply!
Related searches
- finance data engineer Malvern, PA
- data engineer machine learning Malvern, PA
- aws data engineer Malvern, PA
- data center engineer Malvern, PA
- data engineer analytics Malvern, PA
- data developer Malvern, PA
- etl data engineer Malvern, PA
- senior data center engineer Malvern, PA
- data science developer Malvern, PA
- data engineer Malvern, PA