AWS Data Engineer
Programmers.io
Overview:
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
Expertise in building and optimizing ETL pipelines in a cloud environment.
In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
Strong problem-solving and troubleshooting skills.
Familiarity with infrastructure as code tools like AWS CloudFormation.
Experience with version control (e.g., Git) and CI/CD pipelines.
Solid understanding of data security, data privacy, and compliance frameworks.
Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Core Java| GUI & Rich Clients~Core Java
Experience Required: 10 & Above
We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale data processing systems. The ideal candidate will play a critical role in building and maintaining scalable, efficient, and secure ETL pipelines within the AWS ecosystem.
Key Responsibilities:
Design, develop, and maintain robust ETL workflows to extract, transform, and load data from various sources to AWS Redshift and other cloud-based systems.
Leverage AWS Lambda to build serverless computing solutions for real-time data processing and automation.
Work with AWS data storage and processing services, including S3, Redshift, Athena, and Glue, to support data processing pipelines.
Utilize Python and PySpark to create and optimize data transformation logic and analytical queries.
Collaborate with cross-functional teams to understand data requirements and provide efficient data solutions.
Monitor and optimize cloud-based infrastructure for performance, reliability, and cost efficiency.
Troubleshoot and resolve data pipeline issues, ensuring minimal downtime and high data accuracy.
Implement automation and monitoring tools to enhance reliability.
Required Skills and Qualifications:
Proven experience working with AWS services, including Lambda, Redshift, S3, Athena, Glue, and CloudWatch.
Strong knowledge and hands-on experience with Python and PySpark for data transformation and processing.
Expertise in building and optimizing ETL pipelines in a cloud environment.
In-depth understanding of relational databases, data warehousing, and performance tuning in Redshift.
Strong problem-solving and troubleshooting skills.
Familiarity with infrastructure as code tools like AWS CloudFormation.
Experience with version control (e.g., Git) and CI/CD pipelines.
Solid understanding of data security, data privacy, and compliance frameworks.
Ability to work in an agile, fast-paced environment.
Skills: Digital : Python~Digital : Amazon Web Service(AWS) Cloud Computing~Core Java| GUI & Rich Clients~Core Java
Experience Required: 10 & Above
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the AWS Data Engineer in Malvern, PA vacancy
- ...Position : AWS Data Engineering Technical Lead Location: Malvern PA (onsite day 1 for hybrid model) Length: 1+ year (likely to be extended multiple years) Video interview is okay JOB DETAILS - Tech lead experience - Working with Unstructured...Amazon Web Service
Pronix - US
Malvern, PA2 days ago - Job TitleRequired Skills: Python, Spark, AWS services, Glue, IACResponsibilitiesWork with Analytics and Business - derive requirements, build Engineering pipelines aligned to standards, trouble shoot data discrepancies and ensure quality dataAmazon Web Service
Syntricate Technologies
Malvern, PA2 days ago - ...Data Engineer - Assessment required Hybird Malvern, pa Job Description Duties: Build out an extensive transaction stitching... ...over the lifetime of the funds. Skills: Advanced Python, SQL skills and AWS Services (e.g. Glue, S3, Lambda, Redshift...Amazon Web Service
Akaasa Technologies
Malvern, PA3 days ago - ...Data Engineer Location: Malvern, PA - Hybrid Important Submission Requirement Candidates must take and pass a technical... ...Advanced Python Advanced SQL Strong experience with AWS Services , including: AWS Glue Amazon S3 AWS...Amazon Web Service
3B Staffing LLC
Malvern, PA3 days ago - ...Hi , This is Praveen from Pronixinc Role: AWS Big Data Engineer with Databricks (Mid, Sr and SME level role)- W2 Only Location: Malvern, PA (Hybrid 3 days in office and 2 days remote) Duration: long term Contract 1+ year (likely to be extended multiple...Amazon Web ServiceLong term contractWork at officeRemote work
Pronix - US
Malvern, PA3 days ago - Overview: We are seeking an experienced AWS Data Engineer with expertise in ETL processes, AWS Lambda, Redshift, and proficiency in Python and PySpark. This role requires hands-on experience with cloud-based architecture, data engineering, and working with large-scale...Amazon Web Service
Programmers.io
Malvern, PA3 days ago - ...data engineer 5-8 years of exp hybird - malvern, pa or plano, tx, or charlotte, nc assessment required Description... ...skills in data transformation using Python, PySpark, SQL, and AWS services, including experience with creating data products to be...Amazon Web Service
Akaasa Technologies
Malvern, PA2 days ago - ...• Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks, Sagemaker... ...Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-...Amazon Web Service
V-Soft Consulting Group
Malvern, PA2 days ago - ...Responsibilities Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required Skills : Data Warehouse Basic Qualification :...Amazon Web Service
- ...Data Engineer Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline... ...to generate insight and make decisions. Experience building AWS cloud architecture and supporting services and technologies (Eg...Amazon Web Service
Samprasoft
Malvern, PA1 day ago - Role Description: We are looking for strong AWS Data Engineers who are passionate about Cloud technology. Your work will be to: Design and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting...Amazon Web Service
- ...impact, high-visibility role where you will pioneer world-class Data & AI engineering solutions, building the next generation of intelligent, agent... ...agents into business workflows, focusing on Databricks and AWS environments. You will lead data engineering, AI agent...Amazon Web Service
Vanguard
Malvern, PA3 days ago - div:has([data-free-thinking-preview-answer=true])+:is(.text-message,.relative:has(>.text-message))]:-mt-2 grow"> Job Summary The AWS Data Engineer will build and support an extensive transaction stitching application designed to track the journey of money flowing through...Amazon Web Service
- ...with exceptional professionals for this role. Provides advanced data solutions by using software to process, store, and serve data to... ...and batch processing. Experience in cloud data platforms including AWS Redshift, Google Big Query, or Azure Synapse, with proficiency in...Amazon Web ServiceWork experience placement
Vanguard
Malvern, PA4 days ago - ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building... ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with... ..., and citations.Design and build AWS source-to-product pipelines and agent-facing...Amazon Web ServiceFor contractorsWork experience placement3 days per week
Lincoln Financial Group
Radnor, PA2 days ago - Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. • Deep technical knowledge - including...Amazon Web Service
- ...mandates, both passive and active, to conduct data-driven analyses and develop tools that shape trading strategies. As a Full Stack Engineer, you will design, implement, and maintain... ...Test-Driven Development (TDD) practices. AWS Services: Extensive experience with key...Amazon Web Service
- ...The Role at a Glance About AIDA Data Strategy and Governance Lincoln Financial... ...deliver with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams,... ...SQL, Python, orchestration, and approved AWS services. Embed quality, metadata, lineage...Amazon Web ServiceFor contractorsWork experience placementWork at officeRelocation package3 days per week
Lincoln Financial
Radnor, PA4 days ago - Data Engineer (TS/SCI with Full Scope Polygraph) {S} Job Category : ENG Requisition Number : DATAE003244 Apply now Posted : February 23,... ...containers, (e.g. Docker, Docker Compose), cloud services (e.g. AWS), tools for distributed computing (e.g. Spark, Pyspark) Active...Amazon Web ServiceFull timeTemporary workWork experience placementWork at officeVisa sponsorshipRelocation packageFlexible hours
- ...the office for the remote option.)Job SummaryWe are seeking a Data Engineer to join our Data Product Engineering team responsible for managing... ...of interaction data, leveraging modern technologies including AWS, PySpark, Databricks, Kafka, Kubernetes, and Airflow.Job...Amazon Web ServiceFull timeWork at officeRemote workWorldwide
Comcast
West Chester, PA9 days ago - ...Job TitleResponsibilitiesProvides expert level data solutions by using software to process, store, and serve data to others.Tests data quality and optimizes data availability. Ensures that data pipelines are scalable, repeatable, and secure.QualificationsProvides expert...
Samprasoft
Malvern, PA1 day ago - ...with prominent GenAI based solution patterns. Deep knowledge in data science principles, techniques, and development life cycle. Extensive... ...with Python and AI/ML related libraries and SQL. Experience with AWS platform and services preferred. Required Skills: Python, SQL...Amazon Web Service
- ...Job Title: AI ML Engineer Location: Malvern, PA Duration : Full Time Job Description... ...AI and Machine Learning solutions on AWS using Bedrock, SageMaker, Python, MLOps,... ...business problems. Develop scalable data pipelines and MLOps frameworks for model...Amazon Web ServiceFull time
SFE
Malvern, PA3 days ago - ...Job Title: ML Engineer Location: Malvern, PA Duration:6 Months Experience Required: 8-10 ROLE_DESCRIPTION - ML Engineer with strong MLOps expertise on AWS, building, deploying, and monitoring scalable ML pipelines using services like SageMaker, S3, Lambda...Amazon Web Service
I-Flow
Malvern, PA4 days ago - NO VISA CANDIDATES / NO AGENCIES / NO THIRD PARTIES Position Summary We are seeking a highly skilled Data Engineer to design, build, and govern enterprise data platforms that support operational applications, business analytics, and AI initiatives. This role will develop...
- ...agentic architectures and infrastructure.Collaborate with senior engineers and data scientists in feature development, experiments, and... ...developmentSolid Python programming skills, experience with AWS cloud environment, Github pipeline, CI/CD concept and IT domain...Amazon Web ServiceFull time
Vanguard
Malvern, PA2 days ago - ...division, we are seeking a Machine Learning/ AI Engineer to design, build, and scale enterprise AI/... ...end-to-end AI/ML solutions, including data ingestion, feature engineering, model... ...native AI/ML applications and services using AWS technologies such as SageMaker, ECS, Lambda...Amazon Web ServiceVisa sponsorshipMalvern, PA4 days ago
- ...role partners closely with quantitative researchers, data scientists, and investment teams to engineer, deploy, and operate production-grade machine learning... ...practices and operating machine learning solutions on AWS SageMaker. Expertise in Python, cloud-native architectures...Amazon Web ServiceWork experience placement
Vanguard
Malvern, PA3 days ago - ...with RAG, structured output, and evaluation frameworks. AWS ML Engineering: Deploy models using SageMaker pipelines, ECS/ECR, Lambda; manage... ...: Deliver artifacts required by MDLC/MPLC (Model Documents, Data Dictionary, Monitoring Plan). Collaboration: Partner...Amazon Web Service
HCL Global Systems
Malvern, PA5 days ago - ...REQUIREMENT_CITY - Malvern, PA Role Name - AI Engineer EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python~Digital : Amazon Web Service(AWS) Cloud Computing~Digital : Node.js~AI Agents, Langchain ROLE_DESCRIPTION - Python, AWS, Node JS , Agentic AI, Langchain Experience...Amazon Web Service
eTeam
Malvern, PA3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AWS Data Engineer. Be the first to apply!
Related searches
- senior data center engineer Malvern, PA
- data developer Malvern, PA
- data engineer Malvern, PA
- data engineer analytics Malvern, PA
- finance data engineer Malvern, PA
- data science developer Malvern, PA
- data engineer machine learning Malvern, PA
- aws data engineer Malvern, PA
- data center engineer Malvern, PA
- aws devops Malvern, PA