Data Engineer

RIT Solutions, Inc.

Data Engineer

Hybrid Malvern, P

2 OPENINGS

SSESSMENT REQUIRED

Senior Data Engineer

Job Summary

We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources, specifically bridging the gap between S3-based tables and Amazon Redshift. As we transition to a multi-technology data environment, you will be responsible for building resilient, enterprise-grade solutions that prioritize reliability and quick recovery.

Key Responsibilities

Design & Development: Lead the design and development of custom AWS utilities that allow analysts to query disparate data sources (S3 and Redshift) with minimal effort and high productivity.
Pipeline Engineering: Build and maintain scalable data pipelines using AWS Glue, PySpark, and Spark SQL to ingest and transform data across platforms.
Resiliency & Scalability: Apply an "enterprise-grade " development mindset. Build solutions that are resilient, scalable, and capable of quick automated recovery in the event of failure.
Utility Creation: Create tools to streamline the user experience for analysts, such as building automated processes to convert SQL outputs for multi-source tables.
Infrastructure Ops: Utilize CloudFormation to manage and deploy infrastructure components.

Required Technical Skills

WS Core: Deep proficiency in AWS Glue, S3, Lambda, IAM, and Step Functions.
Data Engineering: Expert-level knowledge of PySpark and Spark SQL.
Querying & SQL: Extensive experience with Athena and Presto; strong SQL skills for complex data manipulation.
Infrastructure as Code: Proven experience with CloudFormation for operational excellence.
Development Mindset: Ability to write clean, maintainable code for automation and custom utility development.
Dremio: Experience with Dremio implementation (reflections, views, using system tables).

Required Qualifications

6-8+ years of Data Engineering or IT experience.
4+ years of AWS cloud development experience.
Expert Python programming skills.
dvanced SQL development experience.
Strong hands-on experience with Apache Spark / PySpark.
Experience building cloud-native ETL pipelines.
Strong understanding of distributed computing.
Experience designing scalable data models.
Experience with Git and CI/CD pipelines.
Strong analytical and problem-solving skills.
Build scalable and auto recoverable pipelines.
Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.

Nice to Have

Tableau: Familiarity with Tableau for end-user data consumption.

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Data Engineer in Malvern, PA vacancy

Data Engineer
...Data Engineer As a Data Engineer, you’ll build sophisticated solutions that support a wide range of advanced data driven capabilities. Provides advanced data solutions by using software to process, store, and serve data to others. Tests data quality and optimizes data...
Suggested
Samprasoft
Malvern, PA
3 days ago
Data Engineer/Developer
...Data Engineer Troubleshoots software and processes for data consistency and integrity. Integrates complex data from a variety of sources for business partners to generate insight and make decisions. Translates business specifications into design specifications and...
Suggested
Samprasoft
Malvern, PA
3 days ago
Data Engineer
...Data Engineer Duties And Responsibilities Data engineers create and maintain an optimal data infrastructure. They combine raw information to create machine-readable formats. These are the major duties and responsibilities of a data engineer: Putting...
Suggested
3B Staffing LLC
Malvern, PA
2 days ago
Data Engineer
...Data Engineer At least 5 years of Data Engineering experience with AWS platform using robust technical skills in Python, SQL, PySpark, Glue ETL, AWS Lambda, S3, Redshift, AWS Airflow. Robust problem-solving skills, communication skills, and ability to learn quickly...
Suggested
Randstad
Malvern, PA
2 days ago
Big data engineer
$39.38 - $44.38 per hour
...Responsibilities Design, develop, and maintain scalable data pipelines and data products on the AWS data platform using PySpark... ...business requirements. Collaborate with product owners, data engineers, and stakeholders to deliver data products, troubleshoot issues...
Suggested
Hourly pay
Temporary work
Work experience placement
Randstad
Malvern, PA
4 days ago
Big Data Engineer
...Big Data Engineer Hybrid - Malvern, P Responsibilities Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data...
RIT Solutions, Inc.
Malvern, PA
2 days ago
Data Engineer
...exceptional professionals for this role. Join us in a high-impact, high-visibility role where you will pioneer world-class Data & AI engineering solutions, building the next generation of intelligent, agent-driven systems that power real-time business decisions. We are...
Vanguard
Malvern, PA
2 days ago
Big Data Engineer
...Responsibilities ~ Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required...
Procyon TS
Malvern, PA
1 day ago
Big Data Engineer
...Data Engineer Responsibilities Design, develop, and maintain scalable data pipelines and data products on the AWS data platform using PySpark, SQL, and cloud-native services. Build, optimize, and support ETL/ELT processes, ensuring high-quality, reliable, and performant...
Randstad
Malvern, PA
1 day ago
Senior Data Engineer
...Senior Data Engineer – 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing, building, and optimizing robust data solutions. The ideal candidate must possess top-tier skills in Python, AWS services,...
Samprasoft
Malvern, PA
3 days ago
SENIOR DATA ENGINEER
...ROLE SENIOR DATA ENGINEER INDUSTRY Pharma, Development & Manufacturing DURATION CONTRACT, probable extension LOCATION REMOTE (off-shore or near-shore), Malvern, PA 19355 INTERVIEW MS TEAMS conversation the candidate will be required...
Contract work
Immediate start
Remote work
3B Staffing LLC
Malvern, PA
2 days ago
Senior Data Engineer
...Senior Data Engineer Assessment required hybrid - Malvern, PA Technical Stack & Requirements: Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks, Sagemaker. Infrastructure: CloudFormation...
RIT Solutions
Malvern, PA
1 day ago
Senior Data Engineer
...Request ID: 98311-1 Title: Data Engineer Locations : Malvern, PA / Charlotte, NC - Hybrid Duration: 6 Months Pay Range: $65 - $70/Hour on W2/C2C (All inclusive) Introduction We are seeking a highly skilled professional to join our team. The ideal...
Work experience placement
Work at office
Immediate start
Flexible hours
Artech
Malvern, PA
3 days ago
AWS Data Engineer
$120k - $130k
...Position: AWS Data Engineer Location: Malvern, PA, 19355 Job Type: Fulltime Salary: $120,000-$130,000 a year+ benefits Key Technical Skills: Python AWS Glue (Glue ETL) PySpark Attunity / Qlik Replicate AWS...
Full time
Diverse Lynx
Malvern, PA
3 days ago
AWS Data Engineer
...Role Description: We are looking for strong AWS Data Engineers who are passionate about Cloud technology. Your work will be to: Design and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics...
Syntricate Technologies
Malvern, PA
11 hours ago
AWS Data Engineer
...Overview: Job Description - AWS Data Engineer (Python, AWS Glue, PySpark & Attunity) Job Title AWS Data Engineer - Python, AWS Glue, PySpark & Attunity Job Summary We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS...
Purple Drive
Malvern, PA
2 days ago
Application Engineer III - AWS Data Engineer
Application Engineer III - AWS Data EngineerAbout the RoleJoin Vanguard's mission to deliver investment success for all investors. We're seeking an AWS Data Engineer to help shape and evolve an enterprise data platform that manages assets and cash flow data across our lines...
Full time
Vanguard
Malvern, PA
1 day ago
Analytics Engineer
...AXA XL, a global leader in insurance, seeks a data-driven Pricing Analytics & Models professional to enable pricing actuaries and underwriters with robust data flows and insightful BI dashboards. The role focuses on building scalable data models, dashboards, and reporting...
AXA Group
Exton, PA
2 days ago
Lead Data Engineer
...will deliver over 1 billion gallons of propane from 1,800+ distribution points across the United States.Position SummaryThe Lead Data Engineer is a hands-on technical leader responsible for leading data engineering initiatives, managing a team of 1-3 direct reports, and...
For contractors
UGI Corporation
King of Prussia, PA
2 days ago
Head of Data Engineering & Analytics
...VSH). Learn more at . Do you want to help us build the DNA of tech.? Vishay Italy is currently seeking applicants for Head of Data Engineering & Analytics. In this position, you will be accountable for defining, governing, and evolving the enterprise‑wide data architecture...
Full time
Vishay
Malvern, PA
5 days ago
Data Engineer
DescriptionWe are looking for a Data Engineer to drive the design and delivery of scalable data solutions that support critical business objectives within the oil and gas sector. This position is based in King of Prussia, Pennsylvania, and combines technical execution with...
Robert Half
King of Prussia, PA
20 hours ago
Data Engineer
...with six leading retail brands - QVC, HSN, Ballard Designs, Frontgate, Garnet Hill and Grandin Road.Your Opportunity, Your TeamThe Data Engineer supports QVC and will collaborate with an Agile team to build data pipelines and models, exposing analytics through APIs and...
Full time
Local area
Remote work
Relocation package
QVC
West Chester, PA
1 day ago
Big Data Engineer
...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...
Permanent employment
Full time
H1b
Sonsoft
King of Prussia, PA
3 days ago
Data Senior Engineer
Data Engineer - AIDay to Day job Duties: (what this person will do on a daily/weekly basis)Design and build scalable ingestion, transformation, and processing pipelines for AI/ML.Develop and maintain data storage, such as vector databases, that support Retrieval-Augmented...
NTT DATA
King of Prussia, PA
3 days ago
Snowflake Data Engineer
...Role Overview: This role is for a Data Engineer focused on designing, implementing, and optimizing robust ETL/ELT pipelines and complex data models primarily using PySpark and Snowflake. The engineer will be responsible for ensuring high data quality, integrity, and...
Prophecy Technologies
West Chester, PA
4 days ago
Data Engineer
...Job Summary We are seeking a Data Engineer to design, develop, and maintain scalable data pipelines that enable efficient data integration, processing, and analytics across enterprise platforms. This role is responsible for building ETL solutions, optimizing databases...
Job sharing
3 days per week
Compunnel
Wayne, PA
3 days ago
Data engineer
...Data Analyst Video Interview is okay Project name - Platform Transitions Program Level 3 (5-8 Years) Senior Responsibilities: Direct integration with internal partners to develop business strategies and achieve organizational goals...
RIT Solutions, Inc.
Wayne, PA
4 days ago
Data Engineer
...Data Engineer Location - West Chester, PA Onsite. Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python Job Summary: The Senior Technical Lead will be responsible for leading the technical aspects of projects...
United IT
West Chester, PA
2 days ago
Data Engineer
...Pipeline Development Design and implement robust ETL/ELT pipelines using PySpark and SQL to ingest data from diverse sources including APIs, flat files, and relational databases. Data Modeling Develop and manage complex data models (e.g., Star/Snowflake schemas...
Diverse Lynx
West Chester, PA
20 hours ago
Data Engineer
...Data Engineer | Wayne, Pennsylvania, United States Data Engineer - Contract to Hire (Hybrid, No Sponsorship) Location: Wayne, PA | Plano, TX | Charlotte, NC | Scottsdale, AZ Job Type: Contract to Hire (Hybrid, 3 days on-site/week) Eligibility: US Citizens/Green Card Holders...
Permanent employment
Contract work
Local area
3 days per week
Indotronix International Corporation
Wayne, PA
3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!