Data Engineer
RIT Solutions
Senior Data Engineer
We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources, specifically bridging the gap between S3-based tables and Amazon Redshift. As we transition to a multi-technology data environment, you will be responsible for building resilient, enterprise-grade solutions that prioritize reliability and quick recovery.
Key Responsibilities
- Design & Development: Lead the design and development of custom AWS utilities that allow analysts to query disparate data sources (S3 and Redshift) with minimal effort and high productivity.
- Pipeline Engineering: Build and maintain scalable data pipelines using AWS Glue, PySpark, and Spark SQL to ingest and transform data across platforms.
- Resiliency & Scalability: Apply an "enterprise-grade" development mindset. Build solutions that are resilient, scalable, and capable of quick automated recovery in the event of failure.
- Utility Creation: Create tools to streamline the user experience for analysts, such as building automated processes to convert SQL outputs for multi-source tables.
- Infrastructure Ops: Utilize CloudFormation to manage and deploy infrastructure components.
Required Technical Skills
- AWS Core: Deep proficiency in AWS Glue, S3, Lambda, IAM, and Step Functions.
- Data Engineering: Expert-level knowledge of PySpark and Spark SQL.
- Querying & SQL: Extensive experience with Athena and Presto; strong SQL skills for complex data manipulation.
- Infrastructure as Code: Proven experience with CloudFormation for operational excellence.
- Development Mindset: Ability to write clean, maintainable code for automation and custom utility development.
- Dremio: Experience with Dremio implementation (reflections, views, using system tables).
Required Qualifications
- 6-8+ years of Data Engineering or IT experience.
- 4+ years of AWS cloud development experience.
- Expert Python programming skills.
- Advanced SQL development experience.
- Strong hands-on experience with Apache Spark / PySpark.
- Experience building cloud-native ETL pipelines.
- Strong understanding of distributed computing.
- Experience designing scalable data models.
- Experience with Git and CI/CD pipelines.
- Strong analytical and problem-solving skills.
- Build scalable and auto recoverable pipelines.
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
Nice to Have
- Tableau: Familiarity with Tableau for end-user data consumption.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Data Engineer in Malvern, PA vacancy
- ...Job Title: Data Engineer Duration- Fulltime Permanent Location: Malvern, PA, 19355 / Charlotte, NC, 28262 (Hybrid /requiring 3 days onsite Mandate (Tuesday, Wednesday, and Thursday) (Local Only) Job Description: Data Engineer Python, Glue ETL, Attunity...SuggestedPermanent employmentFull timeLocal area
Q1 Technologies
Malvern, PA18 hours ago - ...Job Title: Data Engineer Location: Malvern, PA Duration:6 Months Experience Required: 10 & Above Job Description: Technical skill sets : Python, Pyspark, AWS Lambda, S3,AWS glue, Kinesis,SQL, Apache Flink, Confluent Kafka, AI-LLM, Gen-AI , Java is optional...Suggested
I-Flow
Malvern, PA16 hours ago - ...Data engineer Malvern pa hybrid Assessment required Minimum 8-10 years of exp n * Location - Preferred Charlotte (but can be in Malvern) * pprox desired start - asap * Role - Level 4 Big Data Engineer, Python developer * required skillset - Python...SuggestedImmediate start
RIT Solutions, Inc.
Malvern, PA16 hours ago - ...Data EngineerClient: Vanguard Location: Malvern, PA or Charlotte, NC or Phoenix, AZ (3 days onsite) - need to be onsite day one Duration: 1 year Video Interview is okayResponsibilitiesData Engineers will be data mining, writing code, and creating tables/environments in...Suggested
RIT Solutions
Malvern, PA4 hours ago - ...Data Engineer Responsibilities Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates...Suggested
Software Technology Inc
Malvern, PA4 days ago - ...Data EngineerData Architecture & Pipeline DevelopmentDesign, construct, and maintain robust data pipelines using Azure Data Factory... ...operationalize ML models using Azure Machine Learning. Support feature engineering, model versioning, and deployment pipelines. Utilize Azure ML...
MHK TECH INC
Malvern, PA4 hours ago - ...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr....Contract workLocal areaImmediate start
Syntricate Technologies
Malvern, PA1 day ago - ...Data EngineerWe are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal candidate will be a hands...
InterSources
Malvern, PA4 hours ago - ...Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing....
Trinity Technology Solutions
Malvern, PA16 hours ago - Job TitleRequired Skills: Python, Spark, AWS services, Glue, IACResponsibilitiesWork with Analytics and Business - derive requirements, build Engineering pipelines aligned to standards, trouble shoot data discrepancies and ensure quality data
Syntricate Technologies
Malvern, PA4 hours ago - ...Big Data Engineer - Malvern, PA Duration - 12+ months Interview Process - Video interview Must Have - Expert in Python Strong expertise in SQL, Presto, HIVE, and Spark Must have hands-on experience working with Trading/Investment data Strong...
Veracity
Malvern, PA4 days ago - ...Trading Analytics And Strategy Engineer Client's Trading Analytics and Strategy (TAS) team works with our global trading desks to optimize... ...classes and passive as well as active mandates, to conduct data-driven analyses and build tools to help shape our trading strategy...
Samprasoft
Malvern, PA5 days ago - ...Data EngineerPython, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture). Infrastructure: CloudFormation... ...data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-level production...
Saxon Global
Malvern, PA5 hours ago - ...Sr. Data EngineeringThe Sr. Data Engineering will manage the data pipeline development from data ingestion to transformation, implementing the Architect’s data models and ensuring high-quality data flow through the bronze, silver, and gold layers. This role involves leading...
Kaav Inc.
Malvern, PA5 hours ago - ...Job TitleWe are looking for strong AWS Data Engineers who are passionate about Cloud technology.ResponsibilitiesDesign and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.Implement...
Syntricate Technologies
Malvern, PA4 hours ago - ...Data Engineer Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of...
Samprasoft
Malvern, PA5 days ago - Build Data Products That MatterDesign and build scalable data platforms, pipelines, and services that power analytics, business decisions... ..., scalability, reliability, observability, and security.Engineer Smarter with AILeverage AI-assisted development tools to accelerate...Full timeTemporary work
- ...will deliver over 1 billion gallons of propane from 1,800+ distribution points across the United States.Position SummaryThe Lead Data Engineer is a hands-on technical leader responsible for leading data engineering initiatives, managing a team of 1-3 direct reports, and...For contractors
- ...VSH). Learn more at . Do you want to help us build the DNA of tech.? Vishay Italy is currently seeking applicants for Head of Data Engineering & Analytics. In this position, you will be accountable for defining, governing, and evolving the enterprise‑wide data architecture...Full time
- ...willing to consider candidates who live greater than 100 miles from the office for the remote option.)Job SummaryWe are seeking a Data Engineer to join our Data Product Engineering team responsible for managing and evolving the enterprise Data Lake that supports critical...Full timeWork at officeRemote workWorldwide
- DescriptionWe are looking for a Data Engineer to join a growing team in Conshohocken, Pennsylvania within the financial services industry. In this role, you will build and enhance modern data pipelines and warehouse structures that support reporting, analytics, and business...
$82.1k - $172.4k
Job Title: Data EngineerJob Category: ScienceTime Type: Full timeMinimum Clearance Required to Start: TS/SCI with PolygraphEmployee Type... ...End: 12/31/2026 The Opportunity: We are looking for a Data Engineer/Scientist looking for new challenging problems. You will support...Full timeContract workWork experience placementFlexible hours- ...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...Permanent employmentFull timeH1b
- ...Data EngineerMandatory Skills:Snowflake, Matillion, Python, ETL • Master's Degree in Computer Science, a related field or equivalent... ...Certification is required. • 10+ years of experience as a data engineer or similar work experience. • Four or more years or relevant experience...Work experience placement
Omni Inclusive
Collegeville, PA2 days ago - ...Data EngineerValorem Reply is an award-winning digital transformation firm focused on delivering solutions around data-driven enterprise... ...transform the way our clients do business.We are seeking a Data Engineer to design, build, and optimize modern data platforms for our...
Reply
Chester, PA1 day ago - ...Data Analyst Project name - Platform Transitions Program Level 3 (5-8 Years) Senior Responsibilities: Direct integration with internal partners to develop business strategies and achieve organizational goals. Brings structure to business requests, translates...
RIT Solutions
Wayne, PA4 days ago - ...Our client is currently seeking a DATA ENGINEERPermanent Fulltime Opportunity, W2 Only Conshohocken, PA POSITION SUMMARY: The"Data Engineer"plays a"crucial role"in designing, developing, and maintaining"scalable data pipelines"and architectures using"...Full timeWork at office
The Judge Group
Conshohocken, PA2 days ago - ...Azure Data Lead Location: West Chester, PA (Onsite) Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks,... ...mentoring, and support to junior team members. Stay uptodate with the latest trends and technologies in data engineering and analytics....
InterSources
West Chester, PA4 days ago - ...retail brands - QVC®, HSN®, Ballard Designs®, Frontgate®, Garnet Hill® and Grandin Road. Your Opportunity, Your Team The Data Engineer supports QVC and will collaborate with an Agile team to build data pipelines and models, exposing analytics through APIs and...Local areaRemote workRelocation package
QVC Group, Inc
West Chester, PA5 days ago - ...Duration: 6 months Requisition ID: 10934365 Note: PLEASE SUBMIT ONLY QUALITY PROFILES MATCHING TO THE JOB DESCRIPTION Role: Data Engineer Skills: Digital : BigData and Hadoop Ecosystem - MapR~Digital : Databricks~Digital : PySpark Experience Required: 6- 8 Years...
Q1 Technologies
West Chester, PA18 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches
- data center engineer Malvern, PA
- etl data engineer Malvern, PA
- data engineer analytics Malvern, PA
- data engineer machine learning Malvern, PA
- finance data engineer Malvern, PA
- data engineer Malvern, PA
- data developer Malvern, PA
- senior data center engineer Malvern, PA
- data science developer Malvern, PA
- aws data engineer Malvern, PA