Data Engineer
RIT Solutions
Senior Data Engineer
We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources, specifically bridging the gap between S3-based tables and Amazon Redshift. As we transition to a multi-technology data environment, you will be responsible for building resilient, enterprise-grade solutions that prioritize reliability and quick recovery.
Key Responsibilities
- Design & Development: Lead the design and development of custom AWS utilities that allow analysts to query disparate data sources (S3 and Redshift) with minimal effort and high productivity.
- Pipeline Engineering: Build and maintain scalable data pipelines using AWS Glue, PySpark, and Spark SQL to ingest and transform data across platforms.
- Resiliency & Scalability: Apply an "enterprise-grade" development mindset. Build solutions that are resilient, scalable, and capable of quick automated recovery in the event of failure.
- Utility Creation: Create tools to streamline the user experience for analysts, such as building automated processes to convert SQL outputs for multi-source tables.
- Infrastructure Ops: Utilize CloudFormation to manage and deploy infrastructure components.
Required Technical Skills
- AWS Core: Deep proficiency in AWS Glue, S3, Lambda, IAM, and Step Functions.
- Data Engineering: Expert-level knowledge of PySpark and Spark SQL.
- Querying & SQL: Extensive experience with Athena and Presto; strong SQL skills for complex data manipulation.
- Infrastructure as Code: Proven experience with CloudFormation for operational excellence.
- Development Mindset: Ability to write clean, maintainable code for automation and custom utility development.
- Dremio: Experience with Dremio implementation (reflections, views, using system tables).
Required Qualifications
- 6-8+ years of Data Engineering or IT experience.
- 4+ years of AWS cloud development experience.
- Expert Python programming skills.
- Advanced SQL development experience.
- Strong hands-on experience with Apache Spark / PySpark.
- Experience building cloud-native ETL pipelines.
- Strong understanding of distributed computing.
- Experience designing scalable data models.
- Experience with Git and CI/CD pipelines.
- Strong analytical and problem-solving skills.
- Build scalable and auto recoverable pipelines.
- Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
Nice to Have
- Tableau: Familiarity with Tableau for end-user data consumption.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Data Engineer in Malvern, PA vacancy
- ...Data Engineer Address:301 Lindenwood Dr Ste 330, Malvern, Pennsylvania Remote: 3 days onsite Hours: 8-5 Position Type: FULL-TIME KEY RESPONSIBILITIES Data Architecture & Pipeline Development Design, construct, and maintain robust data...SuggestedFull timeRemote work
MHK TECH INC
Malvern, PA18 hours ago $120k - $130k
...Attunity, Pyspark, AWS (S3, IAM, Lambda, Cloud Formation) Roles & Responsibilities • Design, develop, and maintain scalable ETL/data integration pipelines using AWS Glue, PySpark, and Python. • Develop and optimize data transformation workflows to process large...SuggestedTata Consultancy Services
Malvern, PA5 days ago- ...Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing....Suggested
Trinity Technology Solutions
Malvern, PA18 hours ago - ...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr....SuggestedContract workLocal areaImmediate start
Syntricate Technologies
Malvern, PA1 day ago - ...Data EngineerWe are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal candidate will be a hands...Suggested
InterSources
Malvern, PA6 hours ago - ...Data EngineerResponsibilitiesWrites ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real... ...consultant for the client.Educates and develops junior data engineers on the team while applying quality control to their work. Develops...
Kaav Inc.
Malvern, PA6 days ago - ...Job Title: Data Engineer Location: Malvern, PA Duration:6 Months Experience Required: 10 & Above Job Description: Technical skill sets : Python, Pyspark, AWS Lambda, S3,AWS glue, Kinesis,SQL, Apache Flink, Confluent Kafka, AI-LLM, Gen-AI , Java is optional...
I-Flow
Malvern, PA18 hours ago - ...Data EngineerClient: Vanguard Location: Malvern, PA or Charlotte, NC or Phoenix, AZ (3 days onsite) - need to be onsite day one Duration: 1 year Video Interview is okayResponsibilitiesData Engineers will be data mining, writing code, and creating tables/environments in...
RIT Solutions
Malvern, PA6 hours ago - ...Data engineer Malvern pa hybrid Assessment required Minimum 8-10 years of exp n * Location - Preferred Charlotte (but can be in Malvern) * pprox desired start - asap * Role - Level 4 Big Data Engineer, Python developer * required skillset - Python...Immediate start
RIT Solutions, Inc.
Malvern, PA18 hours ago - ...Trading Analytics And Strategy Engineer Client's Trading Analytics and Strategy (TAS) team works with our global trading desks to optimize... ...classes and passive as well as active mandates, to conduct data-driven analyses and build tools to help shape our trading strategy...
Samprasoft
Malvern, PA5 days ago - ...Big Data EngineerLocation: Malvern, PADuration: 12+ monthsInterview Process: Video interviewMust Have:Expert in PythonStrong expertise in SQL, Presto, HIVE, and SparkHands-on experience working with Trading/Investment dataStrong experience in building and orchestrating...
Veracity
Malvern, PA1 day ago - Job TitleRequired Skills: Python, Spark, AWS services, Glue, IACResponsibilitiesWork with Analytics and Business - derive requirements, build Engineering pipelines aligned to standards, trouble shoot data discrepancies and ensure quality data
Syntricate Technologies
Malvern, PA6 hours ago - ...Sr. Data EngineeringThe Sr. Data Engineering will manage the data pipeline development from data ingestion to transformation, implementing the Architect’s data models and ensuring high-quality data flow through the bronze, silver, and gold layers. This role involves leading...
Kaav Inc.
Malvern, PA6 hours ago - ...Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture). Infrastructure: CloudFormation/Terraform,... ...Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-level...
Saxon Global
Malvern, PA19 hours ago - ...AWS Data EngineerLocation: Malvern, PA, 19355Job Type: FulltimeSalary: $120,000-$130,000 a year+ benefitsKey Technical SkillsPythonAWS Glue (Glue ETL)PySparkAttunity / Qlik ReplicateAWS Services: Amazon S3, IAM, Lambda, CloudFormationRoles & ResponsibilitiesDesign, develop...
Diverse Lynx
Malvern, PA1 day ago - ...Job TitleWe are looking for strong AWS Data Engineers who are passionate about Cloud technology.ResponsibilitiesDesign and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.Implement...
Syntricate Technologies
Malvern, PA6 hours ago - ...Data Engineer Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of...
Samprasoft
Malvern, PA5 days ago - The Senior Manager, AI & Data Engineering leads teams responsible for building and operating reusable, governed data products and AI-ready semantic intelligence that enable enterprise data integration, activation, analytics, personalization, and decision-making across...Full timeFor contractorsWork at office
- Build Data Products That MatterDesign and build scalable data platforms, pipelines, and services that power analytics, business decisions... ..., scalability, reliability, observability, and security.Engineer Smarter with AILeverage AI-assisted development tools to accelerate...Full timeTemporary work
- ...will deliver over 1 billion gallons of propane from 1,800+ distribution points across the United States.Position SummaryThe Lead Data Engineer is a hands-on technical leader responsible for leading data engineering initiatives, managing a team of 1-3 direct reports, and...For contractors
- ...VSH). Learn more at . Do you want to help us build the DNA of tech.? Vishay Italy is currently seeking applicants for Head of Data Engineering & Analytics. In this position, you will be accountable for defining, governing, and evolving the enterprise‑wide data architecture...Full time
- DescriptionWe are looking for a Data Engineer to join a growing team in Conshohocken, Pennsylvania within the financial services industry. In this role, you will build and enhance modern data pipelines and warehouse structures that support reporting, analytics, and business...
- ...willing to consider candidates who live greater than 100 miles from the office for the remote option.)Job SummaryWe are seeking a Data Engineer to join our Data Product Engineering team responsible for managing and evolving the enterprise Data Lake that supports critical...Full timeWork at officeRemote workWorldwide
$82.1k - $172.4k
Job Title: Data EngineerJob Category: ScienceTime Type: Full timeMinimum Clearance Required to Start: TS/SCI with PolygraphEmployee Type... ...End: 12/31/2026 The Opportunity: We are looking for a Data Engineer/Scientist looking for new challenging problems. You will support...Full timeContract workWork experience placementFlexible hours- ...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...Permanent employmentFull timeH1b
- ...Data Engineer Mandatory Skills: Snowflake, Matillion, Python, ETL • Master's Degree in Computer Science, a related field or equivalent work experience. • Snowflake Certification is required. • 10+ years of experience as a data engineer or similar work experience....Work experience placement
Omni Inclusive
Collegeville, PA18 hours ago - ...Title: Data Engineer Location - West Chester, PA Onsite. Azure Senior Data Lead with Azure Data Factory (ADF),Azure Databricks, SQL, Oracle PL/SQL,Python Job Summary The Senior Technical Lead will be responsible for leading the technical...
United IT Solutions
West Chester, PA3 days ago - ...Data Engineer Valorem Reply is an award-winning digital transformation firm focused on delivering solutions around data-driven enterprise, IT modernization, customer experience, product transformation and digital workplace by leveraging the power of Microsoft technologies...
Reply
Chester, PA2 days ago - ...Duration: 6 months Requisition ID: 10934365 Note: PLEASE SUBMIT ONLY QUALITY PROFILES MATCHING TO THE JOB DESCRIPTION Role: Data Engineer Skills: Digital : BigData and Hadoop Ecosystem - MapR~Digital : Databricks~Digital : PySpark Experience Required: 6- 8 Years...
Q1 Technologies
West Chester, PA20 hours ago - ...Azure Data LeadLocation: West Chester, PA (Onsite)Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle... ...mentoring, and support to junior team members.Stay uptodate with the latest trends and technologies in data engineering and analytics....
InterSources
West Chester, PA6 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches
- data center engineer Malvern, PA
- etl data engineer Malvern, PA
- data engineer analytics Malvern, PA
- data engineer machine learning Malvern, PA
- finance data engineer Malvern, PA
- data engineer Malvern, PA
- data developer Malvern, PA
- senior data center engineer Malvern, PA
- data science developer Malvern, PA
- aws data engineer Malvern, PA