Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Data Engineer

RIT Solutions

Senior Data Engineer

We are seeking a talented Senior Data Engineer to join our infrastructure support team. This role is focused on designing and developing robust AWS utilities that enable our data engineers and analysts to seamlessly query across multiple data sources, specifically bridging the gap between S3-based tables and Amazon Redshift. As we transition to a multi-technology data environment, you will be responsible for building resilient, enterprise-grade solutions that prioritize reliability and quick recovery.

Key Responsibilities
  • Design & Development: Lead the design and development of custom AWS utilities that allow analysts to query disparate data sources (S3 and Redshift) with minimal effort and high productivity.
  • Pipeline Engineering: Build and maintain scalable data pipelines using AWS Glue, PySpark, and Spark SQL to ingest and transform data across platforms.
  • Resiliency & Scalability: Apply an "enterprise-grade" development mindset. Build solutions that are resilient, scalable, and capable of quick automated recovery in the event of failure.
  • Utility Creation: Create tools to streamline the user experience for analysts, such as building automated processes to convert SQL outputs for multi-source tables.
  • Infrastructure Ops: Utilize CloudFormation to manage and deploy infrastructure components.
Required Technical Skills
  • AWS Core: Deep proficiency in AWS Glue, S3, Lambda, IAM, and Step Functions.
  • Data Engineering: Expert-level knowledge of PySpark and Spark SQL.
  • Querying & SQL: Extensive experience with Athena and Presto; strong SQL skills for complex data manipulation.
  • Infrastructure as Code: Proven experience with CloudFormation for operational excellence.
  • Development Mindset: Ability to write clean, maintainable code for automation and custom utility development.
  • Dremio: Experience with Dremio implementation (reflections, views, using system tables).
Required Qualifications
  • 6-8+ years of Data Engineering or IT experience.
  • 4+ years of AWS cloud development experience.
  • Expert Python programming skills.
  • Advanced SQL development experience.
  • Strong hands-on experience with Apache Spark / PySpark.
  • Experience building cloud-native ETL pipelines.
  • Strong understanding of distributed computing.
  • Experience designing scalable data models.
  • Experience with Git and CI/CD pipelines.
  • Strong analytical and problem-solving skills.
  • Build scalable and auto recoverable pipelines.
  • Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
Nice to Have
  • Tableau: Familiarity with Tableau for end-user data consumption.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Data Engineer in Malvern, PA vacancy
  •  ...Data Engineer Address:301 Lindenwood Dr Ste 330, Malvern, Pennsylvania Remote: 3 days onsite Hours: 8-5 Position Type: FULL-TIME KEY RESPONSIBILITIES Data Architecture & Pipeline Development Design, construct, and maintain robust data... 
    Suggested
    Full time
    Remote work

    MHK TECH INC

    Malvern, PA
    18 hours ago
  • $120k - $130k

     ...Attunity, Pyspark, AWS (S3, IAM, Lambda, Cloud Formation) Roles & Responsibilities • Design, develop, and maintain scalable ETL/data integration pipelines using AWS Glue, PySpark, and Python. • Develop and optimize data transformation workflows to process large... 
    Suggested

    Tata Consultancy Services

    Malvern, PA
    5 days ago
  •  ...Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing.... 
    Suggested

    Trinity Technology Solutions

    Malvern, PA
    18 hours ago
  •  ...Position- Data Engineer Duration-Contract Location- Malvern PA (Day 1 onsite) Local Only JD Aws Redshift is mandatory With sql, python If someone worked with Apache Iceberg would be great Regards, Pallavi Verma Sr.... 
    Suggested
    Contract work
    Local area
    Immediate start

    Syntricate Technologies

    Malvern, PA
    1 day ago
  •  ...Data EngineerWe are seeking a highly experienced Data Engineer with 5+ years of experience. This role is critical to hitting product rollout deadlines, as the team's work is a hard, direct dependency for other product feature rollouts. The ideal candidate will be a hands... 
    Suggested

    InterSources

    Malvern, PA
    6 hours ago
  •  ...Data EngineerResponsibilitiesWrites ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real...  ...consultant for the client.Educates and develops junior data engineers on the team while applying quality control to their work. Develops... 

    Kaav Inc.

    Malvern, PA
    6 days ago
  •  ...Job Title: Data Engineer Location: Malvern, PA Duration:6 Months Experience Required: 10 & Above Job Description: Technical skill sets : Python, Pyspark, AWS Lambda, S3,AWS glue, Kinesis,SQL, Apache Flink, Confluent Kafka, AI-LLM, Gen-AI , Java is optional... 

    I-Flow

    Malvern, PA
    18 hours ago
  •  ...Data EngineerClient: Vanguard Location: Malvern, PA or Charlotte, NC or Phoenix, AZ (3 days onsite) - need to be onsite day one Duration: 1 year Video Interview is okayResponsibilitiesData Engineers will be data mining, writing code, and creating tables/environments in... 

    RIT Solutions

    Malvern, PA
    6 hours ago
  •  ...Data engineer Malvern pa hybrid Assessment required Minimum 8-10 years of exp n * Location - Preferred Charlotte (but can be in Malvern) * pprox desired start - asap * Role - Level 4 Big Data Engineer, Python developer * required skillset - Python... 
    Immediate start

    RIT Solutions, Inc.

    Malvern, PA
    18 hours ago
  •  ...Trading Analytics And Strategy Engineer Client's Trading Analytics and Strategy (TAS) team works with our global trading desks to optimize...  ...classes and passive as well as active mandates, to conduct data-driven analyses and build tools to help shape our trading strategy... 

    Samprasoft

    Malvern, PA
    5 days ago
  •  ...Big Data EngineerLocation: Malvern, PADuration: 12+ monthsInterview Process: Video interviewMust Have:Expert in PythonStrong expertise in SQL, Presto, HIVE, and SparkHands-on experience working with Trading/Investment dataStrong experience in building and orchestrating... 

    Veracity

    Malvern, PA
    1 day ago
  • Job TitleRequired Skills: Python, Spark, AWS services, Glue, IACResponsibilitiesWork with Analytics and Business - derive requirements, build Engineering pipelines aligned to standards, trouble shoot data discrepancies and ensure quality data

    Syntricate Technologies

    Malvern, PA
    6 hours ago
  •  ...Sr. Data EngineeringThe Sr. Data Engineering will manage the data pipeline development from data ingestion to transformation, implementing the Architect’s data models and ensuring high-quality data flow through the bronze, silver, and gold layers. This role involves leading... 

    Kaav Inc.

    Malvern, PA
    6 hours ago
  •  ...Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture). Infrastructure: CloudFormation/Terraform,...  ...Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-level... 

    Saxon Global

    Malvern, PA
    19 hours ago
  •  ...AWS Data EngineerLocation: Malvern, PA, 19355Job Type: FulltimeSalary: $120,000-$130,000 a year+ benefitsKey Technical SkillsPythonAWS Glue (Glue ETL)PySparkAttunity / Qlik ReplicateAWS Services: Amazon S3, IAM, Lambda, CloudFormationRoles & ResponsibilitiesDesign, develop... 

    Diverse Lynx

    Malvern, PA
    1 day ago
  •  ...Job TitleWe are looking for strong AWS Data Engineers who are passionate about Cloud technology.ResponsibilitiesDesign and Develop Data Pipelines: Create robust pipelines to ingest, process, and transform data, ensuring it is ready for analytics and reporting.Implement... 

    Syntricate Technologies

    Malvern, PA
    6 hours ago
  •  ...Data Engineer Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of... 

    Samprasoft

    Malvern, PA
    5 days ago
  • The Senior Manager, AI & Data Engineering leads teams responsible for building and operating reusable, governed data products and AI-ready semantic intelligence that enable enterprise data integration, activation, analytics, personalization, and decision-making across... 
    Full time
    For contractors
    Work at office

    Vanguard

    Malvern, PA
    1 hour ago
  • Build Data Products That MatterDesign and build scalable data platforms, pipelines, and services that power analytics, business decisions...  ..., scalability, reliability, observability, and security.Engineer Smarter with AILeverage AI-assisted development tools to accelerate... 
    Full time
    Temporary work

    Vanguard

    Malvern, PA
    3 days ago
  •  ...will deliver over 1 billion gallons of propane from 1,800+ distribution points across the United States.Position SummaryThe Lead Data Engineer is a hands-on technical leader responsible for leading data engineering initiatives, managing a team of 1-3 direct reports, and... 
    For contractors

    UGI Corporation

    King of Prussia, PA
    4 days ago
  •  ...VSH). Learn more at . Do you want to help us build the DNA of tech.? Vishay Italy is currently seeking applicants for Head of Data Engineering & Analytics. In this position, you will be accountable for defining, governing, and evolving the enterprise‑wide data architecture... 
    Full time

    Vishay

    Malvern, PA
    2 days ago
  • DescriptionWe are looking for a Data Engineer to join a growing team in Conshohocken, Pennsylvania within the financial services industry. In this role, you will build and enhance modern data pipelines and warehouse structures that support reporting, analytics, and business... 

    Robert Half

    Conshohocken, PA
    3 days ago
  •  ...willing to consider candidates who live greater than 100 miles from the office for the remote option.)Job SummaryWe are seeking a Data Engineer to join our Data Product Engineering team responsible for managing and evolving the enterprise Data Lake that supports critical... 
    Full time
    Work at office
    Remote work
    Worldwide

    Comcast

    West Chester, PA
    6 hours ago
  • $82.1k - $172.4k

    Job Title: Data EngineerJob Category: ScienceTime Type: Full timeMinimum Clearance Required to Start: TS/SCI with PolygraphEmployee Type...  ...End: 12/31/2026 The Opportunity: We are looking for a Data Engineer/Scientist looking for new challenging problems. You will support... 
    Full time
    Contract work
    Work experience placement
    Flexible hours

    CACI International

    King of Prussia, PA
    3 days ago
  •  ...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed... 
    Permanent employment
    Full time
    H1b

    Sonsoft

    King of Prussia, PA
    6 hours ago
  •  ...Data Engineer Mandatory Skills: Snowflake, Matillion, Python, ETL • Master's Degree in Computer Science, a related field or equivalent work experience. • Snowflake Certification is required. • 10+ years of experience as a data engineer or similar work experience.... 
    Work experience placement

    Omni Inclusive

    Collegeville, PA
    18 hours ago
  •  ...Title: Data Engineer Location - West Chester, PA Onsite. Azure Senior Data Lead with Azure Data Factory (ADF),Azure Databricks, SQL, Oracle PL/SQL,Python Job Summary The Senior Technical Lead will be responsible for leading the technical... 

    United IT Solutions

    West Chester, PA
    3 days ago
  •  ...Data Engineer Valorem Reply is an award-winning digital transformation firm focused on delivering solutions around data-driven enterprise, IT modernization, customer experience, product transformation and digital workplace by leveraging the power of Microsoft technologies... 

    Reply

    Chester, PA
    2 days ago
  •  ...Duration: 6 months Requisition ID: 10934365 Note: PLEASE SUBMIT ONLY QUALITY PROFILES MATCHING TO THE JOB DESCRIPTION Role: Data Engineer Skills: Digital : BigData and Hadoop Ecosystem - MapR~Digital : Databricks~Digital : PySpark Experience Required: 6- 8 Years... 

    Q1 Technologies

    West Chester, PA
    20 hours ago
  •  ...Azure Data LeadLocation: West Chester, PA (Onsite)Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle...  ...mentoring, and support to junior team members.Stay uptodate with the latest trends and technologies in data engineering and analytics.... 

    InterSources

    West Chester, PA
    6 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!