Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Databricks AWS Data Engineer

Compunnel

Job Summary The Databricks AWS Data Engineer will develop and support a new data environment from the ground up, focusing primarily on data pipelines and cloud-based data engineering solutions. The role requires strong development experience with Python, PySpark, and SQL, along with hands-on AWS expertise, particularly AWS Glue and Lambda. The engineer will contribute to building scalable data solutions that support financial data and analytics initiatives. Key Responsibilities

  • Develop and maintain data pipelines using Python, PySpark, and SQL.
  • Contribute to building a new data environment from the ground up.
  • Design and implement data engineering solutions using AWS services.
  • Develop and support data processing workflows using AWS Glue and AWS Lambda.
  • Build, maintain, and optimize data pipelines to support data processing and analytics requirements.
Required Qualifications
  • Strong hands-on experience with Python and PySpark.
  • Strong SQL development skills.
  • Mandatory experience with AWS services, particularly AWS Glue and AWS Lambda.
  • Experience developing and maintaining data pipelines.
  • Ability to participate in technical discussions and whiteboarding exercises. Compunnel

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Databricks AWS Data Engineer in Malvern, PA vacancy
  •  ...Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture)...  ...diverse legacy and modern sources, engineer high-performance scalable pipelines,...  .... Experience with Snowflake or Databricks. Knowledge of Apache Iceberg, Delta... 
    Amazon Web Service
    Databricks

    V-Soft Consulting Group, Inc

    Malvern, PA
    24 days ago
  • $120.38k - $192.6k

     ...Requisition #: 76603The Role at a GlanceAs a Senior Lead Risk Data Engineer and Architect, you will architect and lead the design and...  ...management.   Working on a centralized cloud data platform (AWS, Databricks or Dataiku), you'll design scalable data pipelines, implement... 
    Amazon Web Service
    Databricks
    Work experience placement
    Work at office
    3 days per week

    Lincoln Financial Group

    Radnor, PA
    17 hours ago
  •  ...Databricks Architect Malvern, PA Must Have Technical/Functional Skills: AWS Data Services – S3, Redshift, Glue, Lambda, Athena. Data Modeling – Dimensional Modeling (Star/Snowflake), Data Warehouse Design. Data Lake Architecture – Amazon S3, Lake Formation... 
    Amazon Web Service
    Databricks

    United IT

    Malvern, PA
    1 day ago
  •  ...analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources...  ...to generate insight and make decisions. Experience building AWS cloud architecture and supporting services and technologies (Eg:... 
    Amazon Web Service

    Brains Technology Solutions, Inc.

    Malvern, PA
    17 hours ago
  •  ...architects They have about six platforms (Databricks and AWS being the biggest two, Snowflake and...  ...business stakeholders, architects, and engineering teams to define use cases, translating...  ...into actionable platform and data product capabilities, and ensure successful... 
    Amazon Web Service
    Databricks
    Contract work
    Local area

    The Judge Group

    Malvern, PA
    3 days ago
  •  ...Responsibilities Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required Skills : Data Warehouse Basic Qualification :... 
    Amazon Web Service

    Kaav Inc.

    Malvern, PA
    1 day ago
  • Responsibilities Pulling data from GitHub and Jira transforming it for input into our data warehouse. Qualifications Familiarity with AWS including lambda, event bridge, s3. Proficient in typescript Datalake / Data warehouse experience is a big plus. Required Skills :... 
    Amazon Web Service

    Brains Technology Solutions, Inc.

    Malvern, PA
    1 day ago
  • Job Title: Senior Data Engineer Location: Malvern, PA Onsite Duration:6 Months Experience Required: 10 & Above EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python, AWS- S3, AWS Lambdas, AWS Glue, SQL, DynamoDB , Kafka/ Kinesis , pySpark ROLE_DESCRIPTION - Overview: We are seeking... 
    Amazon Web Service
    Local area

    I-Flow

    Malvern, PA
    1 day ago
  • Data Engineer Location: Malvern, PA - Hybrid Important Submission Requirement Candidates must take and pass a technical assessment before...  ...Required Skills Advanced Python Advanced SQL Strong experience with AWS Services , including: AWS Glue Amazon S3 AWS Lambda Amazon... 
    Amazon Web Service

    3B Staffing LLC

    Malvern, PA
    1 day ago
  • Job Summary The AWS Data Engineer will design, develop, and implement scalable, high-performance data ingestion and transformation pipelines using AWS services and Apache Spark. The role involves integrating complex datasets from multiple sources, developing advanced SQL... 
    Amazon Web Service

    Compunnel

    Malvern, PA
    1 day ago
  • Machine Learning Engineer We are seeking an experienced Machine Learning Engineer to join our...  ...responsible for developing and optimizing complex data pipelines, integrating model pipelines,...  ...experience designing ETL pipelines using AWS services (e.g., Glue, SageMaker).... 
    Amazon Web Service
    Work experience placement

    Samprasoft

    Malvern, PA
    1 day ago
  • Data Lake 2.0 Developer Data Lake 2.0 is a multi-year program aimed at modernizing legacy...  ...that will be in use by Analysts and Data Engineers. The deliverables will help data...  ...integrations between existing Vanguard tools, AWS components, and vendor products. Skillset... 
    Amazon Web Service

    Samprasoft

    Malvern, PA
    1 day ago
  • Data Engineer Troubleshoots software and processes for data consistency and integrity. Integrates complex data from a variety of sources for...  ...required skillsets: Python Java Rest Services - Spring /Spring Boot AWS Cloud Services CloudFront S3 ECS Kinesis Lambda Step functions... 
    Amazon Web Service

    Samprasoft

    Malvern, PA
    1 day ago
  • Data Engineer Duties And Responsibilities Data engineers create and maintain an optimal data infrastructure. They combine raw information to...  ...manual procedures. Building necessary infrastructure using AWS and SQL technologies. This will enable effective data extraction... 
    Amazon Web Service

    3B Staffing LLC

    Malvern, PA
    1 day ago
  • Senior Data Engineer - 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing...  .... The ideal candidate must possess top-tier skills in Python, AWS services, API development, and TypeScript, and have significant... 
    Amazon Web Service

    Samprasoft

    Malvern, PA
    1 day ago
  •  ...leadership for Vanguard's MarTech Data Platform ecosystem, including...  ...the platform landscape.Drive engineering best practices across data...  ...architectures, preferably AWS.Deep expertise in data engineering...  ....Hands-on experience with Databricks, Delta Lake, Customer Lake... 
    Amazon Web Service
    Databricks
    Full time

    Vanguard

    Malvern, PA
    1 day ago
  • Role : Data Engineer with Python Location: Malvern, PA (Onsite) Contract Job Summary We are seeking a Data Engineer with strong Python expertise...  ...and data modeling. Familiarity with cloud platforms such as AWS, Azure, or GCP. Experience with version control tools like Git.... 
    Amazon Web Service
    Contract work

    AceStack LLC

    Malvern, PA
    3 days ago
  • Purple Drive AWS Data Engineer - Python, AWS Glue, PySpark & Attunity We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS Glue, PySpark, Attunity (Qlik Replicate), and AWS Cloud to design, develop, and maintain scalable enterprise data integration... 
    Amazon Web Service

    Purple Drive

    Malvern, PA
    1 day ago
  • Job Title: Data Engineering Lead Location: Malvern, PA Contract Job Description: Skills: Java| AWS | Python | PySpark | Event-Driven Pipelines | Data Architecture Summary We are seeking an experienced Tech lead- Data Engineer (15+ years) with a strong background in Java... 
    Amazon Web Service
    Contract work

    AceStack LLC

    Malvern, PA
    1 day ago
  • Data Engineer Location - West Chester, PA Onsite. Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python Job Summary: The Senior Technical Lead will be responsible for leading the technical aspects of projects related to... 
    Databricks

    United IT

    West Chester, PA
    1 day ago
  • Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. • Deep technical knowledge - including... 
    Amazon Web Service

    Trinity Technology Solutions

    Malvern, PA
    1 day ago
  •  ...Request ID: 110362-1 Title: Databricks Solution Architect...  ...delivering enterprise-scale data platforms, cloud modernization...  ...and collaborate with business, engineering, and enterprise stakeholders...  ...platforms: Azure Databricks, AWS S3, Glue, EMR, Redshift ~ Strong... 
    Amazon Web Service
    Databricks
    Work experience placement
    Immediate start

    Artech LLC

    Malvern, PA
    24 days ago
  • $71.96 - $75.75 per hour

     ...6 - 75.75/hr. Role Name - Databricks Solution Architect Job Description...  ...delivering enterprise-scale data platforms, cloud...  ...and collaborate with business, engineering, and enterprise stakeholders...  ...Databricks, ADF, ADLS, Synapse, AWS S3, Glue, EMR, Redshift •Data... 
    Amazon Web Service
    Databricks

    Varite, Inc

    Malvern, PA
    24 days ago
  •  ...Description ROLE_DESCRIPTION - " Databricks: Lakehouse, Delta Lake, Unity Catalog, DLT, Databricks SQL, MLflow " Big Data: Spark, PySpark, Scala, Kafka, Streaming Architectures...  ...: Azure Databricks, ADF, ADLS, Synapse, AWS S3, Glue, EMR, Redshift " Data... 
    Amazon Web Service
    Databricks

    Diverse Lynx

    Malvern, PA
    24 days ago
  •  ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building...  ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with...  ..., and citations.Design and build AWS source-to-product pipelines and agent-facing... 
    Amazon Web Service
    For contractors
    Work experience placement
    3 days per week

    Lincoln Financial Group

    Villanova, PA
    17 hours ago
  • Reporting to the Manager - Data Architecture & Engineering, the Senior Developer - Data Engineering at Quaker Houghton will hands on in technology...  ...native data engineering technologies (e.g. Snowflake, Databricks) • Experience in a senior management role, with... 
    Databricks
    Work experience placement

    Quaker Houghton

    Conshohocken, PA
    1 day ago
  • Data Engineer, Specialist The Vanguard Group, Inc. - Philadelphia / Apex This role will be hybrid on site 3x per week in any of the following...  ...Integration Machine Learning Data Governance SQL, Python, ETL AWS Services: Glue ETL, Lambda, Redshift, S3, etc AWS infrastructure... 
    Amazon Web Service
    Contract work

    Saxon Global

    Wayne, PA
    1 day ago
  •  ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building...  ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with...  ...models using SQL, Python, and approved AWS services.Build and maintain FastAPI and Pydantic... 
    Amazon Web Service
    Contract work
    Work experience placement
    3 days per week

    Lincoln Financial Group

    Radnor, PA
    8 hours ago
  •  ...career opportunity now! Position Overview: We are looking for a Data Engineer/Scientist looking for new challenging problems. You will...  ...containers, (e.g. Docker, Docker Compose), cloud services (e.g. AWS), tools for distributed computing (e.g. Spark, Pyspark) Active... 
    Amazon Web Service
    Temporary work
    Work experience placement
    Work at office
    Visa sponsorship
    Relocation package
    Flexible hours

    The Stratagem Group

    King of Prussia, PA
    1 day ago
  • $120.38k - $192.6k

     ...: is not available for this opportunity.Requisition #: 76901The Role at a GlanceThe Lead Data Engineer provides technical leadership for Lincoln’s Data & Analytics platform on AWS. This role designs, builds, and maintains scalable data pipelines and optimizes data systems... 
    Amazon Web Service
    Work experience placement
    3 days per week

    Lincoln Financial Group

    Radnor, PA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Databricks AWS Data Engineer. Be the first to apply!