Databricks AWS Data Engineer
Compunnel
Job Summary The Databricks AWS Data Engineer will develop and support a new data environment from the ground up, focusing primarily on data pipelines and cloud-based data engineering solutions. The role requires strong development experience with Python, PySpark, and SQL, along with hands-on AWS expertise, particularly AWS Glue and Lambda. The engineer will contribute to building scalable data solutions that support financial data and analytics initiatives. Key Responsibilities
- Develop and maintain data pipelines using Python, PySpark, and SQL.
- Contribute to building a new data environment from the ground up.
- Design and implement data engineering solutions using AWS services.
- Develop and support data processing workflows using AWS Glue and AWS Lambda.
- Build, maintain, and optimize data pipelines to support data processing and analytics requirements.
- Strong hands-on experience with Python and PySpark.
- Strong SQL development skills.
- Mandatory experience with AWS services, particularly AWS Glue and AWS Lambda.
- Experience developing and maintaining data pipelines.
- Ability to participate in technical discussions and whiteboarding exercises. Compunnel
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Databricks AWS Data Engineer in Malvern, PA vacancy
- ...Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture)... ...diverse legacy and modern sources, engineer high-performance scalable pipelines,... .... Experience with Snowflake or Databricks. Knowledge of Apache Iceberg, Delta...Amazon Web ServiceDatabricks
$120.38k - $192.6k
...Requisition #: 76603The Role at a GlanceAs a Senior Lead Risk Data Engineer and Architect, you will architect and lead the design and... ...management. Working on a centralized cloud data platform (AWS, Databricks or Dataiku), you'll design scalable data pipelines, implement...Amazon Web ServiceDatabricksWork experience placementWork at office3 days per week- ...Databricks Architect Malvern, PA Must Have Technical/Functional Skills: AWS Data Services – S3, Redshift, Glue, Lambda, Athena. Data Modeling – Dimensional Modeling (Star/Snowflake), Data Warehouse Design. Data Lake Architecture – Amazon S3, Lake Formation...Amazon Web ServiceDatabricks
- ...analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources... ...to generate insight and make decisions. Experience building AWS cloud architecture and supporting services and technologies (Eg:...Amazon Web Service
- ...architects They have about six platforms (Databricks and AWS being the biggest two, Snowflake and... ...business stakeholders, architects, and engineering teams to define use cases, translating... ...into actionable platform and data product capabilities, and ensure successful...Amazon Web ServiceDatabricksContract workLocal area
- ...Responsibilities Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required Skills : Data Warehouse Basic Qualification :...Amazon Web Service
- Responsibilities Pulling data from GitHub and Jira transforming it for input into our data warehouse. Qualifications Familiarity with AWS including lambda, event bridge, s3. Proficient in typescript Datalake / Data warehouse experience is a big plus. Required Skills :...Amazon Web Service
- Job Title: Senior Data Engineer Location: Malvern, PA Onsite Duration:6 Months Experience Required: 10 & Above EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python, AWS- S3, AWS Lambdas, AWS Glue, SQL, DynamoDB , Kafka/ Kinesis , pySpark ROLE_DESCRIPTION - Overview: We are seeking...Amazon Web ServiceLocal area
- Data Engineer Location: Malvern, PA - Hybrid Important Submission Requirement Candidates must take and pass a technical assessment before... ...Required Skills Advanced Python Advanced SQL Strong experience with AWS Services , including: AWS Glue Amazon S3 AWS Lambda Amazon...Amazon Web Service
- Job Summary The AWS Data Engineer will design, develop, and implement scalable, high-performance data ingestion and transformation pipelines using AWS services and Apache Spark. The role involves integrating complex datasets from multiple sources, developing advanced SQL...Amazon Web Service
- Machine Learning Engineer We are seeking an experienced Machine Learning Engineer to join our... ...responsible for developing and optimizing complex data pipelines, integrating model pipelines,... ...experience designing ETL pipelines using AWS services (e.g., Glue, SageMaker)....Amazon Web ServiceWork experience placement
- Data Lake 2.0 Developer Data Lake 2.0 is a multi-year program aimed at modernizing legacy... ...that will be in use by Analysts and Data Engineers. The deliverables will help data... ...integrations between existing Vanguard tools, AWS components, and vendor products. Skillset...Amazon Web Service
- Data Engineer Troubleshoots software and processes for data consistency and integrity. Integrates complex data from a variety of sources for... ...required skillsets: Python Java Rest Services - Spring /Spring Boot AWS Cloud Services CloudFront S3 ECS Kinesis Lambda Step functions...Amazon Web Service
- Data Engineer Duties And Responsibilities Data engineers create and maintain an optimal data infrastructure. They combine raw information to... ...manual procedures. Building necessary infrastructure using AWS and SQL technologies. This will enable effective data extraction...Amazon Web Service
- Senior Data Engineer - 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing... .... The ideal candidate must possess top-tier skills in Python, AWS services, API development, and TypeScript, and have significant...Amazon Web Service
- ...leadership for Vanguard's MarTech Data Platform ecosystem, including... ...the platform landscape.Drive engineering best practices across data... ...architectures, preferably AWS.Deep expertise in data engineering... ....Hands-on experience with Databricks, Delta Lake, Customer Lake...Amazon Web ServiceDatabricksFull time
- Role : Data Engineer with Python Location: Malvern, PA (Onsite) Contract Job Summary We are seeking a Data Engineer with strong Python expertise... ...and data modeling. Familiarity with cloud platforms such as AWS, Azure, or GCP. Experience with version control tools like Git....Amazon Web ServiceContract work
- Purple Drive AWS Data Engineer - Python, AWS Glue, PySpark & Attunity We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS Glue, PySpark, Attunity (Qlik Replicate), and AWS Cloud to design, develop, and maintain scalable enterprise data integration...Amazon Web Service
- Job Title: Data Engineering Lead Location: Malvern, PA Contract Job Description: Skills: Java| AWS | Python | PySpark | Event-Driven Pipelines | Data Architecture Summary We are seeking an experienced Tech lead- Data Engineer (15+ years) with a strong background in Java...Amazon Web ServiceContract work
- Data Engineer Location - West Chester, PA Onsite. Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, and Python Job Summary: The Senior Technical Lead will be responsible for leading the technical aspects of projects related to...Databricks
- Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. • Deep technical knowledge - including...Amazon Web Service
- ...Request ID: 110362-1 Title: Databricks Solution Architect... ...delivering enterprise-scale data platforms, cloud modernization... ...and collaborate with business, engineering, and enterprise stakeholders... ...platforms: Azure Databricks, AWS S3, Glue, EMR, Redshift ~ Strong...Amazon Web ServiceDatabricksWork experience placementImmediate start
$71.96 - $75.75 per hour
...6 - 75.75/hr. Role Name - Databricks Solution Architect Job Description... ...delivering enterprise-scale data platforms, cloud... ...and collaborate with business, engineering, and enterprise stakeholders... ...Databricks, ADF, ADLS, Synapse, AWS S3, Glue, EMR, Redshift •Data...Amazon Web ServiceDatabricks- ...Description ROLE_DESCRIPTION - " Databricks: Lakehouse, Delta Lake, Unity Catalog, DLT, Databricks SQL, MLflow " Big Data: Spark, PySpark, Scala, Kafka, Streaming Architectures... ...: Azure Databricks, ADF, ADLS, Synapse, AWS S3, Glue, EMR, Redshift " Data...Amazon Web ServiceDatabricks
- ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building... ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with... ..., and citations.Design and build AWS source-to-product pipelines and agent-facing...Amazon Web ServiceFor contractorsWork experience placement3 days per week
- Reporting to the Manager - Data Architecture & Engineering, the Senior Developer - Data Engineering at Quaker Houghton will hands on in technology... ...native data engineering technologies (e.g. Snowflake, Databricks) • Experience in a senior management role, with...DatabricksWork experience placement
- Data Engineer, Specialist The Vanguard Group, Inc. - Philadelphia / Apex This role will be hybrid on site 3x per week in any of the following... ...Integration Machine Learning Data Governance SQL, Python, ETL AWS Services: Glue ETL, Lambda, Redshift, S3, etc AWS infrastructure...Amazon Web ServiceContract work
- ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building... ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with... ...models using SQL, Python, and approved AWS services.Build and maintain FastAPI and Pydantic...Amazon Web ServiceContract workWork experience placement3 days per week
- ...career opportunity now! Position Overview: We are looking for a Data Engineer/Scientist looking for new challenging problems. You will... ...containers, (e.g. Docker, Docker Compose), cloud services (e.g. AWS), tools for distributed computing (e.g. Spark, Pyspark) Active...Amazon Web ServiceTemporary workWork experience placementWork at officeVisa sponsorshipRelocation packageFlexible hours
$120.38k - $192.6k
...: is not available for this opportunity.Requisition #: 76901The Role at a GlanceThe Lead Data Engineer provides technical leadership for Lincoln’s Data & Analytics platform on AWS. This role designs, builds, and maintains scalable data pipelines and optimizes data systems...Amazon Web ServiceWork experience placement3 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Databricks AWS Data Engineer. Be the first to apply!
Related searches


