Sr. Databricks Engineer

Transcend IT Solutions

Malvern, PA

Job Title: Sr. Databricks Engineer

Location: Malvern, PA

Job Details:

Technical Stack & Requirements:

Core Competencies: Python, Apache Spark (PySpark), AWS Glue ETL, Data Lake concepts (Medallion Architecture), Databricks, Sagemaker.
Infrastructure: CloudFormation/Terraform, CI/CD with GitHub Actions.
Core Responsibilities: Orchestrate data extraction from diverse legacy and modern sources, engineer high-performance scalable pipelines, and provide expert-level production support to ensure system reliability.

Qualifications:

Bachelor’s degree in Computer Science or related field.
Strong hands-on experience with Apache Spark and Glue.
Proven ability to work with Python-based data pipelines.
Experience with Git and GitHub Actions is mandatory. Required Qualifications
Bachelor's degree in Computer Science, Engineering, Information Systems, or related discipline.
5-7+ years of enterprise Data Engineering experience.
Strong Python development experience.
Advanced SQL programming skills.
Extensive experience with Apache Spark and PySpark.
Strong experience with Databricks application.
Hands-on experience with AWS data services including: o S3 o AWS Glue o Athena o Lambda o Step Functions o EventBridge
Experience building enterprise ETL and ELT pipelines.
Strong understanding of distributed data processing.
Experience working with large-scale cloud data platforms.
Strong knowledge of dimensional data modeling.
Experience implementing CI/CD pipelines.
Familiarity with Agile/Scrum methodologies.
Excellent communication and stakeholder management skills.

Preferred Qualifications

Financial Services or Asset Management industry experience.
Knowledge of Apache Iceberg, Delta Lake, or Hudi.
Experience with Airflow.
Knowledge of Kafka or Kinesis streaming.
Experience supporting AI/ML and advanced analytics platforms.
AWS Professional or Specialty Certifications.

Job Responsibilities

We are seeking high-caliber Data Engineers to drive data analysis projects, Migrate jobs from EMR to Databricks,create strong job aids and train analysts. In this pivotal role, you will implement and optimize sophisticated data pipelines, partnering with business analysts to guarantee the seamless availability and absolute integrity of data powering our core financial engines and reporting ecosystems.

Hari Krishna Vanga
Senior Technical Recruiter
E:
L:
A: 25663 Smotherman Rd, Suite 302, Frisco, TX, 75033.

Apply

Vacancy posted 4 hours ago

Similar jobs that could be interesting for youBased on the Sr. Databricks Engineer in Malvern, PA vacancy

QA Automation Engineering
...verification and backend validationCollaborate with developers, data engineers, and product teams to ensure quality throughout SDLC Integrate... ...cloud platforms (Azure/AWS)Exposure to data platforms like Databricks, Snowflake, or SynapseKnowledge of performance testing...
Databricks
Futran Tech Solutions Pvt. Ltd.
Malvern, PA
10 hours ago
Sr. Systems Engineer
...Are you an experienced systems engineering leader looking for the opportunity to shape an innovative aerospace platform from concept through production? Innovative Aerosystems is seeking a Sr. Systems Engineer to provide technical leadership for a highly advanced aerospace...
Senior
Work at office
Innovative Solutions & Support
Exton, PA
4 days ago
Senior Technical Product Manager
...with solutions architects They have about six platforms (Databricks and AWS being the biggest two, Snowflake and Confluence being... ...System Integrators (SIs), business stakeholders, architects, and engineering teams to define use cases, translating requirements into...
Databricks
Senior
Contract work
Local area
The Judge Group
Malvern, PA
8 hours ago
Senior Engineer
Senior EngineerLocation – Malvern, PA - HybridDuration – 6+ monthsInterview – VideoNo AssessmentKEY SKILLSAWS, SQL, PYTHON MUST HAVE TRUE HANDS-ON EXPERIENCE WITH AI AGENTS Must have extensive ai exp - preferably Claude
Senior
RIT Solutions
Malvern, PA
2 days ago
Sr Director, Data Engineering & Architecture
...environment and is looking to add a Senior Director of Data Engineering & Architecture to define technical direction, strengthen the... ...to data. Experience with other enterprise platforms such as Databricks or Snowflake is valuable because this leader should be able to...
Databricks
Senior
Full time
Remote work
Spinnaker Search
Wayne, PA
21 days ago
Senior System Engineer
Job Title Responsibilities Operate in a highly collaborative team emphasizing best practices in software development, automation, DevSecOps and SRE. Build cloud-first, consumer-focused and applying lean agile methodologies. Ability to learn and build in ...
Senior
Samprasoft
Malvern, PA
1 day ago
Senior Workday Integration Engineer
...Job Description Job Description Job Title: Senior Workday Integration Engineer Duration- Fulltime Permanent Location: Malvern PA / Charlotte NC (Onsite From Day 1) Job Description: Senior Workday Integration Engineer 7 years of HR technology experience...
Senior
Permanent employment
Full time
Q1 Technologies, Inc
Malvern, PA
2 days ago
Full Time: Systems Engineering Senior
$95.9k - $178.1k
...Space, the Mission Processing portfolio is looking for a highly motivated individual to join the ARGUS TO2 program area as a Systems Engineer. This exciting position will report into the Systems Integration & Engineering (SI&E) Talent Center within the Engineering &...
Senior
Full time
Temporary work
For contractors
Work experience placement
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
1 day ago
Systems Engineer-IT Sr - E3
$95.9k - $178.1k
...requirements into hardware and software specifications. For positions that do not perform IT responsibilities, use E146, Systems Engineer.Basic QualificationsMust have an active Top Secret / SCI clearance to startMust be able to obtain and maintain an adjudicated polygraphPossesses...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
Full Time: Systems Engineering Senior
$145.1k - $269.5k
...CoreBusiness Unit: RMS CoreStandard Job DescriptionWHO WE ARELockheed Martin Rotary and Mission Systems have an opportunity for a Systems Engineering Integration and Test (SEIT) Lead for operational systems within the Ground Cyber Electronic Warfare portfolio. The successful...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
USA_Technical Architect
$71.96 - $75.75 per hour
...Description Pay Rate Range: $ 71.96 - 75.75/hr. Role Name - Databricks Solution Architect Job Description: We are seeking a... ...scalable Lakehouse solutions, and collaborate with business, engineering, and enterprise stakeholders to enable data-driven transformation...
Databricks
Varite, Inc
Malvern, PA
2 days ago
Senior Reliability Engineering
...Senior Reliability EngineerHybrid - Malvern, PAJob Description As a Senior Reliability Engineer, you will play a critical role in solving impactful operational problems. You are curious and take a proactive approach to identifying problems and making improvements. You...
Senior
RIT Solutions
Malvern, PA
1 day ago
Sr Device Engineer
...flexibility to balance life with work, and the opportunity to better health worldwide, together.Our Team, Your ImpactFor the Senior Device Engineer position we are seeking a motivated and technically curious engineer to join our Combination Product Development team. In this...
Senior
Contract work
Local area
Worldwide
Flexible hours
Shift work
Teva Pharmaceutical Industries
West Chester, PA
4 days ago
System Integration and Test Engineer Sr
$95.9k - $178.1k
...reimagining how space can connect us, ensuring security and prosperity.We are urgently seeking to recruit an Integration and Test Engineer Sr to be based on-site at our King of Prussia, PA facility. This role will serve the Mission Processing portfolio of programs.Key...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
5 days ago
Systems Engineer Sr.
$95.9k - $178.1k
...CoreBusiness Unit: RMS CoreStandard Job DescriptionWithin the Rotary Mission Systems (RMS) AIR6500 campaign we are looking for a Systems Engineer to support the requirements, architecture and design of the AIR6500 Joint Air Battle Management System (JABMS). As a Systems...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Afternoon shift
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
Project Engineer Sr - E3
$50 per hour
...reflects the standard job description for this position: Coordinates planning, organization, control, integration and completion of engineering projects within area of assigned responsibility. Plans and formulates engineering program; reviews product design for compliance...
Senior
Full time
Contract work
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
Sr Project Engineer
...compliance and product performance. Collaborate with Quality, Regulatory, Manufacturing, and Supply Chain teams to evaluate and implement engineering changes. Proactively identify and communicate improvement opportunities based on customer feedback, internal testing, or supplier...
Senior
Full time
Globus Medical
Audubon, PA
2 days ago
Project Engineer Sr - Level 3
$50 per hour
...us, ensuring security and prosperity.Join us in shaping a new era in space and find a career that's built for you!As a Mk7 project Engineer you will coordinate planning, organization, control, integration and completion of engineering projects within area of assigned...
Senior
Full time
Contract work
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
1 day ago
Full Time: Systems Engineering Senior
$85.3k - $158.3k
...internationally!Check out these links for more information about our exciting suite of products:The Work:CommandIQ is seeking a systems engineer to join one of our agile development teams. In this role, you will:- Decompose customer/stakeholder needs into higher-level...
Senior
Full time
Temporary work
Work experience placement
Casual work
Local area
Remote work
Flexible hours
Shift work
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
Full Time: Systems Engineering Senior
$50 per hour
...that will provide enhanced communications capacity, flexibility and resiliency. This position will be a key member of the Systems Engineering, Integration and Test (SEIT) team in support of the final design, development, production, integration, test, and delivery of...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
4 days ago
Project Engineering Sr Mgr
$178.5k - $331.5k
...portfolio projects within the Manufacturing and Test environment.Manages the program execution and program performance of experimental engineering, development, and production lifecycle through the hardware assembly/integration/test processes. Collaborates with Operations,...
Senior
Full time
Temporary work
Work experience placement
Interim role
Casual work
Work at office
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
2 days ago
Radio Frequency Engineer Sr Stf - Level 5 (Secret)
$166.9k - $309.9k
...communications capacity, flexibility and resiliency.The selected candidate will support the Ground segment of the program as a Sr Staff RF Engineer will serve as a recognized subject matter expert by providing architectural guidance and strategic direction for RF systems...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
5 days ago
Systems Engineer Sr - E3
$95.9k - $178.1k
...while maintaining the technical requirements of the strategic deterrence.Lockheed Martin (LM) is looking for an resourceful Systems Engineer, Senior to become a member of the Systems Engineering and Integration Team.As a Systems Engineer, Senior, you will:• Develop CAMEO...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
4 days ago
Senior Mechanical Engineering-Design
...that if it can be dreamed it can also be measured. And if it can be measured, it can also be realized.The Senior Design Mechanical Engineer is a key contributor responsible for design and analysis of mechanical systems, equipment and packaging for FARO's metrology...
Senior
Full time
For contractors
Work at office
Immediate start
Faro
Lionville, PA
1 day ago
Project Engineer Sr - E3
$50 per hour
...amazing new technologies to our customers while maintaining the technical requirements of strategic deterrence.We are seeking a Sr Project Engineer to join the Mk7 Aeroshell Integrated Product Team (IPT). The successful candidate will support the program as they ensure...
Senior
Full time
Contract work
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
3 days ago
Full Time: Systems Engineering Senior
$95.9k - $178.1k
...will be part of a growing team building this new program and customer relationship from the ground up.As a member of the Systems Engineering team you will:• Perform in an accelerated, collaborative, dynamic small team environment, closely integrated with customers and contractors...
Senior
Full time
Temporary work
For contractors
Work experience placement
Interim role
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
2 days ago
Mechanical Engineer Sr - E3
$50 per hour
...description for this position:Researches, plans, designs and develops mechanical products and systems such as instruments, controls, robots, engines, machines and mechanical, thermal hydraulic or heat transfer systems for production, transmission, measurement, and use of energy....
Senior
Full time
Contract work
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago
Full Time: Systems Engineering Senior
$95.9k - $178.1k
...and Reconnaissance Group (363 ISRG) production units and the Standoff Munitions Application Center (SMAC). THE WORKThe JES Systems Engineer will focus on designing, developing, and maintaining test procedures and supporting the automated test infrastructure for a...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
King of Prussia, PA
1 day ago
Project Engineer Sr Stf - E5
$50 per hour
...capabilities. Join us in delivering technology for some of the world's most complex missions!The WorkRMS is seeking an experienced Project Engineer Sr Staff to join our Ground Cyber Electronic Warfare Integrated Engineering organization. You will coordinate engineering project...
Senior
Full time
Temporary work
Work experience placement
For subcontractor
Casual work
Work at office
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
2 days ago
Cyber Systems Security Engineering Sr - E3
$50 per hour
...years)• Possess an approved DoD 8570 baseline certification for IAT Level II (CISSP, CAP, etc.).• Understanding of System Security Engineering (SSE) principals and process.• Understanding of Risk Management Framework (RMF) NIST SP 800-53• Knowledge of Windows Server...
Senior
Full time
Temporary work
Work experience placement
Casual work
Flexible hours
Lockheed Martin
Valley Forge, Montgomery County, PA
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Sr. Databricks Engineer. Be the first to apply!