Data Engineer

InterSources

Azure Databricks Technical Lead

Location: Pleasanton, CA (Hybrid) Job Type: Contract Implementation- UST Global

About the Role: We are looking for a highly skilled Senior Azure Databricks (ADB) Developer to join our Data Engineering team. This role involves developing large-scale batch and streaming data pipelines on Azure Cloud. The ideal candidate will have strong expertise in Python, Databricks Notebooks, Apache Spark (including Structured Streaming), and real-time integration with Kafka. You will work with both relational databases like DB2 and NoSQL systems such as MongoDB, focusing on performance optimization and scalable architecture.

Key Responsibilities:

Design and Develop: Create real-time and batch data pipelines using Azure Databricks, Apache Spark, and Structured Streaming.
Data Processing: Write efficient ETL scripts and automate workflows using Python.
Data Integration: Integrate with various data sources and destinations, including DB2, MongoDB, and other enterprise-grade data systems.
Performance Optimization: Tune Spark jobs for optimal performance and cost-effective compute usage on Azure.
Collaboration: Work with platform and architecture teams to ensure secure, scalable, and maintainable cloud data infrastructure.
CI/CD Support: Implement CI/CD for Databricks pipelines and notebooks using tools like GitHub and Azure DevOps.
Stakeholder Communication: Interface with product owners, data scientists, and business analysts to translate data requirements into production-ready pipelines.

Apply

Vacancy posted 3 days ago

Similar jobs that could be interesting for youBased on the Data Engineer in Pleasanton, CA vacancy

People Data Engineer
...**Responsibilities:Design, build, and maintain scalable people-data models within the enterprise data warehouse.Develop and support... ...Partner with the Enterprise Data Team on source-data onboarding, engineering dependencies, release planning, and production implementation....
Suggested
Local area
Axelon
Pleasanton, CA
4 days ago
Data Engineer
$146.34k - $185.54k
...CA | 10/19/2026Reference #: REF8951DJob Code: SES.2 Science & Engineering MTS 2 / SES.3 Science & Engineering MTS 3Organization: ComputingPosition... ...can make a real impact.Job DescriptionWe have an opening for a Data Engineer to build and modernize data systems that make mission-...
Suggested
Minimum wage
Full time
For contractors
Local area
Work from home
Relocation
Flexible hours
1 day per week
Lawrence Livermore National Laboratory
Livermore, CA
6 hours ago
Lead data engineer
...Title: Lead Data Engineer - Data & AI, Supply Chain Client: Gap Location: Onsite - Pleasanton, CA Duration: 6-12 months Visa: USC/GC Rate: $68/hr W2 Interview Process: 1 video interview followed by onsite interview...
Suggested
3B Staffing LLC
Pleasanton, CA
5 days ago
Big Data Hadoop Engineer - Onsite
Job-ID15409644Reference21-00142Objective & Deliverables The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high...
Suggested
Work experience placement
Intelliswift
Pleasanton, CA
4 days ago
Data Engineer
$135k - $150k
...to excellence. Plus, we know how to have fun while getting the job done! What you'll Be Doing: Vagaro is seeking a skilled Data Engineer to join our dynamic team and partner with our Data Science, Machine Learning, and Business Intelligence teams to build enterprise...
Suggested
Monday to Friday
Vagaro Inc
Pleasanton, CA
1 day ago
DATA ENGINEER
...Purple Drive DATA ENGINEER Share Contractual Pleasanton, CA PDT - 9329 12-15 Job Summary: The Senior Data Engineer will be responsible for designing, building, and maintaining robust data pipelines and architectures on AWS to support scalable data processing...
Purple Drive
Pleasanton, CA
1 day ago
Data Engineer
...Bachelor's degree or equivalent experience in computer science, applied math, physics, engineering, statistics, economics or related field. 3+ years of industry experience in Data Engineering 3+ years of work experience including hands-on technical...
Work experience placement
Apex Informatics
Pleasanton, CA
2 days ago
Data Science Engineer
Associate | Full-time Engineering | Livermore, CA | 04/21/2026Reference #: REF8262VJob Code: SES.2 Science & Engineering MTS 2 / SES.3 Science... ...a real impact.Job DescriptionWe have multiple openings for a Data Science Engineer with a background in applied machine learning...
Minimum wage
Full time
For contractors
Local area
Work from home
Relocation
Flexible hours
1 day per week
Lawrence Livermore National Laboratory
Livermore, CA
3 days ago
Senior Data Engineer
...quality and regulatory teams. The Redica Intelligence Cloud brings data together to anticipate risk, accelerate compliance, and enable... ...Description We're looking for an experienced Senior Data Engineer to take a hands-on role in designing, building, and optimizing...
Full time
REDICA Systems
Pleasanton, CA
3 days ago
Data Analytics Engineer
...Job Description In this role as Data Analytics Engineer, In this role, you will build and maintain data pipelines, tools, and visualizations to enable organizational insights. You'll develop KPI reports, partner with engineers on data architecture, investigate metric...
Remote work
Flexible hours
Venturesoft
Pleasanton, CA
4 days ago
Data Platform Engineer
Purple Drive Data Platform Engineer Share Contractual Pleasanton, CA PDT - 7314 8-14 Overview: Key Skills required: Snowflake, Fivetran, NewRelic, Apache Kafka, Plunk, Monte Carlo, Atlan, AWS, S3, Kubernetes, OAuth 2.0, Private link.
Purple Drive
Pleasanton, CA
2 days ago
Staff Data Engineer
...HRIS — and apply machine learning to turn fragmented operational data into grounded, auditable intelligence for CFOs, controllers, and... ...compliance. The Role We are looking for a Staff Data Engineer to build and own the major subsystems of the data foundation —...
Meeru AI Inc
Pleasanton, CA
8 days ago
Data Engineer
...Data Engineer Lead Location: Fremont, CA (Onsite) Duration: 12 months + Possible Extension Job Description: 1. 10+ years of proven experience with ETL/ELT (Structure & Unstructured, API's), SQL, PySpark, schema design and dimensional data modelling, medallion architecture...
Tekfortune Inc
Fremont, CA
2 days ago
W2-Data Engineer with ADF (Azure Data Factory), .NET, AAS
...Role: Senior Data Lead - Azure Data Factory - .Net, ADF Location: Pleasanton, CA (Onsite from day one) Duration: 2 Years No... ...ADF, AAS, .Net Job Summary: We are seeking a highly skilled engineer skilled in these technologies - Azure Data Factory, Azure...
Temporary work
Shift work
Weekend work
Weekday work
Noblesoft Technologies
Pleasanton, CA
4 days ago
Data Engineer
...Enterprise Technologies Inc. is a recognized provider of professional IT Consulting services in the US. We are actively seeking Data Engineer for one of our direct client in Fremont, CA . Please share your resume with current location & full contact info Job role:...
Remote work
Rootshell Enterprise Technologies
Fremont, CA
3 days ago
Data Engineer
...Job Description The Data Engineer (Senior Level or above preferred) will design, develop, and maintain enterprise data solutions on the Snowflake platform. This role is responsible for building scalable data pipelines, developing curated reporting datasets, implementing...
Insight Global
Dublin, CA
1 day ago
Data Engineer
$105k - $131.26k
...knowledge to make a difference in our members' lives, Patelco could be the perfect fit for you. Overview The Data Engineer is a key contributor to Our Data & Analytics Strategy to transform Patelco as a Data-driven organization and position Credit...
Full time
Temporary work
Part time
Work experience placement
Patelco Credit Union
Dublin, CA
3 days ago
Data Engineer
...Job Title : Data Engineer Location : Dublin, CA Hybrid Onsite 2-3 days Roles & Responsibilities Provide L3 escalation support for existing production systems and assist with related enhancement activities. Ensure proper testing and adherence to business...
Perfict Global, Inc.
Dublin, CA
5 days ago
Lead Data Engineer
$114.4k - $124.8k
...experience with ETL/ELT processes involving structured and unstructured data and APIs, as well as SQL, PySpark, schema design, dimensional... ..., dynamic environment Responsibilities: ~ Lead data engineering work, applying ETL/ELT expertise, big data tools, and data...
Hourly pay
Full time
Temporary work
CYNET SYSTEMS
Fremont, CA
9 days ago
Workday PSA & Financials Data Engineer
$118.8k - $161.9k
...passionate about, in industries you care about.Workday PSA & Financials Data EngineerArmanino is proud to be among the top 20 Largest... ...investigation and escalation by partnering with cross team engineers on Workday design decisions.Collaborate with Workday Security Team...
Full time
Contract work
Local area
Armanino
San Ramon, CA
4 days ago
Lead Data Engineer
$55 - $60 per hour
...Job Description Job Description Pay Range: $55.00hr - $60.00hr Job Overview: The candidate will serve as a Data Engineering Lead, leveraging extensive experience with ETL/ELT processes, big data tools, and data warehousing standards to support complex data modeling...
Temporary work
Local area
CYNET SYSTEMS
Fremont, CA
5 days ago
Lead Data Engineer
...of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Data Engineer Lead Data Engineer Enterprise Credit Risk (ECR) Who is Mastercard? At Mastercard technology, we work to connect and...
Full time
Worldwide
Mastercard
Dublin, CA
24 days ago
Data Engineer
$60k - $148.5k
...Leveraging our holistic portfolio of capabilities in consulting design engineering and operations we help clients realize their boldest ambitions... ...ever-changing world. For additional information visit us at . Data Engineer Position: Data Engineer Key Skills: Track...
Minimum wage
Full time
Local area
Wipro
Fremont, CA
9 days ago
Staff Data Engineer
...energy-intensive world. From revolutionizing power for AI-driven data centers to ensuring resilience for hospitals, electric grids,... ...challenges of the 21st century. We are looking for a Staff Data Engineer to join our team in one of today’s most exciting technologies. This...
Full time
Work at office
Worldwide
Bloom Energy
Fremont, CA
4 days ago
Data & AI Platform Engineer
$91.3k - $124.4k
...network of smart and supportive people, willing to listen to your ideas. This is an early-career engineering role focused on building, operating, and improving cloud data/analytics platforms (e.g., Microsoft Fabric, Snowflake, Databricks) and supporting BI delivery (e....
Full time
Contract work
Internship
Local area
Flexible hours
Armanino
San Ramon, CA
2 days ago
Senior Data & AI Platform Engineer
$112.8k - $153.7k
...a network of smart and supportive people, willing to listen to your ideas. This is a senior engineering role. You will architect, operate, and continuously improve cloud data and analytics platforms (e.g., Microsoft Fabric, Snowflake, Databricks) and own BI delivery infrastructure...
Full time
Contract work
Local area
Flexible hours
Armanino
San Ramon, CA
4 days ago
Senior Mechanical Engineering Analyst
$210.63k - $267.06k
Mid-Senior Level | Full-time Engineering | Livermore, CA | 04/30/2026Reference #: REF8307PJob Code: SES.4 Science & Engineering MTS 4Organization... ...modeling, pre- and post-processing, Simulation Process and Data Management (SPDM), Verification, Validation and Uncertainty...
Minimum wage
Full time
For contractors
Local area
Work from home
Relocation
Flexible hours
1 day per week
Lawrence Livermore National Laboratory
Livermore, CA
4 days ago
Lead Data Scientist, AI Engineering
...products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Data Scientist, AI Engineering Lead Data Scientist, AI Engineering Overview Mastercard's AI Centre of Excellence is building the next...
Full time
Worldwide
Mastercard
Dublin, CA
24 days ago
Azure Data Engineer
...Title: Azure Data Engineer Location-Fremont, CA Job Type-Long Term Interview Mode: Video Hands on Interview on SQL, Python, Pyspark.. Azure, Azure Databricks, Fabric, ADLS/ADF etc. Must have Skills : # Min 5 years of experience in modern...
Lorven Technologies
Fremont, CA
5 days ago
Sr Product Data Scientist
$124.4k - $186.6k
...decision-ready internal products that turn complex organizational data into targeted leadership action. Our vision is to: Power the... ...with the Data Cloud organization (SVP/GM, Product Management, Engineering, and GTM teams) to bring analytical rigor and powerful business...
Full time
Work at office
Remote work
Home office
Flexible hours
Workday
Pleasanton, CA
3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!