Data Engineer

InterSources Inc

Azure Databricks Technical Lead

We are looking for a highly skilled Senior Azure Databricks (ADB) Developer to join our Data Engineering team. This role involves developing large-scale batch and streaming data pipelines on Azure Cloud. The ideal candidate will have strong expertise in Python, Databricks Notebooks, Apache Spark (including Structured Streaming), and real-time integration with Kafka. You will work with both relational databases like DB2 and NoSQL systems such as MongoDB, focusing on performance optimization and scalable architecture.

Key Responsibilities:

Design and Develop: Create real-time and batch data pipelines using Azure Databricks, Apache Spark, and Structured Streaming.
Data Processing: Write efficient ETL scripts and automate workflows using Python.
Data Integration: Integrate with various data sources and destinations, including DB2, MongoDB, and other enterprise-grade data systems.
Performance Optimization: Tune Spark jobs for optimal performance and cost-effective compute usage on Azure.
Collaboration: Work with platform and architecture teams to ensure secure, scalable, and maintainable cloud data infrastructure.
CI/CD Support: Implement CI/CD for Databricks pipelines and notebooks using tools like GitHub and Azure DevOps.
Stakeholder Communication: Interface with product owners, data scientists, and business analysts to translate data requirements into production-ready pipelines.

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Data Engineer in Pleasanton, CA vacancy

Big Data Hadoop Engineer - Onsite
Objective & Deliverables The Big Data Engineer shall lead the Big Data Engineering team. The consultant will play the role of technical lead and provide professional services to support the long term IT strategy and planning to include high level analysis, professional...
Suggested
Work experience placement
Intelliswift
Pleasanton, CA
3 days ago
Lead Data Engineer
...Lead Data Engineer – Data & AI, Supply Chain Location: Pleasanton, CA- Onsite About the Role Experienced Lead Data Engineer to join the Supply Chain Data & AI Organization. This role will support the design, development, and delivery of enterprise data products and analytics...
Suggested
Ingrain Systems Inc
Pleasanton, CA
2 days ago
Lead Data Engineer
$65 - $70 per hour
...Lead Data Engineer - Data & AI, Supply Chain Contract (1 Year + Possible extension or FTE) Pleasanton, CA (Onsite) Pay Rate: $65-70 an hour on W2 (DOE) Job Description We are seeking an experienced Lead Data Engineer to join our Supply Chain Data & AI organization. In...
Suggested
Contract work
Local area
Russell Tobin
Pleasanton, CA
2 days ago
Data Science Engineer
Associate | Full-time Engineering | Livermore, CA | 04/21/2026Reference #: REF8262VJob Code: SES.2 Science & Engineering MTS 2 / SES.3 Science... ...a real impact.Job DescriptionWe have multiple openings for a Data Science Engineer with a background in applied machine learning...
Suggested
Minimum wage
Full time
For contractors
Local area
Work from home
Relocation
Flexible hours
1 day per week
Lawrence Livermore National Laboratory
Livermore, CA
2 days ago
Data Engineer
...Bachelor's degree or equivalent experience in computer science, applied math, physics, engineering, statistics, economics or related field. 3+ years of industry experience in Data Engineering 3+ years of work experience including hands-on technical...
Suggested
Work experience placement
Apex Informatics
Pleasanton, CA
1 day ago
Data Engineer
$215k
...Scion Staffing has been engaged to conduct a search for a Senior Data Engineer for an innovative and rapidly growing transportation and logistics organization focused on modernizing enterprise data and analytics capabilities across multiple business units. This is a hybrid...
The Bronx Inc
Pleasanton, CA
4 days ago
Principal Analytics Engineer
$184k - $276k
...match for you too.About the TeamWorkday has launched an Enterprise Data and Analytics (ED&A) Team to Transform and optimize the way... ...change.As part of ED&A and enabling this strategy, the Analytics Engineering team is at the forefront of transforming data into reliable and...
Full time
Work at office
Immediate start
Remote work
Home office
Flexible hours
Workday
Pleasanton, CA
3 days ago
Data Analytics Engineer
...In this role as Data Analytics Engineer, you will build and maintain data pipelines, tools, and visualizations to enable organizational insights. You’ll develop KPI reports, partner with engineers on data architecture, investigate metric changes, and explore trends for...
Remote work
vstglobal
Pleasanton, CA
3 days ago
Data Analytics Engineer
...vstglobal in Pleasanton, California is hiring a Data Analytics Engineer responsible for building and maintaining data pipelines, tools, and visualizations. You will partner with stakeholders and engineers to enable data-driven decision-making and develop KPI reports. The...
Flexible hours
vstglobal
Pleasanton, CA
3 days ago
Data Analytics Engineer: KPI Insights & Dashboards
A technology services company is seeking a Data Analytics Engineer to develop and maintain data pipelines and visualizations to drive organizational insights. The role requires a Master's degree and 5+ years of experience in data analytics. The candidate must be proficient...
Flexible hours
Premier Inn Hotels LLC (UAE)
Pleasanton, CA
5 days ago
Data Analytics Engineer
$65 per hour
...Trident Consulting is seeking a "Data Analytics Engineer" for one of our clients in " Pleasanton, CA (Onsite)" A global leader in business and technology services. Role: Data Analytics Engineer Location: Pleasanton, CA (Onsite) Employment Type: W2 Contract...
Contract work
Trident Consulting
Pleasanton, CA
3 days ago
Data Migration Engineer
...candidates that will contribute to the diversification and enrichment of ideas and perspectives at AHEAD. AHEAD is looking for Data Migration Engineer to work closely with our dynamic project teams (both on-site and remotely). This Data Migration Engineer will be...
Full time
Work at office
Remote work
Flexible hours
AHEAD
San Ramon, CA
4 days ago
GCP Data Engineer
...Senior GCP Engineer We are looking for an experienced Senior GCP Engineer who is a hands-on developer with deep expertise in Google Cloud Platform and modern data engineering practices. The role requires working onsite from Pleasanton and contributing to the design...
Omni Inclusive
Pleasanton, CA
2 days ago
Azure Data Engineer
...Mandatory Skills: Azure Databricks (ADB), Azure Data Factory (ADF), Azure Analysis Service (AAS), SQL Job Summary We are seeking a skilled Azure Data Engineer with strong hands-on expertise in Azure Databricks (ADB), Azure Data Factory (ADF), and SQL...
Omni Inclusive
Pleasanton, CA
2 days ago
Lead Data Engineer
...About the job Lead Data Engineer Role: Lead Data Engineer Location: Fremont, CA(3 days onsite, 2 days work from home) Duration: Full Time Job Description: ~ Candidate should Provide technical expertise in needs identification, data modelling, data...
Full time
Work from home
Inizio Partners
Fremont, CA
1 day ago
Data Engineer
...Data Engineer (3‑5 years experience) We are looking for a Data Engineer focusing on providing automation and data solutions to Finance and Accounting teams. The role supports initiatives such as Accounting Reporting, Analytics, and more, collaborating with business and...
Contract work
ArrowCore Group
Fremont, CA
2 days ago
Data Engineer
...Job Title Data Engineer Location Dublin, CA Work Arrangement Hybrid Onsite 2-3 days Roles & Responsibilities Provide L3 escalation support for existing production systems and assist with related enhancement activities. Ensure proper testing and adherence to business change...
Contract work
Perfict
Dublin, CA
4 days ago
Data Engineer
...Data Engineer 12 months Dublin, CA (Hybrid 1-2 days onsite per week) Local candidates only. The role is hybrid with 1-2 days per week in office (sometimes 0) TOP THINGS: ~ Experience with data engineering / modeling, interpersonal skills and the ability...
Interim role
Work at office
Local area
2 days per week
1 day per week
Spectraforce Technologies
Dublin, CA
3 days ago
Data Engineer
...personalized customer experiences and seeks an AI Applications Engineer to develop AI-driven solutions on our enterprise AI platform. The... ...services with enterprise systems. You will collaborate with Product, Data Science, and Engineering teams to deliver production AI use...
AT&T
San Ramon, CA
2 days ago
Data Engineer
$119k - $261k
...Lam’s Enterprise AI strategy by building the scalable, AI-ready data foundation that powers generative AI, machine learning, and... ...use cases, and semantic data models. Partner closely with AI/ML engineers, data scientists, business teams, and platform teams to translate...
Local area
Remote work
Flexible hours
2 days per week
3 days per week
1 day per week
Lam Research
Fremont, CA
2 days ago
Data Engineer - GCP
...Responsibilities: Design, develop, and maintain scalable data pipelines using Python and PySpark Build and optimize ETL/... ...and cost efficiency on GCP Participate in code reviews and engineering best practices Required Skills: Must have 2-4 years of...
Relanto
Fremont, CA
5 days ago
Data Engineer
...Launch Your Data Career with Proof, Not Promises-At SynergisticIT If you're serious about starting a high-impact, high-paying career in Data Science, Analytics, or Engineering , it's time to stop guessing and start choosing results. At SynergisticIT , we don't...
SynergisticIT
Fremont, CA
2 days ago
Data Engineer
...A leading data engineering company is seeking a Data Engineer in Dublin, CA to provide L3 support for production systems and enhance data ingestion processes. This role requires expertise in StreamSets and collaborative engagement with cloud applications. Ideal candidates...
Contract work
2 days per week
3 days per week
Perfict
Dublin, CA
2 days ago
Data Engineer
...Responsibilities Design, develop, and maintain enterprise-scale semantic data layers using Dremio to support self-service analytics and... ...modernization initiatives and promote best practices for data engineering and analytics enablement. Required Skills 5+ years of...
Relanto
Fremont, CA
2 days ago
Data Engineer
...out in the most relevant way. You can't do this with 5 different data/sales enablement point solutions that don't share the same data... ...generate more opportunities thanks to our advanced lead generation engine, hyper-personalized sequencing, omnichannel outreach, and AI-...
GrabJobs
Fremont, CA
19 hours ago
Data engineer
...Role: Data engineers (with strong Python and AWS experience) Locations : DC/VA/Client(Onsite) GC ,US CITIZEN & EAD Visa only Need Strong Hands-on experience Role Description: Data Pipeline Development: Design, implement, and...
Info Way Solutions
Fremont, CA
4 days ago
Senior Data Engineer
...Lam Research is seeking an experienced data engineer/architect to build scalable, production-grade data pipelines and AI-ready data assets on Microsoft Azure. You will shape the data foundation for enterprise AI use cases across the company, partnering with AI/ML teams...
Lam Research
Fremont, CA
2 days ago
Senior Data Engineer
...Relanto is seeking a part-time Senior Data Engineer with experience in Informatica IBMC and MWAA to work in Fremont, CA. This role involves administering and maintaining Informatica environments, supporting deployment and migrations, and optimizing data pipelines. The...
Part time
Relanto
Fremont, CA
2 days ago
Senior Data Engineer
...Title: Senior Data Engineer Client: PG&E Location: Hybrid - Dublin, CA Duration: 12+ Months Visa: USC/GC Rate: $98/hr. W2 TOP THINGS: Experience with data engineering / modelling, interpersonal skills and the ability...
3B Staffing LLC
Dublin, CA
2 days ago
Senior BI & Analytics Developer
$103.3k - $140.7k
...and warehouse implementation decisions, working closely with the Data Warehousing team to design and implement data models in... ...teams Requirements  3-5+ years experience in analytics, BI, data engineering, or a related field Strong hands on experience with: Microsoft...
Full time
Contract work
Local area
Flexible hours
Shift work
Armanino
San Ramon, CA
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!