Senior Data Engineer
Compunnel
JOB SUMMARY
Seeking a Data Engineer with Spark & Streaming skills to build real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML.
• Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink).
• Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms.
• Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness.
• Implement data quality checks, monitoring, and alerting systems to ensure data accuracy and consistency. Required Qualifications
• Programming: Strong proficiency in Python, SQL, and potentially Scala/Java.
• Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming).
• Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub.
• Cloud: Familiarity with GCP, Azure data services.
• Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL databases.
• Total IT Experience: Minimum 8 years.
• GCP Experience: 4+ years of recent GCP experience. Preferred Qualifications
• Tools: Experience with Airflow, Databricks, Docker, Kubernetes. Certifications
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer in Sunnyvale, CA vacancy
- ...Sterling B2B Developer Role: Senior Data Engineer Location: Dallas, TX or Mountain View, CA Duration: Long Term Contract As part of the Mail Analytics Data Engineering team, you will be working on large-scale batch pipelines, data serving, data lakehouse,...SeniorLong term contract
Hermitage Infotech
Mountain View, CA10 hours ago - ...A leading autonomous technology company in Mountain View is seeking a Senior Software Engineer specializing in Perception ML Data. This role involves bridging machine learning and autonomy infrastructure, addressing complex data challenges, and developing innovative systems...Senior
Nuro
Mountain View, CA2 days ago - ...Job Title: Senior Data Engineer (High Priority) Location: Preferably Mountain View, CA; Hybrid Start Date: Immediate Duration: 6+ months Openings: 4 Senior Engineers About the Role We are seeking Senior Data Engineers to join an experienced...SeniorImmediate start
Info Way Solutions
Mountain View, CA2 days ago - ...A technology solutions provider is seeking a Senior Data Management Engineer for its San Jose, CA location. The role involves analyzing customer requirements, developing Data Management applications, and supporting software releases. Candidates should have a Bachelor's...Senior
Hitachi Data Systems
Santa Clara, CA2 days ago - ...Scala Engineer We are looking for Scala Engineers with experience with batch and/or streaming jobs. We utilize Spark for batch jobs and... ...and batch use cases, and interacting with infrastructure and data teams to produce complex analysis across data. Required Qualifications...Senior
Omega Solutions Inc
Mountain View, CA10 hours ago - ...Title: Senior Data Engineer Location: Sunnyvale, CA Position: 2 Month (Contract) Summary Provide business insights, while leveraging internal tools and systems, databases and industry data Roles & Responsibilities Ability to...SeniorContract work
TWO95 International
Sunnyvale, CA3 days ago - ...Role Summary We are seeking a highly skilled Senior Data Engineer - Full Stack to build and maintain internal tools, automation frameworks, and workflows that enhance the efficiency, reliability, and scalability of our data and machine learning platforms. This role...Senior
Codvo.ai
Santa Clara, CA1 day ago - ...for your reference. Recent update from client for this data engineer role / Request-ID: 13145-1 _ Bentonville, AR position is no... ...reference, which has closed with client UST. Looking for Senior Data Engineer with Spark, Scala, GCP experience. Location...Senior
Texas State Library and Archives Commision
Sunnyvale, CA2 days ago - ...Job Description: We are looking for a Senior Data Engineer with advanced knowledge of SQL and intermediate knowledge of Python. Nice to have (but not required) beginner or intermediate level java experience Your primary focus will be the writing...Senior
Rootshell Enterprise Technologies
Santa Clara, CA10 hours ago - ...fundamental and applied research in Intelligent data analytics, Hardware/Co-processor... ...Visualization, etc. We are leading researchers and engineers from worldwide to create adaptive and... ...publishing papers. We are recruiting senior data scientist/engineers with multi-disciplinary...SeniorFor contractorsWorldwide
Netpace
Santa Clara, CA10 hours ago - ...Description Hitachi Energy is seeking for a Senior Data Management Engineer for its San Jose, CA location. Your Responsibilities: Analyze and understand customer engineering, business, and software requirements, propose technically feasible solutions, estimate required...SeniorWork experience placement
Hitachi Data Systems
Santa Clara, CA2 days ago $184k - $356.5k
NVIDIA Gruppe in Santa Clara is looking for an experienced engineer to lead performance benchmarking and optimization for their data center products. The ideal candidate will design and execute benchmarking strategies and analyze performance bottlenecks to ensure high performance...Senior- A leading technology firm in Santa Clara, CA is seeking a Senior Data Center Performance Engineer to lead benchmarking and optimization for their data center products. The role requires extensive experience in performance engineering, strong programming skills, and deep...Senior
- ...delivered for millions of patients worldwide. We're a team of engineers, clinicians, and innovators united by one purpose: to make... ...Description Primary Function of Position We are seeking a Senior Data Engineer to join the software team within the Endoluminal...SeniorLocal areaWorldwideFlexible hours
Intuitive
Sunnyvale, CA2 days ago $188.6k - $271.4k
Intuitive Surgical, Inc. is seeking a Senior Data Engineer in Sunnyvale, California to lead AI developer enablement initiatives. You'll design and build scalable data pipelines while ensuring data quality and collaborating across teams. Applicants should have a Bachelor...Senior- ...TITLE: ML Data Infrastructure Engineer LOCATION: Sunnyvale CA or Remote Duration: 12+ Months Rate: DOE Key skills - GCP ML Infrastructure, BigQuery, Dataflow, Airflow ( Cloud composer), Vertext AI , Datapipeline, ML Training Role Overview: We're seeking...SeniorRemote work
Redolent
Sunnyvale, CA3 days ago - NVIDIA Gruppe in Santa Clara is looking for a Senior Data Analysis Engineer to lead performance definitions for autonomous driving products. You will work in a collaborative environment addressing real-time automotive challenges. Key responsibilities include defining metrics...Senior
$152k - $253k
A leading technology company is seeking a Senior Data Analysis Engineer for their autonomous vehicles team. The role involves defining product performance metrics, automating data analysis, and collaborating across teams to improve product efficiency. Candidates should...Senior- A leading technology firm based in Mountain View, California is seeking a Data Engineer to develop and enhance Python frameworks for data processing and build scalable data pipelines on Google Cloud Platform. The ideal candidate should have extensive experience in GCP services...Senior
- Yahoo Holdings Inc. in Mountain View is seeking a Senior Data Engineer to architect and build scalable systems that fuel Yahoo Mail analytics. You will partner with cross-functional teams to define data ontologies, develop high-reliability data pipelines, and optimize...SeniorFlexible hours
- NVIDIA Gruppe is looking for a Senior Data Engineer in Santa Clara, California. This role involves architecting the financial infrastructure for NVIDIA Cloud Services. You'll play a vital role in creating scalable cloud services integrated with financial systems. The ideal...Senior
$152k - $241.5k
Nvidia Corporation is seeking a Senior Data Engineer to build analytics-ready data platforms for its EDA datacenters. The role focuses on transforming large-scale observability data into reliable datasets, ensuring data quality for data scientists and engineers. Applicants...Senior$153k - $222k
Applied Intuition is looking for an Infrastructure Engineer to join their team in Mountain View, California. In this role, you will work with large-scale open-source data technologies and help build intelligent vehicle solutions. Responsibilities include scaling infrastructure...Senior- A major technology company based in Mountain View is looking for experienced Data Engineers to optimize data infrastructures for machine learning and analytics. Candidates should have a strong foundation in algorithms, a background in data engineering or machine learning...Senior
- ...Huawei Data Lake Cloud Services Developer As a core member, you will be part of the team that focuses on the next generation Huawei... ...designing and developing: Distributed storage and query engine for heterogeneous data sources, including graph databases, key/value...SeniorFor contractors
Netpace
Santa Clara, CA10 hours ago $129.4k - $212.3k
Towards AI, Inc. is hiring a Senior Data Engineer to join their Marketing Applied Sciences team. This remote role involves designing and maintaining data pipelines, collaborating with data scientists, and ensuring high-quality data for insights. Required qualifications...SeniorRemote job- ...a software developer in Sunnyvale, California. This role requires over 5 years of software development experience with strong data engineering skills and proficiency in Python and Java, along with knowledge of relational databases like Oracle and MySQL. The candidate should...Senior
- Job Description Primary Function of Position We are seeking a Senior Data Engineer to join the software team within the Endoluminal business unit and help drive AI developer enablement initiatives across the company. The successful candidate will design, build, and scale...Senior
$193.93k - $352.29k
Nuro is seeking a Senior/Staff Software Engineer to lead ML data engineering efforts. This role involves designing data pipelines, working cross-functionally to enhance ML products, and developing tools for data analysis. The ideal candidate has over 7 years of experience...Senior$152k - $241.5k
NVIDIA's Hardware Infrastructure organization is seeking a Senior Data Engineer to build and evolve analytics-ready data platforms that power observability, reliability analysis, and capacity forecasting for EDA datacenters. In this role, you will focus on transforming...SeniorFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
Related searches
- senior data center engineer Sunnyvale, CA
- data engineer manager Sunnyvale, CA
- data science developer Sunnyvale, CA
- etl data engineer Sunnyvale, CA
- entry level big data engineer Sunnyvale, CA
- data engineer Sunnyvale, CA
- big data cloud engineer Sunnyvale, CA
- junior big data engineer Sunnyvale, CA
- remote data engineer Sunnyvale, CA
- senior data engineer Sunnyvale, CA