Senior Data Engineer
Compunnel
Job Summary Seeking a Data Engineer with Spark & Streaming skills to build real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML. Key Responsibilities Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink). Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms. Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness. Implement data quality checks, monitoring, and alerting systems to ensure data accuracy and consistency. Required Qualifications Programming: Strong proficiency in Python, SQL, and potentially Scala/Java. Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming). Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub. Cloud: Familiarity with GCP, Azure data services. Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL databases. Total IT Experience: Minimum 8 years. GCP Experience: 4+ years of recent GCP experience. Preferred Qualifications Tools: Experience with Airflow, Databricks, Docker, Kubernetes. #J-18808-Ljbffr Compunnel
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer in Sunnyvale, CA vacancy
$157k - $210k
...CoreWeave plans, sources, builds, and delivers data center capacity for our customers. We partner closely with Supply Chain, Product Engineering, Data Center Operations, and the... ...analytical findings and recommendations to senior leadership in a clear, actionable format....SeniorPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hours- A leading technology firm is looking for a Remote Data Engineer with at least 10 years of experience, particularly in Google Cloud Platform. The role includes designing and optimizing analytical datasets, building robust data pipelines, and collaborating with product managers...SeniorRemote work
Global Channel Management
Santa Clara, CA1 day ago $150.4k - $277.6k
...telling what you can accomplish. We are seeking an exceptional engineer with a background in large scale software development and delivery... ...with a heavy focus on using cutting‑edge technologies to build data intensive systems, designing and optimizing data models, and collaborating...SeniorApple
Sunnyvale, CA13 hours ago- ...exciting area of tech and are fundamentally reshaping the data storage industry. Here, you lead with innovative thinking,... ...workflows that help surface insights with less manual effort. As a Senior Analytics Engineer, you will help drive that journey end‑to‑end. This role is...SeniorFlexible hours
Pure Storage
Santa Clara, CA4 days ago $90k - $180k
Overview International Data Management (IDM) team powers enterprise‑scale data platforms across multiple global markets including Canada... ...smart design and optimization strategies. Develop AI‑driven engineering solutions : Build internal tools, utilities, and AI agents to...SeniorTemporary work$182k - $242k
...Nasdaq: CRWV) in March 2025. Learn more at . What You’ll Do The Data Engineering Team builds and operates the foundational data infrastructure... ..., and actionable at scale. About The Role We're seeking a Senior Data Engineer to design and build the foundational datasets...SeniorPermanent employmentTemporary workCasual workWork at officeFlexible hours$180k - $270k
...project is harmonious with TRI’s robotics divisions' efforts in Diffusion Policy and Large Behavior Models. We are looking for a Senior Data Engineer to design and build the foundational data infrastructure and tools that power our autonomy research and development...SeniorLocal areaShift work$181.1k - $272.1k
...believe we are just getting started. Do you have a strong passion for using data to drive business decisions, generate ideas, and inspire collaborators? We are seeking for a Senior Data Engineer to join our App Store Data team and play an integral role in helping the...SeniorRelocation$143k - $286k
...longer just building dashboards or static pipelines; we are building the "brain" of the retail experience. As the Senior Engineering Manager for Agentic Data, you will be at the forefront of the AI revolution. Your mission is to evolve our massive data ecosystem into a...SeniorPermanent employmentFull timeTemporary workPart timeFlexible hoursWalmart
Sunnyvale, CA13 hours ago$157k - $210k
...Nasdaq: CRWV) in March 2025. Learn more at We’re hiring a Senior Analytics Engineer to own analytics for the Machine Infrastructure Supply... ...and evolve our enterprise semantic layer—transforming raw data into trusted models, metrics, and analytics‑ready datasets...SeniorPermanent employmentTemporary workCasual workWork at officeFlexible hours$133.1k - $236.4k
...creative generative AI products — and it's growing fast. Our data team sits at the center of that growth, powering the analytics... ...decisions that build what Firefly becomes. We're hiring a Senior Data Engineer to improve our data foundation. This person not only builds pipelines...SeniorFull timeTemporary workLocal areaImmediate startWorldwide$174.72k - $295.68k
...transportation through cutting-edge R&D in AI, machine learning, and smart connectivity.We are seeking a Machine Learning Data Curation Engineer to spearhead the data pipeline development and dataset management for our core AI initiatives. You will bridge the gap between...SeniorFull timePart timeXPENG Motors
Santa Clara, CA16 hours ago$184k - $287.5k
...do their best work.NVIDIA has a rapidly expanding ecosystem of data center platform designs. From single node HGX/DGX systems all the... ...and HPC software stack. We are searching for a highly motivated engineer to lead performance benchmarking and optimization efforts for our...SeniorFull timePart timeNvidia
Santa Clara, CA16 hours ago$152k - $253k
...deepen our efforts in the autonomous driving field and strive to bring sustained growth to our customers.We're looking for a Senior Data Analysis Engineer, AD critical metrics with strong technical ability to deeply understand architectures and strong scripting experience to...SeniorFull timePart timeNvidia
Santa Clara, CA16 hours ago- ...DescriptionGeneral information:Hitachi Energy is seeking for a Senior Data Management Engineer for it's San Jose, CA location.Your Responsibilities:Analyze and understand customer engineering, business, and software requirements, propose technically feasible solutions,...SeniorPart time
Hitachi
Santa Clara, CA16 hours ago - ...millions of patients worldwide.We’re a team of engineers, clinicians, and innovators united by one... ...in robotic surgery requires high-quality data across the robotic platform, the surgical... ...and the operating room environment. The Senior AI/Data Science Engineer will advance the...SeniorPart timeLocal areaWorldwideFlexible hours
Intuitive Surgical
Sunnyvale, CA16 hours ago - ...tomorrow with advanced photonic technologies that enable AI, data centers, telecom, industrial, and sensing applications. As... ...a network of R&D, manufacturing, and sales locations. Senior Manager, Data Engineering Responsibility Architect and implement scalable cloud-based...SeniorFull time
- ...ambitious goals. World‑class leadership team: Our Heads of AI, Engineering, and Product bring extensive experience from some of the world’... ...and rewarded as we grow together. The Role Kai is hiring a Sr. Data Engineer to join our data infrastructure team. This is a hands‑...Senior
Kai
San Jose, CA1 day ago - ...Senior Software Engineer - Data Protection Software Engineering (C, C++)Infrastructure Solutions Group (ISG) builds the products that power infrastructure, solutions, and data management our customers need most. Our teams design and develop the hardware and software that...SeniorPart time
Dell Technologies
Santa Clara, CA16 hours ago $203.45k - $344.3k
...DescriptionDefine the overall architecture of the vehicle-cloud integrated data closed-loop from a strategic perspective, making it the core... ...of data lifecycle costs, data architecture and closed-loop engineering system.Job ResponsbilitiesResponsible for the design and...SeniorFull timeTemporary workPart timeWork experience placementXPENG Motors
Santa Clara, CA16 hours ago- ...Responsibilities Design, build, and own the key data infrastructure to manage data and usage across our Snowflake account Interface... ...pipelines sourcing data from application apis Collaborate closely with Engineering, Product Management, IT, Finance, and Compliance to inform...SeniorRemote workWork from homeFlexible hours
Snowflake Computing
Menlo Park, CA1 day ago $187k - $198k
...and so are the rewards. Robinhood is a metrics driven company and data is foundational to all key decisions from growth strategy to... ...optimization to our day-to-day operations. We are looking for a Data Engineer to build and maintain foundational datasets that will allow us...SeniorWork at officeShift workI did my part and supported the Regular Toilet
Menlo Park, CA1 day ago$184k - $287.5k
...Within NVIDIA, our Financial Services Engineering (FSE) group’s mission is to architect the financial infrastructure powering NVIDIA... ...accurate and frictionless across the globe.We are looking for a Senior Data Engineer to join our Financial Systems Engineering team. This...SeniorFull timePart timeNvidia
Santa Clara, CA16 hours ago$175k - $200k
...Overview East West Bank is looking for a motivated Senior Data Engineer to contribute towards the success of our Data and Analytics Technology initiatives. This is a hands‑on technical role and this person will be responsible for the development and build of scalable...SeniorWork experience placementEast West Bank
Palo Alto, CA1 day ago$168k - $258.75k
...We are looking for a Senior Technical Program Manager (TPM) to join NVIDIA’s Server Engineering Operations Team. You will be the cross-section between execution and strategy... ...take on more of these unique opportunities in data-center solutions.What you will be doing:The...SeniorFull timePart timeNvidia
Santa Clara, CA16 hours ago- Senior Data Engineer We are looking for a Senior Data Engineer to work with a leading generative AI company in healthcare. Location: Palo Alto, CA office. This role is expected to be in our Palo Alto office five days a week. Benefits include 401k, equity, and on-site...SeniorWork at office
$204k - $259k
...efficient autonomous operation. In this hybrid role, you will report to the Senior Engineering Manager of Semantics. You Will: Develop and implement new features in the VLM data infra and validate the changes for the model platform to support the continuing...SeniorFull timeWork at officeRemote workWaymo
Mountain View, CA13 hours ago$139k - $257.55k
About the Role We are looking for a Senior Data Engineer to join our Data & Analytics Engineering team. In this role, you will develop and maintain scalable data pipelines that support product usage reporting and enterprise analytics across our products. You will operate...Senior$179k - $210k
...are high, and so are the rewards. Robinhood’s Analytics Engineering team, part of the Data Science organization, is the backbone of our decision-making... ...raw data into clear, actionable intelligence. As a Senior Analytics Engineer , you will be a key architect of Robinhood...SeniorWork at officeFlexible hoursShift work3 days per weekUnchain Data
Menlo Park, CA4 days ago$174.72k - $295.68k
...Infrastructure team, you will be responsible for building the end-to-end data pipeline for autonomous driving, covering the entire chain from... ...'s degree or higher in Computer Science, Software Engineering, Artificial Intelligence, or related fields.3-5+ years of experience...SeniorFull timePart timeOverseasXPENG Motors
Santa Clara, CA16 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
Related searches
- big data cloud engineer Sunnyvale, CA
- big data engineer Sunnyvale, CA
- big data developer Sunnyvale, CA
- big data devops engineer Sunnyvale, CA
- finance data engineer Sunnyvale, CA
- sr information security engineer Sunnyvale, CA
- junior big data engineer Sunnyvale, CA
- entry level big data engineer Sunnyvale, CA
- senior data engineer Sunnyvale, CA
- data developer Sunnyvale, CA