Software Engineer, Data Ingestion
$209.7k - $266.8kFull-time
Wayve
Before the detail, here's the challenge you'd help us solve.
We build the embodied intelligence that moves real vehicles safely, and the ecosystem a billion machines will run on in the future. Very few people in AI can say this. Every role here, whatever the team, plugs into that.
Here’s what this particular role covers. ️ About Our Ingestion Systems Team Our Ingestion Systems team sits within Wayve’s AI Platform organisation and builds the data foundations behind our self-driving technology. We process vast amounts of real-world driving data so it can be used for annotation, model training and evaluation. Operating at petabyte scale, the team focuses on making these critical data flows reliable, efficient and easy to operate. Your day-to-day- Debug and unblock failing ingestion pipelines, stalled jobs and processing queues
- Investigate corrupt, malformed or unexpected data from a variety of sources.
- Optimise Spark jobs and batch-processing pipelines for throughput, reliability and compute efficiency.
- Design and implement more resilient pipelines so individual bad data segments do not block wider workflows.
- Improve monitoring, retries and failure recovery across multi-step ingestion workflows.
- Help prioritise and unblock important datasets for downstream annotation, data science and model-training teams, and contribute to the technical direction of Wayve's ingestion platform.
- High-volume pipelines that process real-world driving data for annotation, data science, model training and evaluation.
- Resilient systems that isolate problematic data rather than allowing a single bad segment to block a wider workflow.
- Orchestration across complex dependencies, queues, retries and priority data flows.
- Pragmatic improvements to the performance, cost and operability of our Spark-based data platform.
- Ingestion of varied data formats from partners, suppliers and other third-party sources.
- You have strong production experience with Apache Spark and Python.
- You have built, operated or debugged large-scale ingestion, ETL or distributed data-processing pipelines.
- You are comfortable diagnosing production failures involving corrupt data, stalled queues, retries and complex dependencies.
- You understand how to optimise high-throughput systems for reliability, performance and cost.
- You can learn an existing system quickly, work independently and deliver practical improvements in a fast-moving environment.
- Experience with Airflow, Flyte, Databricks, Delta Lake, Scala or Java would be an advantage.
- Initial call / recruiter screen (30 mins)
- Competency interview Python programming (60 mins), Hiring Manager interview (60 mins)
- Deep-dive technical interview: SQL domain (60 mins), Systems design (60 mins)
- Final interview: Mission and values alignment (45 mins)
Vacancy posted 10 days ago
Similar jobs that could be interesting for youBased on the Software Engineer, Data Ingestion in California vacancy
$172.5k - $260.1k
...Agentforce is the future of AI, and you are the future of Salesforce.The ExperienceSlack is looking for a senior software engineer to join the Data Ingestion Team. As part of the Data Engineering organization, we build and operate the platform that ingests data from...SuggestedFull time$70 per hour
...in simulation across 15+ U.S. states. Software Engineering builds the brains of Waymo's fully autonomous... ...car. We are looking for experienced data-minded software engineers to help us... ...Hands-on experience with systems that ingest, store, transform, and output data at scale...SuggestedHourly payFull timeInternshipSummer internshipRelocation package$172.5k - $260.1k
...ensure you are not duplicating efforts. Job Category Software Engineering Job Details About Salesforce Salesforce is the... ...Slack is looking for a senior software engineer to join the Data Ingestion Team. As part of the Data Engineering organization, we...SuggestedFull time$145k - $155k
...we are obsessed with enabling data teams to solve the world's... ...the whole data lifecycle from ingestion to ETL, BI, and all the way up... ...scheduling Vectorized execution engine Data security Resource... ...projects We are hiring Software Engineer - New Grads,...SuggestedFull timeWorldwide- ...addresses.About the roleThe Data Cloud team is building an end... ...We are building systems for ingestion, transformation, lineage, data... ...will doDevelop high-quality software with attention to detail using... ...platform spacesBuild and grow your engineering skills in different...SuggestedWork at office3 days per week
$125k - $160k
...with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, DATA (STARLINK) At SpaceX we're leveraging our experience in building... ...and scalable enterprise data platform empowering users to ingest, process, store, and query at petabyte-scale Create new...Temporary workWorldwideWeekend work$125k - $160k
...with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, FULL STACK (DATA & VIDEO ANALYSIS) The Vehicle Analysis Tooling team... ...flight depend on. You will own the full path: platforms that ingest and process high-volume engineering data, such as...Temporary work$178k - $223k
...Opportunity We're doing an AI-first engineering rebuild for a company that already... ...from scratch. You'll build the software, services, and privacy-safe data infrastructure that power products... ...APIs, and distributed systems for ingesting, processing, and serving data....Work at officeRelocation packageFlexible hours3 days per week$152k - $241.5k
...The NVIDIA DGXC Data Services team builds cloud-native systems, frameworks, and services... ...problems in AI: storage, access, ingestion, governance, observability, and data management... ...production environments. Use modern software engineering practices, including AI-assisted and...Full time- ...locations for team members for our Data Platform roles are our... ...putting trust at risk? As a Senior Engineer on our Data Platform team,... ...role at the intersection of Software Engineering and Data, where... ...entire data lifecycle, from ingestion to analytics to machine learning...Full timeWorldwideHome officeFlexible hours
- ...on the verge of an "infinite software crisis." How will we support,... ...observability platform for fast-moving engineering teams. Sazabi gives teams... ...Build and scale our log ingestion and processing pipeline... ...high-throughput, low-latency data processing Own storage, indexing...Full timeWork at office
- ...solved when researchers and engineers are hands-on with the physical... ...experience. That starts with a data engine: the pipelines and... ...training-ready datasets — from ingestion off real robots and capture devices... ...Requirements: ~2+ years of software engineering experience, with...Full timeImmediate start
$162k - $216k
...design and ship category-defining software that enables Ryder and its 50... ...-facing software to the data platform that will power the... ...and creating impact for the engine of the American economy, you’... ...data pipelines end to end from ingestion through processing to consumption...Full timeWork at officeImmediate startRemote workMonday to Friday- Andrenam in Torrance, CA is seeking a Software Engineer Intern to join our software team and help build the platform that brings ocean data to life. You will work on a real, scoped... ...learn the full-stack stack, from data ingestion to visualization, in a #J-18808-Ljbffr...Internship
$120k - $200k
Santa Clara, CAData Engineering - Data Infrastructure /Full-time /HybridPlusAI is a Physical AI... ...company pioneering AI-based virtual driver software for factory-built autonomous trucks.... ...libraries for vehicle-data transfer, ingestion, validation, domain-specific format...Full time$167.7k - $245.2k
...telemetry from more than 25 million devices worldwide, ingesting petabytes of data every day. This data powers critical network experiences... ...capabilities across Cisco’s platform.We’re looking for a Senior Software Engineer on the Data Pipeline team to help build and operate the...Full timeTemporary workLocal areaWorldwideFlexible hours- PLEASE CLICK HERE TO SEE *ALL* OF OUR JOB OPENINGS!Senior Software Engineer - API & Data ConnectorsThis role sits in Data Products, which... ...API & Data Connectors, you will own and extend core data ingestion framework and SDK abstractions. You will design and maintain...Local areaShift work
- ...com addresses.About The RoleThe Data Cloud team is building an end-to... ...pipelines. We are building systems for ingestion, transformation, lineage, data... ...on distributed processing engines and distributed databasesDevelop high-quality software with attention to detail using tech...Work at office3 days per week
- PLEASE CLICK HERE TO SEE *ALL* OF OUR JOB OPENINGS!Senior Software Engineer - Backend, Data ProductsAs a Senior Software Engineer on Backend, Data... ...pipelines composable and self-correcting.Develop connectors, ingestion, and transformation tooling for diverse, messy marketing...
- ...sent from @Rippling.com addresses.About the Data Cloud TeamRippling Data Cloud is a new... ...promptsTransformations: full publish access Pipelines: ingest data via managed connectors Lineage &... ...Rippling.Data Access - Owns the query engine (RQL, Rippling Query Language) that...Work at office3 days per week
$193.93k - $291.15k
...generalists where ML and systems engineering converge to push autonomy... ...forward. As a Senior Perception ML Data Infrastructure Engineer, you... ...tooling.Design the ingestion and pipelines that move sensor... ...YouBS/MS in Computer Science, Software Engineering, Robotics, or a related...Immediate startFlexible hours$155k - $185k
The OpportunityWe are looking for an experienced Software Engineer with a passion for building robust and scalable data infrastructure to join our Data Platform team.... ..., trustworthy dataDrive innovation in data ingestion, transformation, and delivery across the companyEstablish...Permanent employment$180k - $237.8k
...industrial robotics in the near future - with software and data at the core.Our mission is to make... ...developers. We are a dynamic team of engineers, roboticists, designers, and... ...engineering. You will be building distributed ingestion systems and services that interface...Full timeLocal area$185.2k - $326.8k
...Information Security, Platform Infrastructure Engineering, and more — provide support and guidance... ...yours to Block. The Role The Data Enablement team builds the platforms and... ..., and Data Science. As a Senior Software Engineer on the team, you will design, build...Full timeLocal areaRemote workFlexible hours$171k
...turn real-world operations into high-quality data for our autonomous partners. This team is... ...efficiency no one else can match.We are seeking a Software Engineer II to build the data infrastructure for AV Labs: the systems that ingest, store, and index petabyte-scale multimodal...Full timeWork at officeRemote work$125k - $160k
...SpaceX is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, DATA - TOP SECRET CLEARANCE (STARLINK) At SpaceX we're leveraging our experience in building rockets and spacecraft to deploy...Temporary workWorldwideWeekend work$125k - $150k
...actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, PROPULSION SIMULATION & DATA ANALYSIS (RAPTOR) The Raptor Systems Modeling and Control Team is responsible for the power-balance,...Temporary work$224k - $356.5k
...can make a lasting impact on the world.NVIDIA is seeking a Software Engineer, Data Systems to join our Santa Clara, CA team. This opportunity... ...query path for engineering performance data. This includes ingesting, joining, storing, and delivering it at the scale of a GPU...Full timeContract work$125k - $160k
...is actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER (AI DATA ENGINEERING) Be a member of the Artificial Intelligence Data Engineering Software team, building AI-powered tools and automation...Temporary workWeekend work$135k - $175k
...actively developing the technologies to make this possible, with the ultimate goal of enabling human life on Mars. SOFTWARE ENGINEER, INFERENCE (AI DATA ENGINEERING) The application software team is the central nervous system of SpaceX - we create mission critical...Temporary workRemote workWorldwideWeekend work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer, Data Ingestion. Be the first to apply!
Related searches
- software engineer internship remote California
- software engineer remote California
- graduate software developer no experience California
- information technology software engineer California
- senior software engineer remote California
- rust software engineer California
- part time software developer California
- junior software developer remote California
- software engineer travel California
- startup software engineer California




