Senior AI Data Infrastructure/Pipeline Engineer
$174.72k - $295.68kFull-time
XPENG
XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing (eVTOL) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting-edge R&D in AI, machine learning, and smart connectivity. As a core member of our AI Infrastructure team , you will be responsible for building the end-to-end data pipeline for autonomous driving, covering the entire chain from onboard data upload → cloud-based preprocessing → dataset production → model training / simulation input. In autonomous driving systems, the stability and efficiency of the pipeline directly determine the speed of algorithm iteration. We look forward to building a reliable, observable, and cost-effective data pipeline that supports the daily flow of petabyte-scale sensor data. Key Responsibilities
- Responsible for the design and construction of core data closed loop pipelines. Develop toolchains for data cleaning, annotation quality inspection, and data mining to support the algorithm team in quickly locating model error cases and driving iterative model optimization.
- Data Support for Production and R&D Processes. This includes log event tracking, connected vehicle data, internal and external data collection, data synchronization, data cleaning and standardization, data modeling, offline and real-time data processing, data as a service, and data visualization. Support business operations such as autonomous driving, smart cockpits, overseas data collection, and robotics data collection.
- Responsible for optimizing the performance of the entire data pipeline (collection, cleaning, conversion). Solve bottlenecks in large-scale data transmission, memory management, I/O, etc., and build a distributed data processing system with high throughput and low latency.
- Responsible for building a data management platform covering the entire process from data collection to data lake ingestion to model training. Implement capabilities for data version control, data lineage tracing, metadata management, and fast data retrieval to support unified data access and collaboration across multiple teams.
- Collaborate with the large model team and other technical teams to deeply understand business requirements, respond quickly, and ensure successful implementation.
- Bachelor's degree or higher in Computer Science, Software Engineering, Artificial Intelligence, or related fields.
- 3-5+ years of experience in large-scale data processing or data platform development.
- Proficiency in at least one programming language among Python / Go / Java. Solid software engineering foundation, good coding standards, and a strong sense of code quality.
- Hands-on project experience in at least two of the following areas:
- Design and development of large-scale data pipelines / ETL systems, with end-to-end experience in data cleaning, transformation, and loading.
- Production-level experience with distributed message queues (Kafka / Pulsar / RabbitMQ), familiar with stream processing paradigms.
- Experience with distributed data lake systems (e.g., Apache Iceberg), familiar with Iceberg's table format, partition evolution, snapshot isolation, etc., with practical performance tuning and deployment experience.
- Experience with columnar storage formats (e.g., Lance) and related query engines, with practical application in large model training.
- Hands-on experience using and optimizing relational databases (MySQL / PostgreSQL) and NoSQL databases (Redis / MongoDB). Understand metadata management and caching strategies.
- Experience in performance optimization and troubleshooting for large-scale distributed systems, able to quickly locate and resolve complex performance bottlenecks. Experience with Kubernetes / Docker containerization deployment.
- Strong cross-team communication and collaboration skills, high sense of responsibility, and proactive problem-solving attitude.
- Familiarity with closed-loop data in the embodied AI industry will be a huge plus.
- Some understanding of the autonomous driving industry, awareness of data closed loop and data flywheel concepts, and enthusiasm for this field.
- Experience with AI infrastructure or model training workflows (e.g., data loading, feature engineering, data preparation for model evaluation).
- Familiarity with data lake / data warehouse systems, with practical experience implementing data version control and data lineage tracing.
- Open-source contributions on GitHub or a technical blog, with continuous attention to the latest technological trends in big data / AI infrastructure.
- A fun, supportive and engaging environment.
- Infrastructures and computational resources to support your work.
- Opportunity to work on cutting edge technologies with the top talents in the field.
- Opportunity to make significant impact on the transportation revolution by the means of advancing autonomous driving.
- Competitive compensation package.
- Snacks, lunches, dinners, and fun activities.
Vacancy posted 10 days ago
Similar jobs that could be interesting for youBased on the Senior AI Data Infrastructure/Pipeline Engineer in California vacancy
$174.72k - $295.68k
...Senior AI Data Infrastructure/Pipeline Engineer Santa Clara, CA XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric...DataSeniorFull timeOverseasXPENG
Santa Clara, CA5 days ago$70 per hour
...technical talent with leading AI research labs. Headquartered... .... Position: Network Engineer - Data for Autonomous Systems annotation... ...for large-scale data pipelines . Interface directly with... ...Curiosity about how raw infrastructure data becomes machine learning...DataSeniorContract workSummer workRemote workMercor
San Francisco, CA20 days ago$160k - $300k
...pioneering foundational AI company for physical... ...technical data into real-time, actionable... ...to revolutionize how engineering decisions are made, turning... ...the Role As a Senior / Staff Infrastructure Engineer at Apiphany,... ...and maintaining CI/CD pipelines for production systems...DataSeniorWork at officeVisa sponsorshipFlexible hoursApiphany
San Francisco, CA15 hours ago$180k - $250k
...Senior Infrastructure Engineer Title of Role: Senior Infrastructure Engineer Location... ...Company Stage of Funding: Seed - AI, Devtools, Enterprise, Data Office Type: Hybrid Salary:... ...applications. Develop and maintain CI/CD pipelines to ensure seamless integration...DataSeniorWork at officeRecruiting from Scratch
San Francisco, CA4 days ago- A leading AI research organization is looking for a Senior Software Engineer, Infrastructure to build and scale infrastructure for experimentation and rollout tools. The role involves optimizing data ingestion, enhancing system reliability, and designing low-latency systems...DataSenior
- ...Hyperbolic Labs is on a mission to democratize AI by breaking down the barriers to... ...About the Role We're seeking a Senior Infrastructure Engineer to help build and scale Hyperbolic's... ...implementation Experience with storage and data infrastructure for AI/ML workloads,...DataSeniorRemote work
Hyperbolic Labs
San Francisco, CA5 days ago - ...decades-old legacy systems with AI-native technology that... ...Overview We're looking for a Senior Infrastructure Engineer to own and evolve the... ...quickly and reliably—CI/CD pipelines, cloud infrastructure, deployment... ..., and used telemetry data to diagnose production issues...DataSenior
Cascading AI Inc
San Francisco, CA4 days ago - AI Talent Now is seeking a Senior Software Engineer, Infrastructure to join their team in San Francisco. Ideal candidates will have 6-10 years of experience in building scalable, reliable infrastructure and a history of mentoring junior engineers. The role requires proficiency...DataSenior
- ...Judgment Labs builds infrastructure for Agent Behavior... ...We are looking for a Senior Infrastructure Engineer to architect and... ...compliance solutions (data residency, sovereignty... ...-throughput data pipelines, large-scale telemetry... ...infrastructure supporting AI systems...DataSenior
Judgment Labs
San Francisco, CA2 days ago $100k - $300k
...Senior Robot Infrastructure Engineer San Francisco Company Overview At Skild AI, we are building the world's first general purpose robotic... ...massive scale through data-driven machine learning is... ...designing secure, low-latency pipelines into cloud and web...DataSeniorLocal areaSkild AI
San Francisco, CA4 days ago$200k
...building the financial infrastructure that makes it easier... ...we build. As a Senior Infrastructure Engineer, you will shape the... ...platforms, deployment pipelines, infrastructure automation... ...intelligence (AI) tools to support parts... ...about how your data is processed, please...DataSeniorWork at office2 days per weekAngelList
San Francisco, CA2 days ago- ...Senior HPC & GPU Infrastructure Engineer Sciforium is an AI infrastructure company developing next-generation multimodal AI models and a proprietary, high-efficiency... ...system load. Vendor Liaison: Coordinate with data center staff, hardware vendors, and on-site...DataSeniorFlexible hours
Sciforium
San Francisco, CA7 days ago - ...TITLE: ML Data Infrastructure Engineer LOCATION: Sunnyvale CA or Remote Duration: 12+ Months Rate... ..., Airflow ( Cloud composer), Vertext AI , Datapipeline, ML Training Role... ...and implement scalable data processing pipelines for ML training and ~ validation...DataSeniorRemote work
Redolent
Sunnyvale, CA2 days ago $104k - $289.3k
...guide the Network Engineering and Operations team... ...next-generation data center fabric architectures... ..., Kubernetes, and AI/ML-enabled... ...Terraform, and CI/CD pipelines, including the... ...network security infrastructure, including operating... ...team outcomes to senior leadership. ~ Proven...DataSeniorTemporary workWork at officeRemote workWorldwide3 days per weekFive9
San Ramon, CA15 days ago- Granica, based in San Francisco, is seeking an expert in distributed systems to enhance their data infrastructure. This role involves architecting a global metadata substrate, developing intelligent data layouts, and implementing algorithms for efficient data representation...DataSeniorFlexible hours
- Decagon AI, Inc. is looking for a Senior Data Infrastructure Engineer to design and operate the data systems that power its AI products. The successful candidate will own critical data pipelines and storage layers, improving reliability and creating clear data pathways...DataSenior
- ...Reducto helps AI teams ingest real world enterprise data with state of the art accuracy. The... ...Capital, and are looking for senior engineers for our Platform team.... ...optimize document processing pipelines, and build the platform and infrastructure that makes state-of-the-...DataSeniorWork at officeLocal area
Reducto, Inc.
San Francisco, CA2 days ago - ...2004 when a visionary engineer, Fred Luddy, saw the potential... ..., bringing innovative AI-enhanced technology to... ..., agent-native infrastructure foundation that agents... ...enterprise data. Build and own the... ...ingestion and enrichment pipeline, and the entity, identity...DataSeniorFull timeWork at officeRemote workFlexible hoursShift work
ServiceNow
Mountain View, CA4 days ago $120k - $180k
...Position Overview: The Senior IT Infrastructure Engineer is responsible for designing, implementing, supporting... ...protect business systems, users, and data. The position requires hands-on... ...We may use artificial intelligence (AI) tools to support parts of the hiring...DataSeniorFull timeWork experience placementLocal areaRemote workVelo3D
Fremont, CA2 days ago$137k - $215k
...unprecedented speed and accuracy. Our AI-enabled platform turns siloed and disconnected data into operational intelligence —... ...ROLE We are looking to add a Network Engineer to expand our small but exceptional Infrastructure team. We need engineers who love taking...DataSeniorLocal areaPeregrine Technologies
San Francisco, CA4 days ago$190k - $250k
...Senior Network Engineer San Francisco About the Role As... ...maintaining our network infrastructure to ensure seamless... ...technologies in HPC-based data center networking.... ...developing network automation pipelines using Python, Ansible... ...Understanding of AI training workloads...DataSeniorFull timeTogether AI
San Francisco, CA5 days ago$225k - $275k
...Senior Staff Network Operations Engineer Crusoe Cloud is seeking a Senior Staff Network Operations Engineer... ..., including edge, backbone, data center fabric, and GPU cluster interconnects... ...that keep our hyperscale AI infrastructure healthy at scale. This is a senior...DataSeniorG2 Venture Partners
San Francisco, CA15 hours ago- ...We're looking for engineers who are excited about... ...We're seeking a Senior Platform Engineer... ...the foundational infrastructure that powers Orchestra... ...scientific data, complex financial modeling, and AI workloads with the... ...reliably. Design CI/CD pipelines, monitoring systems...DataSenior
Orchestra Bio
San Francisco, CA15 hours ago $225k - $275k
...As the only vertically integrated AI infrastructure company built from the ground up,... ...experts across energy, manufacturing, data center construction, and cloud... ...Role Crusoe Cloud is seeking a Senior Staff Network Operations Engineer to own production reliability across...DataSeniorTemporary workCrusoe
San Francisco, CA12 days ago- ...Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. Our novel... ...are seeking a highly skilled WAN Network Engineer to design, implement, manage, and optimize... ...Service Provider network, or Hyper-scale Data Center environment. ~ Industry certifications...DataSenior
CEREBRAS SYSTEMS INC.
Sunnyvale, CA3 days ago - A frontier research laboratory in San Francisco is seeking a Senior / Principal ML Engineer to enhance their ML infrastructure. The role involves designing experimental frameworks for data scientists, collaborating with various teams, and ensuring rigorous practices in...DataSenior
$132.1k - $279.8k
About Groq Groq delivers fast, efficient AI inference. Our LPU-based system powers... ..., anything is possible. Build fast. Senior Infrastructure Engineer Mission At Groq, we’re building a custom cloud from the ground up — one data center at a time. Our Infrastructure Platform...DataSenior$210k - $285k
...Description Job Description Senior Backend Engineer - $210K-$285K San... ...A rapidly growing AI-powered logistics... ...build and scale the core infrastructure powering mission-critical data systems, AI workflows,... ...Build and optimize data pipelines, messaging systems, and...DataSeniorRelocationDirect Line Workforce Solutions
San Francisco, CA15 days ago$150k - $200k
...our overall quality of life. As a Senior Wireless Network Design Engineer , you will lead the design, planning... ...We may use artificial intelligence (AI) tools to support parts of the hiring... ...would like more information about how your data is processed, please contact us....DataSeniorFull timeWork at officeImmediate startVisa sponsorshipShift workNight shiftE-Space
Saratoga, CA19 days ago$225k - $275k
...As the only vertically integrated AI infrastructure company built from the ground up,... ...experts across energy, manufacturing, data center construction, and cloud... ...this Role Crusoe Cloud is seeking a Senior Staff Network Deployment Engineer to serve as the technical owner of...DataSeniorTemporary workRemote workCrusoe
San Francisco, CA24 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior AI Data Infrastructure/Pipeline Engineer. Be the first to apply!
Related searches