Senior Spark Platform Engineer: Scale Data & ML Infra
Socket
DoorDash is seeking a Senior Software Engineer for the Spark Platform to define the technical direction of our in-house Spark deployment and shape the architecture for data, analytics, and ML compute. You will own cross-cutting runtime, shuffle service, and scheduler reliability, collaborating with product engineering and data teams. You will report to the Engineering Manager for Spark Platform and work from Bay Area/NYC/Seattle/other eligible locations in a hybrid setup. #J-18808-Ljbffr Socket
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Spark Platform Engineer: Scale Data & ML Infra in San Francisco, CA vacancy
- ...Software Solutions is hiring a Senior Data Engineer (Apache Spark) in San Francisco, USA . Lead the design of large-scale distributed data processing... ...Apache Spark and cloud platforms at APPIT Software in San Francisco... ...-tolerance Partner with ML engineers to deliver...SeniorFlexible hours
- DoorDash is hiring a Senior Software Engineer to lead the Spark Platform, setting the long-term direction for our in-house Spark deployment. You... ..., and scheduler, ensuring reliability at scale across data, analytics, and ML workloads. You will partner with the Engineering...Senior
- ...need through the platforms they already... ...seasoned software engineer to join Parafin... ...-generation Data Platform. This... ...businesses.As a Senior Software Engineer... ...dashboards, ML models, and customer... ...to dbt, scaling data ingestion,... ...and SQL.Hands-on Spark development experience...SeniorFlexible hours
$240k - $360k
About the RoleWe're looking for a Senior Staff Data Engineer to be the technical backbone of our Data & ML Platform team — the foundation powering... ...as the company's AI ambitions scale. You'll work in a modern stack including Python, SQL, Spark, dbt, Kafka, Flink, Databricks...SeniorWork at officeLocal areaImmediate startRemote workWorldwide3 days per week- ...Software Engineer We are looking for a foundational... ..., and evolution of our data platform. You will be part of the... ...Development: Develop and scale robust, fault-tolerant... ...Enable Data Science & ML: Create the foundational... ...technologies such as Spark, Flink, or Dask. Pipeline...Senior
Crustdata (YC F24)
San Francisco, CA1 day ago - ...cutting-edge technology company in San Francisco is seeking a Data Platform Engineer to drive architecture and implementation of core systems. The... ...and demonstrates strong analytical skills in fields such as ML and statistics. Responsibilities include planning technical roadmaps...
Hive
San Francisco, CA1 day ago - About the TeamThe Spark Platform team owns and operates DoorDash'... ...that powers the company's data, analytics, and ML workloads. We run Spark... ...the company at significant scale and continue to expand... ...tooling.About the RoleAs a Senior Software Engineer on Spark Platform, you will...SeniorHourly payWork at officeLocal areaRemote workRelocationFlexible hours
$186.07k - $218.9k
...the emerging onchain platform — and with it, the... ...Product Group engineers is to build a trusted... ...Infrastructure. The Data Platform team builds... ...batch processing (Spark, SQL), streaming services... ...data analysts, ML, and security &... ...design patterns such as scale-out, caching, key/...SeniorLocal area$155.6k - $306.8k
...Forward Deployed Engineers (FDE) don’t just... ...into enterprise-scale impact, pairing leading... ...you’ll doAs a Senior Microsoft AI&Data FDE, you will... ...and end users on platform capabilities and... ...experience with Spark, Airflow/dbt, streaming... ...data modeling or ML/data science...SeniorLocal area$170.4 per hour
...passionate about enabling Data & AI teams to solve... ...AI infrastructure platform so our customers... .... Founded by engineers, we leap at every opportunity... ...interaction to scaling our services and... ...dive forensics into Spark core internals and... ...Experience managing the ML lifecycle,...SeniorLocal areaWorldwide- Komodo Health is seeking a Senior Data Engineer to design, build, and operate large-scale data pipelines powering the Healthcare Map. You... ...and enable analytics, product, and AI/ML use cases. You’ll work with Python, SQL, Airflow, Spark, and AWS, driving reliability,...SeniorRemote job
$190k - $240k
...Staff Data Platform Engineer San Francisco, California, United States Tatari... ...team of former founders and senior leaders from companies... ...Yahoo, Tatari continues to scale rapidly as TV advertising enters... ...Distributed data processing (Spark, PySpark, or similar big...SeniorWork at office2 days per weekTatari
San Francisco, CA4 days ago- ...about enabling data teams to solve... ...infrastructure platform so our customers... ...business. Founded by engineers — and customer... ...with data to scaling our services... ...leveraging AI/ML to revolutionize... ...comfortable challenging senior leadership with... ..., Apache Spark, Delta Lake and...SeniorWorldwide
$160k - $200k
...the globe. We operate at a global scale and are looking for practical... ...essential goods where they are needed. As a Senior Data Engineer on the Data Platform team, you will build cloud data... ...system decisions, product analytics, and ML. You will own the data backbone...SeniorFull timeLocal areaZipline
San Francisco, CA9 days ago$170k - $220k
...Senior Software Engineer, Data Platform Emeryville, California, United States; Hybrid (2... ...Capital, Bezos Expeditions, Spark Capital, Insight Partners,... ...help design, build, and scale Profluent's data platform.... ...You will work closely with ML, bioinformatics, and program...SeniorProfluent
Emeryville, CA4 days ago- A growing tech company in San Francisco is looking for a Software Engineer to design and build scalable data systems. The ideal candidate has at least 3 years of experience in software engineering, particularly in data engineering. You will develop robust data pipelines...Senior
$156.06k - $211.14k
Afresh, the AI platform for grocery, began by tackling... ...in 2025, we have scaled to 6 enterprise-... ...them is not.As a Senior AI Platform Engineer, you build the AI and data platform that powers... ...software, data, or ML systems; an... ...stack (Databricks/Spark, MLflow, cloud warehouses...SeniorFull timeLive inWork at officeLocal areaRemote workWork from homeHome officeFlexible hours3 days per week$295k - $405.5k
...technology wholesale platform built on the... ...power of tech, data, and machine learning... ...this roleAs the Senior Staff Machine Learning Platform Engineer, you will own... ...of Faire’s ML platform. You will... ...science velocity at scale. This role will... ...workflows using Spark, Delta Lake, and...SeniorWork experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per week$300k
...stealth-mode hyperscale data center startup building an AI and cloud platform, powered by thousands of... ...for experimentation, full-scale model training, or... ..., and operate inference engines such as vLLM, SGLang, and... ...tolerant distributed systems (ML inference, HPC, or...SeniorPermanent employmentWorldwide- Physical Intelligence seeks a systems-focused ML infrastructure engineer to own scheduling, placement, and cluster management for large-scale model training. You will design multi-... ...engineers to translate workloads into scalable infra, optimize utilization, and build...
- ...defining AI agentic operating platform that healthcare teams rely... ...the role We are hiring a Senior Software Engineer to own the data platform that powers... ...operating production systems at scale, including incident... ...structured data, not pure ML research Comfort in customer...SeniorWork at officeFlexible hours2 days per week
$148k - $260k
...more visit: As a Senior/Staff Software Engineer embedded within our Autonomy... ...build the scalable ML data pipelines necessary... ...s autonomous driving platform. Working closely with... ...and evaluation at scale. - Improve data sampling... ...(Apache Hadoop/Spark) or orchestration frameworks...SeniorFull timeWork at officeWork from homeFlexible hoursWaabi
San Francisco, CAa month ago$120k - $150k
The Opportunity As a Data Engineer, you are passionate and drive excellence... ...how data gets delivered, and scale our data systems where needed... ..., and monitoring of the platform Create data tools for the organization... ...Science algorithms and/or ML ops Experience with a variety...Full time3 days per week- ...Amsterdam.Making data-driven decisions is... ...that, we need to scale our data systems while... ...to teams across engineering, product, and... ...the broader Data Platform team. You will collaborate... ...technologies like Spark, Kafka.You... ...needs while balancing infra and business needs...SeniorFull timeWork experience placementLocal areaFlexible hours
$192k - $240k
...is the intelligent finance platform that enables companies to... ...need to grow your career.Data at BrexOur Scientists and Engineers work together to make data... ...professional experience in a data infra or data platform... ...building and maintaining large-scale modern data...SeniorWork at office3 days per week- ...Omni is the AI analytics platform that turns company data into a trusted source of... ...What you'll do Build and scale the data platform that... ...backend systems, workflow engines, or data / ML infrastructure ~ Deep experience... ..., BigQuery, dbt, Spark, Airflow, Dagster, or...Full time
Omni
San Francisco, CA21 hours ago $146.4k - $235.38k
...unleashes business-critical data that is trapped inside of... ...Intelligent Agreement Management platform, companies can create,... ...doThe Data and AI Platform Engineer will design, build, and operate... ..., data science, and AI/ML capabilities at scale. Reporting to the Sr. Director...SeniorContract workWork at officeLocal areaRemote work2 days per week- ...About the job Senior Data Engineer Role: Senior Data Engineer We... ...products and leverage our PB scale data. Our business is data driven... ...systems Hadoop/MapReduce, Spark, Hive, Kafka and Oozie/Airflow... ...knowledge of open-source ML frameworks and end-to-end model...SeniorWork experience placement
Inizio Partners
San Francisco, CA20 hours ago $150.4k - $277.6k
...Senior Data Engineer Imagine what you could do here. At Apple, new... ...with a background in large scale software development and... ...team that drives Apache Spark's state-of-the art features... ...experience with big data platforms ~ Familiarity with ML/AI workflows and feature...SeniorRelocationApple
San Francisco, CA4 days ago- ...application development and AI/ML, and our people-first... ...We are looking for a Senior Data Engineer to design and build... ...a thematic research platform that processes large... ...work with Snowflake, Spark, and AWS to deliver... ...ETL/ELT processes at scale using Python, incorporating...SeniorWork at officeRemote workVisa sponsorshipWork visaFlexible hours
AgileEngine
San Francisco, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Spark Platform Engineer: Scale Data & ML Infra. Be the first to apply!
Related searches
- data platform engineer San Francisco, CA
- client platform engineer San Francisco, CA
- platform engineering manager San Francisco, CA
- platform developer San Francisco, CA
- senior platform engineer San Francisco, CA
- platform engineer San Francisco, CA
- aws data engineer San Francisco, CA
- data engineer graduate San Francisco, CA
- big data devops engineer San Francisco, CA
- big data developer San Francisco, CA