Spark Platform Engineer - Scale, Scheduling & Observability
DoorDash
DoorDash is seeking a Software Engineer for the Spark Platform team to manage its in-house Spark deployment that serves the entire company. You will engage in tasks like Spark runtime upgrades, automation, and fostering partnerships across teams. Minimum qualifications included a B.S. in Computer Science and significant experience with distributed systems. The position is hybrid, requiring the candidate to be located in San Francisco, Sunnyvale, Seattle, or New York City. #J-18808-Ljbffr DoorDash
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Spark Platform Engineer - Scale, Scheduling & Observability in San Francisco, CA vacancy
$150k - $215k
Nscale is looking for a Principal Observability Platform Engineer in San Francisco, California. You will own the technical strategy for Nscale's observability... ...decisions that impact infrastructure and operations at scale. The ideal candidate will have over 8 years in SRE or...Suggested$320k - $405k
...seeking a Senior Kubernetes Engineer. In this role, you will own,... ...operate, and extend the Kubernetes scheduler for our accelerator fleets,... ...that the control plane scales efficiently and reliably. Your... ...operational teams to deliver robust platform capabilities. This position...SuggestedFlexible hours- About the Team The Spark Platform team owns and operates... ...shuffle service, cluster scheduler, and reliability... ...company at significant scale and continue to expand... ...Role As a Software Engineer on Spark Platform, you... ...automation, and the observability and incident automation...SuggestedHourly payWork at officeLocal areaRemote workRelocationFlexible hours
- About The Team The Spark Platform team owns and operates... ...shuffle service, cluster scheduler, and reliability... ...company at significant scale and continue to expand... ...As a Senior Software Engineer on Spark Platform,... ...Kubernetes operator), or the observability and SRE patterns that...SuggestedHourly payWork at officeLocal areaRemote workRelocationFlexible hours
- DoorDash USA is seeking a Software Engineer for the Spark Platform team to enhance its in-house Spark ecosystem. The role involves optimizing performance and operational capabilities across Spark deployments while partnering with other teams. Candidates should have extensive...SuggestedFlexible hours
- Manufact (YC S25), based in San Francisco, is seeking a cloud engineer to build and operate their cloud platform hosting customer MCP servers. You will set up observability systems, ensure security, and maintain system reliability as user load increases. The ideal candidate...
- ...Nscale Nscale is the GPU cloud engineered for AI. We provide cost-... ...About The Role As a Staff Observability Platform Engineer, you'll play a critical... ..., building platforms that scale, and helping engineering... ...with Slurm, Kubernetes GPU scheduling, or AI infrastructure...
- ...infrastructure in San Francisco. We seek an ML Platforms Engineer to build and run the orchestration layer that schedules workloads, allocates GPUs, and... ...multi-tenant primitives, and drive observability with metrics, logs, and traces at scale. #J-18808-Ljbffr Cumulus Labs (YC...
$310k
A leading AI research organization is looking for a Software Engineer for their Platform Systems team in San Francisco. You will design and build systems for large-scale AI training workloads, focusing on reliability and performance. Ideal candidates should have a deep...$160k - $225k
...Fable is the human risk platform that directly shapes... ...simplicity and enterprise scale, our agentic platform... ...Work closely with engineering, data science, and product... ...system reliability, observability, and performance... ...systems (e.g., Kafka, Spark, or similar technologies...Work experience placementRelocation packageFlexible hours$182k - $250k
Grow Therapy is looking for a Senior Platform Reliability Engineer to enhance reliability across teams. This role includes defining SLOs/SLAs, improving observability, and evolving incident response procedures. You will foster a culture of reliability while collaborating...$166k - $201k
...intelligence. We’re crafting the engine that powers a world where... ...AI — without sacrificing scale, speed, or sustainability.... ...expertise in building and operating observability platforms at scale. You will design,... ...paid time off and holiday schedule ~ Cell phone reimbursement...Temporary work- About Stack AI Stack AI is a no-code platform for designing, testing, and deploying... ...analytics you build. We're hiring a Platform Engineer, APIs & Observability to own both: the public API the... ...or distributed tracing at scale. Analytics datastores (ClickHouse, Druid...Contract work
$160k - $300k
About Hebbia The AI platform for investors and bankers that generates... .... The Role Platform engineering at Hebbia is about excellent... ...and can solve for exponential scale. Responsibilities Own critical... ...Proven experience enabling observability patterns. Ability to...Work experience placement$210k - $240k
...looking for a senior engineer who loves building... ...shape the cloud platform that powers next‑... ...who wants to help scale mission‑critical infrastructure... ...with reliability, observability, automation, and... ...rotating on‑call schedule, currently... ...tools such as Spark, Airflow, Kafka, or...Full timeContract work$180k - $200k
Lightning AI seeks an Observability Infrastructure Engineer to enhance observability systems across large-scale, GPU-enabled infrastructure. This position focuses on designing telemetry pipelines, improving data insights, and creating monitoring solutions. With a commitment...Remote job$150k - $215k
Principal Observability Platform Engineer - Nscale About Nscale Nscale is the GPU cloud engineered for AI. We provide cost‑effective, high‑performance... ...engineering bar across teams, and ensure our platform scales ahead of the business, not behind it. You understand...Flexible hours$150k - $250k
...startup in San Francisco seeks a founding platform engineer to build robust infrastructure... ...scalability. You will tackle challenges such as scaling systems for 10× traffic increases, optimizing data pipelines, and enhancing observability. The ideal candidate has 3-5 years in...- Systems Engineers at Inngest build the core of our product: the durable... ...able to build something that scales and functions well, a Systems... ...You understand how to observe, monitor, and maintain the systems... ...source project and cloud platform. You also helped define the specs...Live in
$295k - $405.5k
...Staff Machine Learning Platform Engineer, you will own the... ...science velocity at scale. This role will also... ...lifecycle management, and observability Lead adoption and... ...scalable ML workflows using Spark, Delta Lake, and... ...future. Work Location and Schedule Hybrid Faire...Work experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per week- Garuda Ventures is hiring a Sr Data Engineer to manage petabyte-scale data pipelines that will power Factory OS. The role involves designing data architectures... ...of data engineering experience with frameworks like Spark and proficiency in Python and SQL. This position offers a...
- ...TrueFoundry, and we're building it. We're looking for a Staff ML Platform Engineer - Large Scale Training (LLMOps/MLOps) to join the team. The Problem We... ...a control plane that handles: Intelligent routing with observability, cost policies, and fallback logic Centralized tool and...Flexible hours
$190k - $240k
...streaming TV ads. Our platform combines... ...Tatari continues to scale rapidly as TV advertising... ...a Data Platform Engineer, you will be... ...health using observability tooling; build alerting... ...DAG-based schedulers — or general job... ...data processing (Spark, PySpark, or similar...Work at office2 days per week- Edison Smart® is seeking a Senior Developer Platform Engineer in San Francisco, CA, to create and scale a next-generation Financial Cloud platform. This role focuses on designing and developing internal developer platforms to improve deployment reliability and developer...
- Rootly, based in San Francisco, is looking for an early engineer to join their team. This role focuses on shaping infrastructure that... ...with over 5 years of experience. You will enhance service observability, own CI/CD pipelines, and drive reliability improvements. Expect...
- StackAI is seeking a Platform Engineer to develop and maintain its public API and observability analytics. This role involves designing API standards, ensuring consistent user experience, and implementing extensive tracing and reporting for agent performance. Candidates...
- Crusoe is looking for a Senior Software Engineer in San Francisco to design, build, and scale their billing platform. This role is vital for delivering accurate and reliable billing experiences. Candidates should have experience in complex billing systems and a strong...
- Whatnot is seeking an AI/ML Engineer to design and scale core infrastructure powering LLM applications across the company. You will work with ML scientists to productionize models, build retrieval systems, and ground model outputs in Whatnot’s context. This role offers...Remote job
- Nscale is seeking a Staff Observability Platform Engineer based in San Francisco, California, to design and implement observability solutions for GPUs and AI workloads. The role entails partnering with SRE and engineering teams to enhance system visibility and reliability...
$190.8k - $267.1k
Reddit, Inc. is seeking a Senior Engineer for the Core Platform team to develop innovative solutions for scaling Reddit. You will mentor junior engineers and work on platform API projects that impact millions of users worldwide. The ideal candidate will have 5+ years in...Remote jobWorldwide
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Spark Platform Engineer - Scale, Scheduling & Observability. Be the first to apply!
Related searches
- data platform engineer San Francisco, CA
- client platform engineer San Francisco, CA
- platform engineer San Francisco, CA
- platform developer San Francisco, CA
- senior platform engineer San Francisco, CA
- platform engineering manager San Francisco, CA
- director of digital platform San Francisco, CA
- digital platform specialist San Francisco, CA
- power platform San Francisco, CA
- platform manager San Francisco, CA