Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Software Engineer, Data Infrastructure

Full-time

Thinking Machines Lab

Responsibilities

  • Design, build, and operate scalable, fault-tolerant distributed compute, data orchestration, and multimodal storage infrastructure for LLM research.
  • Develop high-throughput systems for data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search.
  • Build traceability, reproducibility, and quality-control systems across the data lifecycle.
  • Implement and maintain monitoring and alerting for platform reliability and performance.
  • Collaborate with research teams to improve data quality, unlock features, and accelerate training cycles.
  • Own projects end-to-end and work across teams and technology stacks to deliver infrastructure.

Requirements

  • Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
  • Proficiency in at least one backend language, specifically Python or Rust.
  • Fluency with distributed compute frameworks such as Apache Spark or Ray.
  • Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
  • Comfort operating across the stack and owning projects end-to-end.
  • Preferred: hands-on experience with Kafka, dbt, Terraform, and Airflow.
  • Preferred: experience building a web crawler and scaling deduplication, data mining, and search systems.
  • Preferred: strong knowledge of file formats and storage systems, including Parquet and Delta Lake.
  • Preferred: proactive documentation, testing, and development of tools that empower teammates.

Benefits

  • Based in San Francisco, California or New York, New York.
  • Generous health, dental, and vision benefits.
  • Unlimited paid time off.
  • Paid parental leave.
  • Relocation support as needed.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Software Engineer, Data Infrastructure in San Francisco, CA vacancy
  •  ...About the Team Data Platform at OpenAI owns the foundational data stack powering critical...  ...with Airflow; and support ML feature engineering tooling such as Chronon. Our mission is...  ...and analytics. We’re not just scaling infrastructure – we’re redefining how people interact... 
    Suggested
    Work at office
    Relocation package

    OpenAI

    San Francisco, CA
    5 days ago
  • $202k

     ...; Seattle, WA, United States; New York City, NY, United StatesSchedule: Full time, Level 5Posted: 2026-11-12Senior Software Engineer - Data Infrastructure, Mobility Simulation and PlanningAbout the role and teamHow can Uber plan pricing and incentives to meet financial... 
    Suggested
    Full time
    Work at office
    Remote work

    Uber

    San Francisco, CA
    4 days ago
  • $266k

     ...Platform at OpenAI owns the foundational data stack powering critical product,...  ...orchestration with Airflow; and support ML feature engineering tooling such as Chronon. Our mission is...  ..., and analytics.We’re not just scaling infrastructure - we’re redefining how people interact... 
    Suggested
    Work at office
    Local area
    Relocation package
    Flexible hours

    OpenAI

    San Francisco, CA
    2 days ago
  • $162k - $216k

     ...Software Engineer - Infrastructure, Data Platform San Francisco, California, United States Who We Are Baton is Ryder's in-house product development group focused on harnessing emerging technologies to redefine transportation and logistics. With $10B in freight... 
    Suggested
    Full time
    Work at office
    Immediate start
    Remote work
    Monday to Friday

    Baton

    San Francisco, CA
    5 days ago
  • $162k - $216k

     ...design and ship category-defining software that enables Ryder and its 50...  ...-facing software to the data platform that will power the...  ...and creating impact for the engine of the American economy, you’...  ..., and maintain the core data infrastructure that powers Baton's near real... 
    Suggested
    Full time
    Work at office
    Immediate start

    Baton Trucking

    San Francisco, CA
    2 days ago
  • $7.5k

     ...benefits packages, technology talks by our experts, a beautiful modern office, daily catered lunches, and more.As a Software Engineer on our Data Infrastructure team, you will contribute to scaling and advancing our entire data infrastructure. This includes building data-... 
    Work at office
    Local area

    The Voleon Group

    Berkeley, CA
    3 days ago
  •  ...London, and Amsterdam.Making data driven decisions is key to...  ...and guidance to teams across engineering, product, and business and help...  ....Engineers on Data Infrastructure are domain experts in Data Warehouse...  ....Qualifications6+ years of software engineering... 
    Work experience placement
    Local area

    Plaid Financial

    San Francisco, CA
    5 days ago
  •  ...Responsibilities Build scalable data-platform services supporting data ingress and egress for internal and external customers...  ...usage. Requirements At least 1 year of professional software engineering experience. Prior software engineering internship experience... 
    Full time
    Internship
    Work at office
    Flexible hours
    3 days per week

    Benchling

    San Francisco, CA
    18 hours ago
  •  ...intelligence for the AI economy. We build the infrastructure that enables frontier AI labs and...  ...Person Owns The infrastructure every data-driven team at Mercor runs on: Snowflake...  ...team, you'll help set the standards other engineers build against. What They Do Day-to-... 
    Full time
    Work at office
    Local area
    Relocation package

    mercor

    San Francisco, CA
    5 days ago
  •  ...community. Join us in using data, bold thinking, and a commitment...  ...on Databricks. Our Data Engineers, Data Scientists, and Analytics...  ...the platform frameworks and infrastructure that make downstream...  ...capabilities About You10+ years of software or data platform engineering... 
    Full time
    Contract work
    Work at office
    Local area
    Remote work
    Relocation
    Flexible hours

    Lending Club

    San Francisco, CA
    2 days ago
  • $153k - $376k

     ...design and collaboration, join us! The Data Platform team at Figma builds and operates...  ...AI researchers, machine learning engineers, data scientists, product engineers, and...  ...ML Datalake, orchestration and pipeline infrastructure, and large-scale data ingestion and processing... 
    Full time
    Remote work
    Work from home

    Figma

    San Francisco, CA
    5 days ago
  •  ...The Knowledge Platform team builds Exa's knowledge engine. We continuously synthesize information from across the web and beyond into...  ...graph of what is true about the world, and keeping it correct as data changes. This is greenfield work with a massive scope. There... 
    Full time

    exa

    San Francisco, CA
    5 days ago
  • $300k - $400k

     ...building is human, and we're hiring people who want to build it. About the Role We’re looking for an engineer to join us and contribute to data infrastructure. You'll join a small, high-impact team responsible for architecting and scaling the core infrastructure... 
    Full time
    Visa sponsorship
    Work visa
    Relocation package

    thinkingmachines

    San Francisco, CA
    1 day ago
  •  ...clean, structured scientific data and AI are built into how...  ...of Benchling’s Data Platform engineers, you’ll join a rapidly growing...  ...warehouse. The Big Data Infrastructure team is responsible for enabling...  ...~1+ years of professional software engineering experience ~ A... 
    Remote job
    Full time
    Temporary work
    Internship
    Work at office
    Local area
    Flexible hours
    3 days per week

    benchling

    San Francisco, CA
    20 hours ago
  • $200k - $400k

     ...grow as a team. About the Team The Infrastructure team builds and operates the foundations that power Decagon: networking, data, ML serving, developer platform, and real...  ...'re hiring a Senior Data Infrastructure Engineer to design, build, and operate the data... 
    Full time
    Work at office
    Local area

    decagon

    San Francisco, CA
    5 days ago
  • $180k - $225k

     ...generation of AI products. We build the infrastructure, tools, and model access that teams...  ...As a c at fal, you will build the data infrastructure that turns our internal...  ...This role sits at the intersection of software engineering and data engineering. You'll partner closely... 
    Currently hiring
    Relocation package

    fal

    San Francisco, CA
    2 days ago
  •  ...Whatnot updates on our news and engineering blogs and join us as we...  ...commerce. Role The Data Platform team at Whatnot builds...  ..., multi-tenant streaming infrastructure rather than simply building...  ...understanding You As our next Software Engineer, Data Platform, you... 
    Local area
    Remote work
    Work from home
    Home office

    Whatnot

    San Francisco, CA
    5 days ago
  • $165k - $185k

     ...more, go to .   THE ROLE   ~ As a Senior Data Platform Engineer, you will build and operate the data infrastructure that supports Zeta’s products, analytics, machine...  ....  QUALIFICATIONS   ~7+ years of software engineering experience.  ~ Strong experience... 
    Work at office

    Zeta Global

    San Francisco, CA
    4 days ago
  • $130.6k - $192k

     ...reliable on-demand, logistics engine for delivery! We're looking...  ...help us develop a 24x7, global infrastructure system that powers DoorDash's...  ...the TeamData Platform’s Data User Experience Engineering builds...  ...of industry experience in software engineeringStrong backend fundamentals... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    1 day ago
  • $165.2k - $223.6k

     ...listening platforms, and a flexible infrastructure to continuously enhance...  ...Development Engineers on the ART19 team are end-to...  ...designing, developing and deploying software solutions while holding a high...  ...Experience building scalable data pipelines and analytics platforms... 
    Internship
    Local area
    Flexible hours

    Amazon

    San Francisco, CA
    1 day ago
  • $187k - $259k

    About the RoleChime’s Data Platform team builds the infrastructure every engineering and analytics team depends on - ingestion, transformation, quality, governance, and self-serve tooling across batch and streaming workloads.You’ll own core platform systems at scale, design... 
    Full time
    Work at office
    Local area
    Remote work

    Chime

    San Francisco, CA
    5 days ago
  • $152k - $190k

     ...management, bill pay, and travel software, Brex enables founders and finance...  ...support you need to grow your career.Data at BrexOur Scientists and Engineers work together to make data — and...  ...numbers. The Data team at Brex develops infrastructure, statistical models, and products... 
    Work at office
    3 days per week

    Brex

    San Francisco, CA
    3 days ago
  • $130.6k - $192k

    About the TeamDoorDash is a data driven organization and relies...  ...organization owns all the infrastructure necessary to run an operationally...  ...simplify data workflows for engineers, analysts, and ML...  ...years of industry experience in software engineeringProficiency in using... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Relocation
    Flexible hours

    Doordash

    San Francisco, CA
    1 day ago
  •  ...Job Description Job Description About the Role We are looking for a Software Engineer to build and scale the data infrastructure powering next-generation robotics and Physical AI. In this role, you will develop high-volume data pipelines, processing systems, and... 

    IntelliPro Group Inc.

    San Francisco, CA
    14 days ago
  • $2,000 per month

     ...About Build AI Build AI is the data hyperscaler for Physical AI. Were vertically integrated across hardware manufacturing logistics...  ...or Eventual/Daft. Demo-scale ETL is not this job Strong software engineering. Python and at least one systems language. Linux You... 
    Full time
    Contract work
    Work at office
    Relocation package

    Build AI

    San Francisco, CA
    4 days ago
  • Mercor is hiring for a Project Lead to run and scale distinct data production businesses. You will own major parts of a self‑optimizing...  ...large‑scale staffing of experts. The role requires a strong engineering background and comfort working with ambiguity. You will lead... 

    Mercor

    San Francisco, CA
    2 days ago
  •  ...Greylock to attack the biggest software category in the world. We previously...  ...the role We're building the infrastructure foundation for a fast-growing...  ...systems, infrastructure, and data infrastructure. We're hiring a staff-level engineer who spikes in data... 
    Full time
    Second job
    Work from home
    Flexible hours

    Lightfield

    San Francisco, CA
    5 days ago
  • Mercor in San Francisco is hiring a Project Lead to run and scale data production businesses, hiring and upskilling thousands of experts while managing throughput and quality. You will contribute to a self-optimizing automation platform that enables one person to run a... 

    Mercor

    San Francisco, CA
    2 days ago
  •  ...person. The Role You will own the data flywheel that trains our robots, from...  ...dataset curation Own training data infrastructure: versioning, lineage, and...  ...to fix What You Bring Strong software engineering in Python and at least one systems language... 

    Kovari

    San Francisco, CA
    4 days ago
  •  ...Software Engineer, InfrastructureAt Sierra, we're building a platform to enable every company...  ...What You'll DoAs a Software Engineer, Infrastructure at Sierra, you will be responsible for...  ...and ML models.Develop and maintain core data serving abstractions along with... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Software Engineer, Data Infrastructure. Be the first to apply!