Software Engineer, Data Infrastructure
Full-time
Thinking Machines Lab
Responsibilities
- Design, build, and operate scalable, fault-tolerant distributed compute, data orchestration, and multimodal storage infrastructure for LLM research.
- Develop high-throughput systems for data ingestion, processing, transformation, training-data catalogs, deduplication, quality checks, and search.
- Build traceability, reproducibility, and quality-control systems across the data lifecycle.
- Implement and maintain monitoring and alerting for platform reliability and performance.
- Collaborate with research teams to improve data quality, unlock features, and accelerate training cycles.
- Own projects end-to-end and work across teams and technology stacks to deliver infrastructure.
Requirements
- Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
- Proficiency in at least one backend language, specifically Python or Rust.
- Fluency with distributed compute frameworks such as Apache Spark or Ray.
- Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
- Comfort operating across the stack and owning projects end-to-end.
- Preferred: hands-on experience with Kafka, dbt, Terraform, and Airflow.
- Preferred: experience building a web crawler and scaling deduplication, data mining, and search systems.
- Preferred: strong knowledge of file formats and storage systems, including Parquet and Delta Lake.
- Preferred: proactive documentation, testing, and development of tools that empower teammates.
Benefits
- Based in San Francisco, California or New York, New York.
- Generous health, dental, and vision benefits.
- Unlimited paid time off.
- Paid parental leave.
- Relocation support as needed.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Software Engineer, Data Infrastructure in San Francisco, CA vacancy
- ...About the Team Data Platform at OpenAI owns the foundational data stack powering critical... ...with Airflow; and support ML feature engineering tooling such as Chronon. Our mission is... ...and analytics. We’re not just scaling infrastructure – we’re redefining how people interact...SuggestedWork at officeRelocation package
$202k
...; Seattle, WA, United States; New York City, NY, United StatesSchedule: Full time, Level 5Posted: 2026-11-12Senior Software Engineer - Data Infrastructure, Mobility Simulation and PlanningAbout the role and teamHow can Uber plan pricing and incentives to meet financial...SuggestedFull timeWork at officeRemote work$266k
...Platform at OpenAI owns the foundational data stack powering critical product,... ...orchestration with Airflow; and support ML feature engineering tooling such as Chronon. Our mission is... ..., and analytics.We’re not just scaling infrastructure - we’re redefining how people interact...SuggestedWork at officeLocal areaRelocation packageFlexible hours$162k - $216k
...Software Engineer - Infrastructure, Data Platform San Francisco, California, United States Who We Are Baton is Ryder's in-house product development group focused on harnessing emerging technologies to redefine transportation and logistics. With $10B in freight...SuggestedFull timeWork at officeImmediate startRemote workMonday to Friday$162k - $216k
...design and ship category-defining software that enables Ryder and its 50... ...-facing software to the data platform that will power the... ...and creating impact for the engine of the American economy, you’... ..., and maintain the core data infrastructure that powers Baton's near real...SuggestedFull timeWork at officeImmediate start$7.5k
...benefits packages, technology talks by our experts, a beautiful modern office, daily catered lunches, and more.As a Software Engineer on our Data Infrastructure team, you will contribute to scaling and advancing our entire data infrastructure. This includes building data-...Work at officeLocal area- ...London, and Amsterdam.Making data driven decisions is key to... ...and guidance to teams across engineering, product, and business and help... ....Engineers on Data Infrastructure are domain experts in Data Warehouse... ....Qualifications6+ years of software engineering...Work experience placementLocal area
- ...Responsibilities Build scalable data-platform services supporting data ingress and egress for internal and external customers... ...usage. Requirements At least 1 year of professional software engineering experience. Prior software engineering internship experience...Full timeInternshipWork at officeFlexible hours3 days per week
- ...intelligence for the AI economy. We build the infrastructure that enables frontier AI labs and... ...Person Owns The infrastructure every data-driven team at Mercor runs on: Snowflake... ...team, you'll help set the standards other engineers build against. What They Do Day-to-...Full timeWork at officeLocal areaRelocation package
- ...community. Join us in using data, bold thinking, and a commitment... ...on Databricks. Our Data Engineers, Data Scientists, and Analytics... ...the platform frameworks and infrastructure that make downstream... ...capabilities About You10+ years of software or data platform engineering...Full timeContract workWork at officeLocal areaRemote workRelocationFlexible hours
$153k - $376k
...design and collaboration, join us! The Data Platform team at Figma builds and operates... ...AI researchers, machine learning engineers, data scientists, product engineers, and... ...ML Datalake, orchestration and pipeline infrastructure, and large-scale data ingestion and processing...Full timeRemote workWork from home- ...The Knowledge Platform team builds Exa's knowledge engine. We continuously synthesize information from across the web and beyond into... ...graph of what is true about the world, and keeping it correct as data changes. This is greenfield work with a massive scope. There...Full time
$300k - $400k
...building is human, and we're hiring people who want to build it. About the Role We’re looking for an engineer to join us and contribute to data infrastructure. You'll join a small, high-impact team responsible for architecting and scaling the core infrastructure...Full timeVisa sponsorshipWork visaRelocation package- ...clean, structured scientific data and AI are built into how... ...of Benchling’s Data Platform engineers, you’ll join a rapidly growing... ...warehouse. The Big Data Infrastructure team is responsible for enabling... ...~1+ years of professional software engineering experience ~ A...Remote jobFull timeTemporary workInternshipWork at officeLocal areaFlexible hours3 days per week
$200k - $400k
...grow as a team. About the Team The Infrastructure team builds and operates the foundations that power Decagon: networking, data, ML serving, developer platform, and real... ...'re hiring a Senior Data Infrastructure Engineer to design, build, and operate the data...Full timeWork at officeLocal area$180k - $225k
...generation of AI products. We build the infrastructure, tools, and model access that teams... ...As a c at fal, you will build the data infrastructure that turns our internal... ...This role sits at the intersection of software engineering and data engineering. You'll partner closely...Currently hiringRelocation package- ...Whatnot updates on our news and engineering blogs and join us as we... ...commerce. Role The Data Platform team at Whatnot builds... ..., multi-tenant streaming infrastructure rather than simply building... ...understanding You As our next Software Engineer, Data Platform, you...Local areaRemote workWork from homeHome office
$165k - $185k
...more, go to . THE ROLE ~ As a Senior Data Platform Engineer, you will build and operate the data infrastructure that supports Zeta’s products, analytics, machine... .... QUALIFICATIONS ~7+ years of software engineering experience. ~ Strong experience...Work at office$130.6k - $192k
...reliable on-demand, logistics engine for delivery! We're looking... ...help us develop a 24x7, global infrastructure system that powers DoorDash's... ...the TeamData Platform’s Data User Experience Engineering builds... ...of industry experience in software engineeringStrong backend fundamentals...Hourly payWork at officeLocal areaRemote workFlexible hours$165.2k - $223.6k
...listening platforms, and a flexible infrastructure to continuously enhance... ...Development Engineers on the ART19 team are end-to... ...designing, developing and deploying software solutions while holding a high... ...Experience building scalable data pipelines and analytics platforms...InternshipLocal areaFlexible hours$187k - $259k
About the RoleChime’s Data Platform team builds the infrastructure every engineering and analytics team depends on - ingestion, transformation, quality, governance, and self-serve tooling across batch and streaming workloads.You’ll own core platform systems at scale, design...Full timeWork at officeLocal areaRemote work$152k - $190k
...management, bill pay, and travel software, Brex enables founders and finance... ...support you need to grow your career.Data at BrexOur Scientists and Engineers work together to make data — and... ...numbers. The Data team at Brex develops infrastructure, statistical models, and products...Work at office3 days per week$130.6k - $192k
About the TeamDoorDash is a data driven organization and relies... ...organization owns all the infrastructure necessary to run an operationally... ...simplify data workflows for engineers, analysts, and ML... ...years of industry experience in software engineeringProficiency in using...Hourly payWork at officeLocal areaRemote workRelocationFlexible hours- ...Job Description Job Description About the Role We are looking for a Software Engineer to build and scale the data infrastructure powering next-generation robotics and Physical AI. In this role, you will develop high-volume data pipelines, processing systems, and...
$2,000 per month
...About Build AI Build AI is the data hyperscaler for Physical AI. Were vertically integrated across hardware manufacturing logistics... ...or Eventual/Daft. Demo-scale ETL is not this job Strong software engineering. Python and at least one systems language. Linux You...Full timeContract workWork at officeRelocation package- Mercor is hiring for a Project Lead to run and scale distinct data production businesses. You will own major parts of a self‑optimizing... ...large‑scale staffing of experts. The role requires a strong engineering background and comfort working with ambiguity. You will lead...
- ...Greylock to attack the biggest software category in the world. We previously... ...the role We're building the infrastructure foundation for a fast-growing... ...systems, infrastructure, and data infrastructure. We're hiring a staff-level engineer who spikes in data...Full timeSecond jobWork from homeFlexible hours
- Mercor in San Francisco is hiring a Project Lead to run and scale data production businesses, hiring and upskilling thousands of experts while managing throughput and quality. You will contribute to a self-optimizing automation platform that enables one person to run a...
- ...person. The Role You will own the data flywheel that trains our robots, from... ...dataset curation Own training data infrastructure: versioning, lineage, and... ...to fix What You Bring Strong software engineering in Python and at least one systems language...
- ...Software Engineer, InfrastructureAt Sierra, we're building a platform to enable every company... ...What You'll DoAs a Software Engineer, Infrastructure at Sierra, you will be responsible for... ...and ML models.Develop and maintain core data serving abstractions along with...Full timeFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Software Engineer, Data Infrastructure. Be the first to apply!
Related searches
- senior software design engineer San Francisco, CA
- software web developer San Francisco, CA
- senior software engineer San Francisco, CA
- software developer fintech San Francisco, CA
- software support engineer San Francisco, CA
- security software engineer San Francisco, CA
- software developer night shift San Francisco, CA
- software engineer co-op San Francisco, CA
- integration software engineer San Francisco, CA
- scientific software engineer San Francisco, CA




