Data Infrastructure Engineer for Large-Scale AI Pipelines
Jobleads-US
Thinking Machines Lab Inc. in San Francisco is seeking an engineer to help build and scale the core data infrastructure for distributed training pipelines and multimodal data catalogs.
You will work with researchers to accelerate experiments, develop datasets, and improve reliability across LLM research platforms, using Spark, Kafka, Beam, Ray, and Delta Lake.
This role emphasizes end-to-end ownership, collaboration across teams, and building from the ground up in a fast-moving environment.
#J-18808-Ljbffr Jobleads-USVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Infrastructure Engineer for Large-Scale AI Pipelines in San Francisco, CA vacancy
- ...Labs is seeking a Senior Software Engineer, Data Infrastructure, to own the petabyte-scale Postgres serving layer, optimize... ...queries, and cut storage costs with AI-assisted analysis. You will... ...changes, and maintain real-time data pipelines across CDC tools. You will...Pipeline
- ...Collinear AI, Inc. is seeking a hands-on Member of Technical Staff - Engineering (Data Infrastructure) to own systems that distill terabytes of real-world data into actionable intelligence. You will design pipelines, graph-based record linkage, and high-performance...Pipeline
$176k - $179.5k
...and cutting-edge engineers. Your team is led... ...senior leaders and is scaling to a large team of full-stack engineers, data engineers, data... ...and/or generative AI will serve you well... ...modification of CI/CD pipelines, working with multiple cloud infrastructures (AWS, Azure, GCP)...PipelineApprenticeshipEasy workMcKinsey & Company
San Francisco, CA4 days ago$153k - $250k
...where economies of scale for volume... ...is the founding data hire. You will... ...work with the Infrastructure team to manage... ...peer to Platform Engineering and to the team... ...expect you to use AI agents as part... ...: We collect large amounts of telemetry... ...in the pipeline: Security defines...PipelinePermanent employmentContract workImmediate start$246.5k - $339k
...the power of tech, data, and machine... ...About this role:Our Engineering organization owns... ...migration of existing pipelines onto the new one without... ...and run data infrastructure that other teams... ...on, at meaningful scale, and you've been the... ...latest enterprise AI tools, to help you...PipelineWork experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per weekFaire
San Francisco, CA2 days ago$150k - $200k
...Aircall is a unicorn, AI-powered customer... ...issues faster, and scale customer-facing teams... ...customer‑obsessed, data‑driven, and focused... .... Right now, our engineers wear two hats — owning the infrastructure and the business datasets... ...own their own pipelines without filing a...PipelineWorldwideEngg
San Francisco, CA16 hours ago- ...Airwallex is the AI-native... ...5 to build the infrastructure global commerce... ...agnostic, and expect engineers to own problems... ...Senior Manager of Data Platform... ...optimizing for scale, performance, reliability... ...from legacy pipelines to the... ...Databricks, and large-scale data workloads...Pipeline
$140k - $200k
...future of Physical AI. Our advanced vision... ...robots to smart infrastructure - building a safer... ...world. The Sr. Data Infrastructure / Quality Engineering role will play a crucial... ...for scalable pipelines capable of handling... ...validation, and production-scale release...PipelineFull timeWork experience placementLocal areaOuster
San Francisco, CAmore than 2 months ago$208.59k - $429.45k
....At Pinterest, AI isn't just a feature... ...here.The Data Product Platform... ...and an exadata-scale data warehouse.... ...assets and capable infrastructure effectively.... ...Staff Software Engineer to provide... ...visualization, data pipeline authoring, and... ...experience with large scale data...PipelineWork at officeLocal areaRemote workRelocationRelocation packagePinterest
San Francisco, CA3 days ago- ...Google Cloud Data Engineer Are you ready to step up to... ...the world. The sheer scale of our capabilities and... ...of considerations for large scale architecting, solutioning... ..., provision infrastructure, secure and reliable data... ...of the predictive pipeline to minimize labor in development...PipelineWork experience placement
ClifyX
San Francisco, CA1 day ago - ...Built on a native, AI-powered platform... ...decade of proprietary data—including billions... ...Data Platform Engineer at Later, you'll own... ..., and evolve the infrastructure, backend services,... ..., dashboards, and pipeline maintenance, and it... ...data systems at scale and improving their...PipelineFull timeLive in
Later
San Francisco, CA2 days ago - ...behind the curve on AI. Today,... ...growing, unsolved data problem as a consequence... ...do. The Data Infrastructure team builds the... ..., reporting engine, and AI agents.... ...Ingestion at scale: incremental,... ...→ apply pipeline so schema intent... ...and give them large mandates with high...Pipeline
Numeric
San Francisco, CA16 hours ago $211k - $251k
...depth in either platform engineering or product... ...designed, built, and run large-scale data systems processing terabytes... ...a strong interest in AI products and a high... ...develop real-time pipelines, analytics products,... ...own real-time event infrastructure, streaming and batch...PipelineFull timeFlexible hoursSierra
San Francisco, CA5 days ago- Monaco is building an AI-native revenue... ...recording, enrichment, pipeline management with one unified... ...the chance to both scale our core systems and... ...We're looking for a Data Platform Engineer to help build Monaco'... ...context systems, and infrastructure that power our AI-driven...PipelineWork at office
- ...Gamma is seeking a data engineer to own our end-to-end event pipeline and data infrastructure as we scale to hundreds of millions of users. You will define the technical strategy for our data stack, architect distributed systems, and drive data-driven decision making...Pipeline
Gamma
San Francisco, CA3 days ago - ...Meta seeks a Data Engineer, Analytics (Technical Leadership) to lead... ...own data architectures for large-scale projects, design data models... ...drive improvements to data pipelines and governance. This... ...Meta's mission to scale data infrastructure for adjacent products and user...Pipeline
- ...invites a strategic Head of Data Engineering to lead the UK&I Data Engineering... ..., governance, and AI adoption while aligning with... ...of data products, ingestion pipelines, and reporting, building a culture... ..., Security, and Analytics to scale data platforms in a compliant...Pipeline
$150.4k - $225.3k
...Software Engineer - Generative Data Platform, Evaluation San Francisco... ...we operate.In the AI & Machine Learning (... ..., and images into large-scale generated datasets.... ...on production infrastructure.Our team builds with... ...multi-stage production pipeline. Operate and debug...PipelineWork experience placementRelocation- ...intelligence to power the AI economy. We're a leading AI data company, building... ...brings this same infrastructure to Fortune 500... ...the standards other engineers build against. What... ...Airflow/Astronomer pipeline reliability and on-... ...security, PII) at scale - Comfort owning vendor...PipelineWork at officeRelocation package
$190k - $240k
Senior Data Platform Engineer San Francisco, California, United States... ...application of AI and machine learning,... ...Tatari continues to scale rapidly as TV advertising... ...is a systems and infrastructure position first. As a... ...operating message queues and pipelines) Infrastructure-as-...PipelineWork at office2 days per week- ...the foundational data infrastructure that powers cutting-edge AI applications leveraging... ...scalable data pipelines, manage secure data... ...hypotheses into engineered features, manage secure... ...on innovating and scaling next-generation AI... ...in a complex and largely autonomous work...PipelineApprenticeshipWork at officeLocal areaEasy work
McKinsey & Company
San Francisco, CA4 days ago $86.7k - $170.9k
...Join Deloitte’s Core AI & Data practice and help organizations... ...data foundations, and scale analytics and... ...As a Databricks Data Engineer, you will support the... ...solutions that enable large-scale transformation.... ...build, and maintain data pipelines and data engineering solutions...PipelineLocal areaVisa sponsorshipDeloitte
San Francisco, CA3 days ago- ...shape the future of AI. Our mission: make... ...the trusted data backbone for the world... ...data systems and pipelines that power our research... ...credibility as an engineer and now multiplies... ..., scalable data infrastructure. What you'll do... ...pipelines at large scale (tens of TBs to PBs...PipelineWork at officeVisa sponsorship
$180k - $270k
...Job Description Staff Data Engineer (Founding Team) Location:... ...elite team in SF building an AI-native platform that acts... ...and serving Design and scale robust ETL/ELT pipelines across multiple enterprise... ...pipelines Build and optimize large-scale data processing...PipelineWork at officeImmediate startXpertalent
San Francisco, CAa month ago- ...Data Infrastructure Engineer You'll build and scale the data infrastructure that powers Surf — the... ...Build resilient multi-chain pipelines with unified schemas... ...evaluation loops to improve AI answer quality Drive... ...(Flink/Spark) for large, heterogeneous datasets...Pipeline
Cyber Services Inc
San Francisco, CA16 hours ago $150k - $200k
Our client, a growing AI and Data organization, is hiring a Founding Data Engineer to join the team. The successful... ...the backend systems, data pipelines, connector frameworks and graph... ..., building and operating large-scale data infrastructure in production environments....PipelineFull timeAlldus International Consulting Ltd
San Francisco, CAa month ago- ...Cartesia is hiring a Research Engineer, Data Infrastructure to build datasets and systems powering pretraining... ...scalable, high-throughput data pipelines and collaborate with research to optimize... ...-case, with a focus on cutting-edge AI research. #J-18808-Ljbffr Jobleads...PipelineWork at officeVisa sponsorship
$196k - $269.5k
...using the power of tech, data, and machine learning... ...passionate about using data and AI to shape the future of... ...Analysts and Software Engineers to support product... ...AI to streamline data pipeline development and... ...developing and supporting large-scale, high-performance data...PipelineWork experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per weekFaire
San Francisco, CA1 day ago$85 - $95 per hour
...technology company to hire a Data Engineer, Principal for its... ...and shared data infrastructure for nonprofit, community... ...automation and AI deployment across core... ...implementation of scalable data pipelines supporting enterprise... ...to design and scale large-volume, high-...PipelineContract workReleady
San Francisco, CAa month ago$50 per hour
...DescriptionJoin Lockheed Martin’s Data & AI Enablement... ...transformation. As a lead data engineer, you will design,... ...data models and pipelines - from raw data ingestion... ...performance with auto‑scaling, caching, and Delta... ...related field.7 + years of large-scale data...PipelineFull timeTemporary workWork experience placementCasual workFlexible hoursLockheed Martin
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Infrastructure Engineer for Large-Scale AI Pipelines. Be the first to apply!
Related searches
- senior data center engineer San Francisco, CA
- senior cloud data engineer San Francisco, CA
- senior data integration developer San Francisco, CA
- data developer San Francisco, CA
- entry level data engineer San Francisco, CA
- data engineer San Francisco, CA
- big data cloud engineer San Francisco, CA
- data platform engineer San Francisco, CA
- data engineer intern San Francisco, CA
- data engineering intern summer San Francisco, CA