Data Infrastructure Engineer — Scale Spark, Kafka & AI Data
OpenAI
A leading AI research organization located in San Francisco is seeking an experienced data infrastructure engineer to design and operate data infrastructure supporting extensive compute fleets. You will manage the lifecycle ownership and ensure high performance, scalability, and security of data access. Candidates should have over 4 years of relevant experience and be comfortable with fast-paced environments. The position offers a hybrid work model and relocation assistance. #J-18808-Ljbffr OpenAI
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Infrastructure Engineer — Scale Spark, Kafka & AI Data in San Francisco, CA vacancy
- APPIT Software Solutions is hiring a Senior Data Engineer (Apache Spark) in San Francisco, USA . Lead the design of large-scale distributed data processing systems using Apache... ...is a leading technology company building AI-powered enterprise products. Join 150+ professionals...SuggestedFlexible hours
- Perplexity AI is seeking a Senior/Staff Software Engineer to lead the architecture of their data platform. This role involves... ..., designing large-scale data pipelines, and... ...engineering and data infrastructure, with strong... ...tools like Airflow and Kafka as well as proficiency...Suggested
$250k - $380k
...Type Full time Department Scaling Compensation $250K - $3... ...training and inference infrastructure that powers frontier... ...We are looking for an engineer to design and implement... ...including for multimodal (MM) data that cannot fit in... ...OpenAI OpenAI is an AI research and deployment...SuggestedFull timeWork at officeLocal areaRelocation packageFlexible hours- ...Monaco is building an AI-native revenue... ...possible when all the data lives under one... ...exist in both scaling core systems and workflows... ...Data Platform Engineer to help build... ...context systems, and infrastructure that power our AI‑... ...ElasticSearch, Turbopuffer, Kafka, Spark, Databricks,...SuggestedWork at officeShift work
Slope
San Francisco, CA1 day ago - Garuda Ventures is hiring a Sr Data Engineer to manage petabyte-scale data pipelines that will power Factory OS. The role involves designing data architectures... ...of data engineering experience with frameworks like Spark and proficiency in Python and SQL. This position offers...Suggested
- ...Neon Redwood is a data services... ...on cutting-edge AI and data-driven... ...of passionate engineers and data experts... ...expand our data infrastructure and analytics capabilities... ...with large-scale data sets. Help... ..., Hadoop, Spark). Experience with... ...(e.g., Kafka, Pub/Sub). #J-...Full timeWork experience placement
$120k - $150k
The Opportunity As a Data Engineer, you are passionate and drive excellence... ...data gets delivered, and scale our data systems where... ...enhance our data platform and infrastructure Required to work on-site 3 days... ...Familiarity with Apache Kafka, or Kinesis Familiarity with...Full time3 days per week$104.33k - $156.49k
...organizations to use data and AI responsibly. Our platform... ...Senior Data Platform Engineer will be part of an... ...and architecture for scale. Your Mission You... ...applications, and provision of infrastructure. Perform... ...technologies (Hadoop, Hive, Spark, Kafka, Kinesis)....Work experience placementWork at officeLocal areaFlexible hours3 days per week1 day per week- ...identity platform company in San Francisco is looking for a Data Infrastructure Engineer to design, build, and maintain their data platform. The... ...proficiency in Python, and knowledge of technologies like Kafka and Snowflake. You will collaborate with various teams to...
- ...and supported the Regular Toilet is seeking a Software Engineer to build large-scale models that support our mission of creating useful general... ...while utilizing cloud technology to create reliable data infrastructure. The ideal candidate has 5+ years of software engineering...
- Judgment Labs builds infrastructure for Agent Behavior Monitoring... ...context retrieval loss in scaled production environments.... ...are looking for a Senior Data Infrastructure Engineer to build and scale the real... ...and batching systems (Kafka, Spark, Flink, or Ray) operating...
- ...is building the data and ML backbone that... .... As a Data Engineering Manager leading our... ...learning reliable at scale. You’ll guide our... ...is not a pure infrastructure or ML engineering... ...of the platform. AI-forward... ...technologies such as Kafka, Flink, Spark, or equivalent....Work at officeLocal areaRemote workWorldwideFlexible hours3 days per week
$160k - $230k
Open role Founding Data Infrastructure Engineer San Francisco (On-site) About Us Constellation is creating the AI-human translation layer that ensures... ...that can support TB‑scale data collected daily. You will... ...queuing tools (e.g., ZeroMQ, Kafka, RabbitMQ, Redis Pub/Sub) for...Work at officeLocal areaRelocation package- ...world's most dynamic AI companies, like... ...research, flexible infrastructure, and seamless... ...including BOND, IVP, Spark Capital, Greylock,... ...build the platform engineers turn to to ship AI... ...Network Engineer (Data Centers) at Baseten... ...how we build and scale data center networks...Flexible hours
$140k - $200k
...foundational member of our engineering team: a highly... ...evolution of our data platform. You will... ...and management infrastructure that powers... ...Development: Develop and scale robust, fault-... ...technologies like Kafka or Flink to power... ...technologies such as Spark, Flink, or Dask....Full timeCrustdata (YC F24)
San Francisco, CA2 days ago- ...building the world's most advanced AI infrastructure ecosystem. The Scaling Analytics team serves as the data backbone for this effort,... ...the Role We are seeking a Data Engineer to help build and scale the... ...processing frameworks such as Spark. Experience with transformation...
$230k - $280k
...build of a world-class data platform from the... ...Principal Data Platform Engineer to lead the buildout of... ...challenges and shaping the infrastructure that supports real-time analytics and AI/ML at scale, this role is for you.... ...-in-class tools like Kafka, Iceberg, ClickHouse,...Remote workHarnham
San Francisco, CA4 days ago- ...combination of proprietary infrastructure and software, we... ...at a global scale. Proudly founded... ...end‑to‑end. You use AI to work smarter... ...of our company’s data and AI strategy. We... ...infrastructure (Databricks, Spark, Kafka, etc.), the... ...architect (staff engineer), you will oversee...Work at officeWorldwide
Airwallex-
San Francisco, CA2 days ago - A leading data and AI company in San Francisco is seeking a Sr. Staff Software Engineer to join their executive leadership team. This role involves designing the Data... ...Intelligence Platform, developing scalable tools and infrastructure, and collaborating with tech leaders. The...
- A tech-driven AI company in San Francisco is looking for a Data Engineer to build and scale its internal data platform. The successful candidate will design data models,... ...and familiarity with tools like Apache Beam and Kafka. The role offers competitive compensation and...
- NimbleRx in San Francisco is seeking a Senior Data Engineer to own and optimize the data platform that underpins all business operations. In this role, you will ensure the platform’s reliability and compliance while accelerating workflows across analytics, product features...
- DoorDash is seeking a Data Platform Engineer based in the Bay Area to lead the vision and strategy for a rapidly growing analytics framework. You will scale the platform for increasing data workloads and work closely with stakeholders. Ideal candidates possess a degree...
- ...is seeking an experienced Data Engineer in San Francisco. This role... ...designing and maintaining large-scale data pipelines for their... ...engineering, expertise in Kafka, Spark, and ClickHouse, and possess... ...and high-performance data infrastructure. #J-18808-Ljbffr RZR Global...
$200k - $250k
Duckbill is seeking a Data Engineer to build and manage our Skyway platform, responsible for data infrastructure and billing processes. You will collaborate across teams to optimize... ...and ensure quality in financial data for AI-driven companies. We offer a competitive salary...$135k - $180k
...— in real time, at scale, across every screen. Our data exists with the consent... ...using Apache Spark (PySpark/Scala) Build... ..., and software engineers to deliver governed... ...of cloud-based data infrastructure Who You Are Bachelor... ...data pipelines (Kafka, Flink) $135,000 -...- ...Francisco, is seeking an experienced analytics engineer to drive key technical projects across... ...will lead the development of scalable data models and enhance data processing quality... ...candidate will have experience in large-scale data processing and be proficient in tools...Work at office
$130.6k - $192k
...Team DoorDash is a data‑driven... ...organization owns all the infrastructure necessary to run... ...technologies such as Spark, Flink, Kafka, Airflow, and... ...data workflows for engineers, analysts, and ML... ...business. Scale the analytical platform... ...Proficiency in using AI coding tools (e.g...Hourly payWork at officeLocal areaRelocationFlexible hours$196k - $269.5k
About This Role Our Engineering organization owns... ...Engineering, Data Science, Product Management... ...maintenance, and scaling to serve millions... ...with select infrastructure vendors. We’re Looking... ...Airflow Spark Python Kotlin Salary... ...latest enterprise AI tools, to help you...Work experience placementWork at officeRemote workMonday to FridayFlexible hours3 days per week$150k - $170k
...an experienced Senior Data Platform Engineer, with significant experience in building, scaling, and optimizing... ...Databricks, Dbt, and Spark Develop and maintain... ...and implement scalable infrastructure for Large Language Model... ...workflows Collaborate with AI/ML teams to build and...Full timeRemote workFlexible hours$182k - $228k
...collecting and analyzing large‑scale data sets. Your primary... ...to refine the data engineering roadmap and drive... ...data platform tools and infrastructure to boost team productivity... ...understanding of AI capabilities and constraints... ...optimize efficiency, spark creativity, and...Full timeWork at office3 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Infrastructure Engineer — Scale Spark, Kafka & AI Data. Be the first to apply!
Related searches
- remote data engineer San Francisco, CA
- data engineer intern San Francisco, CA
- entry level big data engineer San Francisco, CA
- big data devops engineer San Francisco, CA
- data engineer San Francisco, CA
- data engineer contract San Francisco, CA
- software data engineer San Francisco, CA
- data engineer graduate San Francisco, CA
- big data cloud engineer San Francisco, CA
- junior big data engineer San Francisco, CA