Data Infrastructure Engineer — Scale Spark, Kafka & AI Data
OpenAI
A leading AI research organization located in San Francisco is seeking an experienced data infrastructure engineer to design and operate data infrastructure supporting extensive compute fleets. You will manage the lifecycle ownership and ensure high performance, scalability, and security of data access. Candidates should have over 4 years of relevant experience and be comfortable with fast-paced environments. The position offers a hybrid work model and relocation assistance. #J-18808-Ljbffr OpenAI
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Infrastructure Engineer — Scale Spark, Kafka & AI Data in San Francisco, CA vacancy
- APPIT Software Solutions is hiring a Senior Data Engineer (Apache Spark) in San Francisco, USA . Lead the design of large-scale distributed data processing systems using Apache... ...is a leading technology company building AI-powered enterprise products. Join 150+ professionals...SuggestedFlexible hours
$250k - $380k
...Type Full time Department Scaling Compensation $250K - $3... ...training and inference infrastructure that powers frontier... ...We are looking for an engineer to design and implement... ...including for multimodal (MM) data that cannot fit in... ...OpenAI OpenAI is an AI research and deployment...SuggestedFull timeWork at officeLocal areaRelocation packageFlexible hours- ...Monaco is building an AI-native revenue... ...possible when all the data lives under one... ...exist in both scaling core systems and workflows... ...Data Platform Engineer to help build... ...context systems, and infrastructure that power our AI-... ...ElasticSearch or Turbopuffer, Kafka, Spark, Databricks or...SuggestedWork at officeShift work
Monaco
San Francisco, CA1 day ago $190k - $240k
...Staff Data Platform Engineer San Francisco, California... ...intelligent application of AI and machine... ...continues to scale rapidly as TV advertising... ...is a systems and infrastructure position first.... ...data processing (Spark, PySpark, or... ...systems (Kafka, Flink, or similar...SuggestedWork at office2 days per weekTatari
San Francisco, CA4 days ago- ...Role As a Staff Data Engineer, you will be a technical... ...of the data infrastructure that powers real-time... ..., analytics, and AI across Hinge... ...services — across Kafka event streams and... ...data platforms at scale. 5+ years of strong... ..., PySpark, Flink, Spark) and data warehouse...SuggestedWork at officeLocal areaImmediate startRemote workWorldwide3 days per week
Hinge Health
San Francisco, CA2 days ago $120k - $150k
The Opportunity As a Data Engineer, you are passionate and drive excellence... ...data gets delivered, and scale our data systems where... ...enhance our data platform and infrastructure Required to work on-site 3 days... ...Familiarity with Apache Kafka, or Kinesis Familiarity with...Full time3 days per week$104.33k - $156.49k
...organizations to use data and AI responsibly. Our platform... ...Senior Data Platform Engineer will be part of an... ...and architecture for scale. Your Mission You... ...applications, and provision of infrastructure. Perform... ...technologies (Hadoop, Hive, Spark, Kafka, Kinesis)....Work experience placementWork at officeLocal areaFlexible hours3 days per week1 day per week- ...Judgment Labs builds infrastructure for Agent Behavior Monitoring... ...retrieval loss in scaled production environments.... ...are looking for a Senior Data Infrastructure Engineer to build and scale the real... ...streaming and batching systems (Kafka, Spark, Flink, or Ray) operating...
Judgment Labs
San Francisco, CA17 hours ago - ...identity platform company in San Francisco is looking for a Data Infrastructure Engineer to design, build, and maintain their data platform. The... ...proficiency in Python, and knowledge of technologies like Kafka and Snowflake. You will collaborate with various teams to...
- ...and supported the Regular Toilet is seeking a Software Engineer to build large-scale models that support our mission of creating useful general... ...while utilizing cloud technology to create reliable data infrastructure. The ideal candidate has 5+ years of software engineering...
- ...world's most dynamic AI companies, like... ...research, flexible infrastructure, and seamless... ...including BOND, IVP, Spark Capital, Greylock,... ...build the platform engineers turn to to ship AI... ...Network Engineer (Data Centers) at Baseten... ...how we build and scale data center networks...Flexible hours
- ...is building the data and ML backbone that... .... As a Data Engineering Manager leading our... ...learning reliable at scale. You’ll guide our... ...is not a pure infrastructure or ML engineering... ...of the platform. AI-forward... ...technologies such as Kafka, Flink, Spark, or equivalent....Full timeWork at officeLocal areaRemote workWorldwideFlexible hours3 days per week
- ...expanding, the data challenges we're... ...the platform is scaling to match. As a Distinguished Engineer on the Data Engineering... ...of the hardest infrastructure problems at... ...systems such as Kafka, Kinesis, Flink, or Spark Streaming at real... ...the best AI empowers users with...Day shift
CloudZero
San Francisco, CA3 days ago $135k - $180k
...Data Engineer Opportunity Samba is a media intelligence... ...about — in real time, at scale, across every screen.... ...using Apache Spark (PySpark/Scala) Build... ...performance of cloud-based data infrastructure Who You Are... ...streaming data pipelines (Kafka, Flink) $135,000...Samba TV
San Francisco, CA1 day ago- ...Software Engineer We are looking for a foundational... ...evolution of our data platform. You will... ...and management infrastructure that powers... ...Development: Develop and scale robust, fault-... ...technologies like Kafka or Flink to power... ...technologies such as Spark, Flink, or Dask....
Crustdata (YC F24)
San Francisco, CA4 days ago $220k - $250k
...Senior / Staff Software Engineer, Data Platform Title of Role: Senior... ...Round — Software Development, AI Office Type: Hybrid Salary... ...technologies such as Python, Kafka, Spark, and PostgreSQL to deliver... ...distributed systems that handle large-scale data operations....Work at officeRecruiting from Scratch
San Francisco, CA2 days ago- A tech-driven AI company in San Francisco is looking for a Data Engineer to build and scale its internal data platform. The successful candidate will design data models,... ...and familiarity with tools like Apache Beam and Kafka. The role offers competitive compensation and...
- ...is seeking an experienced Data Engineer in San Francisco. This role... ...designing and maintaining large-scale data pipelines for their... ...engineering, expertise in Kafka, Spark, and ClickHouse, and possess... ...and high-performance data infrastructure. #J-18808-Ljbffr RZR Global...
- A leading data and AI company in San Francisco is seeking a Sr. Staff Software Engineer to join their executive leadership team. This role involves designing the Data... ...Intelligence Platform, developing scalable tools and infrastructure, and collaborating with tech leaders. The...
- A rapidly growing data company in San Francisco is seeking a Senior Engineer specializing in data infrastructure to drive the technical direction of their data platform. In this role, you'll design robust systems for data ingestion and transformation while partnering closely...
- Veeva Systems, Inc. is looking for a Data Engineer to lead the development of scalable data processing systems. You will architect and build resilient data platforms using Python and Spark on AWS, design ETL workflows, and implement the Medallion Architecture for data governance...Flexible hours
$171.12k - $213.9k
...Artificial Intelligence (AI) to help make our... ...next Staff Software Engineer on our Data & Analytics Platform... ...complex data projects at scale.... ...understanding of data infrastructure and distributed systems... ...technologies such as Hadoop, Spark, Kafka, and other distributed...Local areaRemote workWorldwideTwilio
San Francisco, CA7 days ago$268k - $368.5k
...the power of tech, data, and machine... ...this role Our Engineering organization owns... ...about underlying infrastructure. We enable product... ...maintenance, and scaling to serve millions... ...Snowflake, Airflow, Spark experience especially... ...Artificial Intelligence (AI) to screen and...Work experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per weekFaire Inc
San Francisco, CA4 days ago- Cursor is seeking an Analytics Platform Engineer to take ownership of data foundations and work on optimizing their data lakehouse infrastructure. In this role, you will collaborate... ...including expertise in managing large-scale data ingestion. This is an opportunity...
- ...combination of proprietary infrastructure and software, we... ...at a global scale. Proudly... ...end-to-end. You use AI to work smarter and... ...of our company's data and AI strategy. We... ...infrastructure (Databricks, Spark, Kafka, etc.), the... ...architect (staff engineer), you will oversee...Work at officeWorldwide
Airwallex
San Francisco, CA2 days ago $194k - $267k
...Identity, from AI to Human Identity... ..., neutral infrastructure that enables organizations... ...Team The Data Platform team... ...for Okta to scale Telemetry of... ...from our engineers and reward them... ...such as Kinesis, Kafka ~ Data processing... ...like Flink, Spark, Beam ~...Permanent employmentWork at officeLocal areaWorldwideFlexible hoursOkta, Inc.
San Francisco, CA2 days ago$197.3k - $313.7k
...Category Software Engineering Job Details... ...is the #1 AI CRM, where humans... ...the Enterprise Data Platform is the... ...designing and scaling our advanced... ...ownership. Infrastructure as Code (IaC) Evangelism... ...Snowflake and Spark. AI... ...velocity pipelines (Kafka/Airflow) that...Salesforce
San Francisco, CA3 days ago- ...experiences with AI. We are... ...join a full-stack data team building the... ...handle data at scale with low latency... ...-time eventing infrastructure, streaming and... ...areas: Platform Engineering You... ...technologies like Spark, Flink, Trino/... ...(e.g., Kafka, Kinesis) and real...Full timeFlexible hours
Sierra
San Francisco, CA3 days ago $199k
...Engineering Manager, Data Platform Chime was created with the... ...experimentation, and AI-driven product experiences... ...support both high-scale batch workloads and... ...just about running infrastructure - it's about setting... ..., such as Spark, Flink, Kafka, Airflow, Kubernetes...Full timeWork at officeLocal areaRemote workNight shiftSoftbank Investment Advisers
San Francisco, CA1 day ago- ...combination of proprietary infrastructure and software, we... ...at a global scale. Proudly... ...end-to-end. You use AI to work smarter and... ...of our company's data and AI strategy. We... ...infrastructure (Databricks, Spark, Kafka, etc.), the... ...include scaling the engineering teams, defining...Worldwide
Airwallex
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Infrastructure Engineer — Scale Spark, Kafka & AI Data. Be the first to apply!
Related searches
- staff data engineer San Francisco, CA
- data visualization developer San Francisco, CA
- data science developer San Francisco, CA
- senior data center engineer San Francisco, CA
- sr information security engineer San Francisco, CA
- junior big data engineer San Francisco, CA
- entry level big data engineer San Francisco, CA
- data engineer contract San Francisco, CA
- aws data engineer San Francisco, CA
- data engineer manager San Francisco, CA