Databricks Architect: Lead Real-Time Data Pipelines
Inizio Partners Corp
Inizio Partners Corp is seeking a Senior Data Engineer to design and optimize ETL pipelines on cloud Databricks, with a focus on insurance data and real-time integration via Spark. You will lead data engineering projects, collaborate with global teams, and implement automated testing and CI/CD practices.
The role requires strong SQL/Python skills, experience with Databricks, Delta Lake, and Spark, plus familiarity with Airflow, Splunk, Kubernetes, and Azure DevOps.
#J-18808-LjbffrVacancy posted 11 hours ago
Similar jobs that could be interesting for youBased on the Databricks Architect: Lead Real-Time Data Pipelines in New York, NY vacancy
- ...Databricks Architect (Lead Data Platform Architect) New York, United States | Posted on 06/25/2026... ...Design scalable systems for: Batch & real-time data processing Structured & unstructured... ...BI tools (Power BI, Tableau) ML pipelines and AI platforms Collaborate with...DatabricksPipeline
iLink Digital
New York, NY11 hours ago - SMBC seeks a Databricks Engineer with AWS expertise to design, build, and maintain an enterprise data Lakehouse infrastructure in the US. The role focuses on high-performance data pipelines, real-time reporting, and analytics environments, integrating Databricks with core...DatabricksPipeline
$215.2k - $244.7k
...seeking a Senior Architect to join our thriving... ...developing Big Data strategy and roadmaps... ...advice for Databricks, Snowflake, and/or... ...flow and processing pipelines for analysis using... ...unlimited flexible time off and ten paid holidays... ..., delivering real-world solutions.The...DatabricksPipelineLocal areaImmediate startFlexible hours$201.68k - $322.69k
...believe that belonging leads to better outcomes and... .... Enjoy generous time-away and health benefits... ...can succeed as a Lead Data Architect at Capital Group.”As a... ...including but not limited to Databricks, Dremio, Snowflake,... ..., ensuring that data pipelines are AI-ready,...DatabricksPipelineFull timeTemporary workLocal areaFlexible hours- Senior GCP Data ArchitectLocation:... ...RemoteEmployment Type: Full-Time... ...with over 400 leading technology providers... ...Senior GCP Data Architect to lead the... ...streaming, and real-time data processing... ...(CDC) data pipelines.* Strong communication... ...with Databricks, Snowflake, Terraform...DatabricksPipelineFull timeContract workLocal areaFlexible hours
- Sportradar in New York, NY, USA is seeking a Senior Data Engineer to design, build, and optimize large-scale data pipelines for analytics and data science products. The... ...on ingestion pipelines in cloud services for real-time and batch data, data modeling and ETL development...Pipeline
- A cybersecurity startup in New York seeks a Data Engineer to develop secure data infrastructure for real-time analytics. The role involves designing ETL/ELT pipelines, optimizing cloud storage solutions, and collaborating with engineering teams. Ideal candidates have a...PipelineRemote job
- Asylon in Norristown, PA is seeking an experienced Data Engineer to architect, build, and maintain real-time data streams from our fleet of robots. You will design pipelines for video, telemetry, and sensor data with low latency for monitoring and ML training. You will...Pipeline
- ...in New York City, NY, is seeking a Mid-Senior Data Engineer with strong expertise in AWS-based data engineering, real-time streaming, and data quality frameworks. The... ...designing, building, and optimizing scalable data pipelines that support enterprise analytics. You will...Pipeline
- Capital One in New York, NY is hiring a Lead Data Engineer to join Risk Tech. You’ll design and deploy data-driven, AI-enhanced... ...data stores in cloud environments, building scalable, real-time data pipelines and ML-enabled tools to empower financial decisions at scale...Pipeline
- ...graphics developer to design, implement, and maintain real-time broadcast graphics for live productions. You will work... ...include building reusable workflows, debugging graphics pipelines, and supporting AR/VR or data-visualization graphics across live broadcasts. A strong...PipelineRemote job
- ...metro area is seeking a Principal Machine Learning Engineer to own the ML infrastructure behind real-time compliance enforcement systems. You will build training pipelines, evaluation workflows, and production-serving capabilities to train, measure, and deploy models with...Pipeline
On behalf of Next Deavor
New York, NY11 hours ago - ...SMBC seeks a Senior Databricks Engineer with deep AWS expertise to build and maintain enterprise data Lakehouse infrastructure. You will design high-performance data pipelines, implement regulatory and real-time reporting, and ensure seamless Databricks–AWS integration...DatabricksPipeline
SMBC
New York, NY3 days ago $150k - $190k
How you will help As a Lead Analyst, Real World Data (RWD) Insights you will serve as a strategic partner... ...data needs by monitoring their pipeline, research activity, and emerging use... ...to deliver data licensing projects on time and with accuracy. Mentor junior team...PipelineContract workWork at officeLocal areaRemote workWork from homeFlexible hours3 days per week$147.29k - $199.28k
...Type of Requisition Pipeline Clearance Level... ...Intelligence (BI), Data Requirements, Data... ...No Job Description Lead Business Data... ...data engineers and architects to validate data models... ...artifacts and timely deliverables. QUALIFICATIONS... ...(e.g., Snowflake, Databricks, or AWS S3/...DatabricksPipelineTemporary workWork at officeImmediate startRemote workWorldwideFlexible hoursRemote Jobs
New York, NY6 hours ago$215.2k - $244.7k
...seeking a Senior Architect to join our thriving... ...developing Big Data strategy and roadmaps... ...with emphasis on Databricks, while gaining exposure... ...and processing pipelines for analysis using... ...flexible time off and ten paid holidays... ...clients, delivering real-world solutions....DatabricksPipelineLocal areaImmediate startFlexible hours2 days per week1 day per week- ...Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.) Experienced in implementing... ...Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs Lead Data Engineering Projects to manage...DatabricksPipeline
Inizio Partners Corp
New York, NY13 hours ago $190k
...performance through data, technology and... ...most complex and time-sensitive challenge... ...transforming strategies into leading-edge tech... ...ll DoLead AI Tech Architects - AI Platforms at... ...models (LLMs), data pipelines, and enterprise... ...(batch training, real-time inference) with...PipelineWork at officeLocal area- Olik Global is seeking an experienced Data Engineer for projects in New Jersey, Irving, and Tampa. You'll be responsible for designing and optimizing data pipelines with a focus on MEM SQL (SingleStore) for real-time processing. The ideal candidate will require strong...Pipeline
- ...seeking an AI Platform Architect, Strategy &... ...management contexts.Lead Platform Strategy... ...the platform's core data, application, and... ...operate the platform's real-time performance... ...SQL, Azure Synapse/Databricks for analytics and... ...mining, and retrieval pipeline...DatabricksPipelineFull timeLocal areaRemote workWork visa
- ...iLink Digital is seeking a Databricks Architect (Lead Data Platform Architect) in New York, NY to design, build, and scale enterprise-grade Lakehouse data platforms. You will lead architecture strategy, platform standardization, and modernization initiatives leveraging...Databricks
iLink Digital
New York, NY11 hours ago - A leading performance marketing engine in New York is seeking a Senior Data Engineer to contribute to the development of Micro Batch/Real Time streaming pipelines. This role involves working closely with the engineering team and collaborating with key stakeholders to drive...Pipeline
- ...Tabs, the leading AI-native revenue platform, seeks a Senior Software Engineer for the Data Ingest team in our Integrations org. You will own parts of the usage-to-billing pipeline, ingest real-time data from CRM systems, and help scale data flows to support AI-era billing...Pipeline
Tabs
New York, NY11 hours ago - ...Games is seeking a Senior Software Engineer in Data Engineering to enhance our analytics platform and... ...throughput data processing services, developing real-time applications, and ensuring the scalability of data pipelines. This is a full-time role based in New York City...PipelineFull time
Rockstargames
New York, NY11 hours ago $140k - $160k
Thenewyorktimes seeks a Senior Data Engineer to join the Customer-Facing Data Products... ...in data engineering with a focus on real-time solutions and event-driven architectures... ...Responsibilities include developing data pipelines, ensuring data quality, and collaborating...Pipeline- ...driven by delivering real results for our... ...currently seeking an AI Architect to join our Audit Technology... ...services, RAG pipelines, and intuitive human... ...integrating with data platforms like Databricks or Microsoft FabricIn... ...KPMG provides Personal Time Off per fiscal year....DatabricksPipelineLocal area
$156.28k - $223.25k
We are looking for a Data/Backend Engineer to join our team. We are at the forefront of deriving foundational knowledge through... ...every day. What You'll Do Build large-scale batch and real-time data pipelines with data processing frameworks like Scio, Google Cloud Platform...PipelineFlexible hours- ...is seeking a Senior Software Engineer for the Data Ingest team, part of Integrations, to scale an usage-to-billing pipeline and CRM integrations. You will own... ...grow its platform to support AI-era usage-based billing and real-time reconciliation. #J-18808-LjbffrPipeline
tabs platform inc
New York, NY11 hours ago - ...Sports Content Product to own the full sports data and monetization stack, including real-time ingestion, analytics, AI-driven highlight generation... ..., networks, leagues, and sponsors. You will lead low-latency data pipelines, predictive analytics, and sponsorship...Pipeline
- The New York Times is seeking a Senior Data Engineer in New York City to contribute to the Customer-Facing Data Products team. This role involves developing real-time data pipelines and APIs that serve customer needs. The ideal candidate has over 5 years of experience...Pipeline
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Databricks Architect: Lead Real-Time Data Pipelines. Be the first to apply!
Related searches