Data Engineer: Real-Time Pipelines on AWS & Databricks
Bitbybit Solutions
Bitbybit Solutions is seeking a skilled data engineer to build and manage robust data pipelines using AWS and Databricks. The ideal candidate will be proficient in Scala, Spark, and Kafka, and experienced in designing scalable systems with observability and logging. You will collaborate with Product, Regulatory, and Legal teams to ensure compliance while monitoring system robustness and developing pipelines for both structured and unstructured data. #J-18808-Ljbffr
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Engineer: Real-Time Pipelines on AWS & Databricks in New York, NY vacancy
- ...the United States is seeking a Senior Data Engineer to build and maintain a reliable... ...role involves developing historical and real-time data pipelines, mentoring junior staff, and ensuring... ...based environments using Terraform and AWS/GCP. This position offers significant...Amazon Web ServicePipeline
Ethena Labs
New York, NY1 day ago - ...Data Foundations Engineer The Data Foundations Engineer designs... ...high-performance data pipelines and enabling... ...scalable batch and near-real-time data pipelines. Develop... ...~ Experience with AWS, Azure, or GCP. ~... ...with Snowflake, Databricks, Trino, OLAP/NRT systems...DatabricksAmazon Web ServicePipeline
Apolis
New York, NY3 days ago - ...Big Data Engineer Mandatory skills: Apache Spark/ Hive... ...to build scalable data pipelines, develop ML models, and... ...Apache Spark, Talend, AWS Glue, Google Dataflow,... ..., Apache Hive. Stream real-time data using Kafka,... ...with AWS, Azure, GCP, Databricks. Store and manage data...DatabricksAmazon Web ServicePipeline
Omni Inclusive
Jersey City, NJ3 days ago - ...and related products. The Data Engineering team handles the full data... ...building data architectures and pipelines to empower evidence-based... ...for both batch and real-time streaming data. Architect and... ...Cloud DataWorks, MaxCompute, AWS DataBricks, Hadoop, Hive, Spark and HBase...DatabricksAmazon Web ServicePipeline
Framework Ventures
New York, NY1 day ago - ...to design and build data platforms that power... ...complex distributed pipelines, and real business impact. We’re... ...looking for a Senior Data Engineer to join our remote... ...data solutions using Databricks on AWS . This is a hands‑on... ...Central, or Eastern U.S. time zones to ensure...DatabricksAmazon Web ServicePipelineImmediate startRemote workFlexible hours
Lumenalta
New York, NY6 hours ago - # AWS Data Engineer**Remote**Full time785d019a-e687-42e7-8685-61bbf5e351... ...and maintain scalable data pipelines on AWS. You will work closely... ...native technologies and Databricks for data transformations and... ...Data pipelines (batch & near-real-time)· Integration with external...DatabricksAmazon Web ServicePipelineRemote work
Qode
New York, NY1 day ago - ...Taskus India is looking for a Senior Data Engineer to support their data strategy. The role involves building resilient pipelines, optimizing SQL queries, and ensuring data reliability... ...experience with modern data stacks like Databricks or Snowflake is essential. This role...DatabricksPipeline
Taskus India
New York, NY1 day ago - Senior Data Engineer - Media Org (Hybrid NYC or LA) I’m working on a Senior... ...will involve designing batch and real-time data pipelines, working heavily within a Databricks environment, helping shape... ...Airflow or similar Familiarity with AWS environments, data lakes, and...DatabricksAmazon Web ServicePipelineFull time
- ...collection of executives, engineers, data scientists, and... ...and maintain the data pipelines that power our deep learning... ...layers — from real-time feeds to analytics-ready... ...transformation using Databricks, Airflow, or Dagster.... ...infrastructure (AWS, GCP, or Azure) and containerization...DatabricksAmazon Web ServicePipelineRemote workFlexible hours
SumerSports LLC
New York, NY1 day ago - ...OCI is seeking experienced Data Engineers to design, develop, and... ...optimize large-scale data pipelines (Azure, AWS, GCP) Develop data ingestion... ...Competencies ETL data engineering (Databricks, SQL Server, Snowflake,... ...driven data solutions and real-time data processing. Expertise...DatabricksAmazon Web ServicePipeline
Object Computing
New York, NY1 day ago - ...A leading technology firm is seeking a Data Engineer to support major cloud migration initiatives. The role involves modernizing on-prem Data Lakes to a cloud-based architecture on AWS. Candidates should possess strong data engineering skills, experience in cloud migration...Amazon Web ServicePipeline
Insight Global Technologies
New York, NY1 day ago - ...As a Lead Data Engineer at Rearc, you'll play a pivotal... ...high-quality data pipelines and analytical... ...platforms such as AWS Redshift, Azure... ...frameworks, including Databricks, Spark , and... ...desire to make a real impact—you’ll feel... ...together. Generous time away and flexible...DatabricksAmazon Web ServicePipelineFlexible hours
Rearc
New York, NY1 day ago - ...Data Engineer Hiring Location: Gurugram, India (Work... ...Type: Full-time Company Profile NPS... ...experience working on Databricks and cloud platforms... ...such as Azure and AWS. A solid... ...practices and CI/CD pipelines will be advantageous... ...processing, analytics, and real‑time data...DatabricksAmazon Web ServicePipelineFull timeWork from home
NPS Prism
New York, NY1 day ago $122.43k - $183.64k
...position. The Senior Data Infrastructure Engineer designs, builds, and... ...data infrastructure on AWS using technologies... ...build and operate CI/CD pipelines for data... ...QUALIFICATIONS Experience with real-time or streaming data platforms (Kafka, Databricks). Exposure to data governance...DatabricksAmazon Web ServicePipelineFull timeVisa sponsorshipLyric
New York, NY1 day ago$241k - $338k
...AI research and engineering, and state-of-... ...is part of the Data Engineering team... ...coordinates, time series, molecular... ...when a pipeline output is technically... ...science, with real autonomy and the... ...frameworks like Databricks, Spark, or Ray... ...infrastructure (AWS preferred) and...DatabricksAmazon Web ServicePipelineWork at officeWorldwideRelocation package3 days per weekBiohub
New York, NY4 days ago$128.14k - $252.19k
...Manager to join our Data Service team at... ...with our Engineering organization to... ...data models and pipelines. You will partner... ...and aligned to real business needs.... ...(Redshift, Databricks, Snowflake, or... ...cloud environment (AWS/GCP/Azure).... ...Gemini) to solve time‑consuming critical...DatabricksAmazon Web ServicePipelineLocal areaMercury General
New York, NY1 day ago- ...to grow. Now it's time to make a real impact. We’re a growing... ...help bring their data, AI, cloud and... ...the shiny tech : Databricks, Snowflake, Azure, GCP, AWS. You’ll use it,... ...talk. As a Data Engineer (Principal... ...Design and build pipelines to ingest, transform...DatabricksAmazon Web ServicePipelineWeekly payWork from home
Vivanti Consulting
New York, NY3 days ago - ...Lead Data Engineer Locations: Richmond - VA / McLean -... ...design and implement data pipelines and workflows.... ...smooth integration and real-time processing of data using... ...cloud technologies like AWS. Develop and maintain... ...AWS Glue, Spark, and Databricks. Optimize database...DatabricksAmazon Web ServicePipelineLong term contractWork at office
InterSources
New York, NY5 days ago - ...Senior Consultant, Data Engineer Work... ...DC Type: Full-Time Sector: Data,... ...platforms like Snowflake, Databricks, and Google Cloud.... ...GCP; Azure, GCP, AWS What We're... ...Exposure to ML pipelines, MLOps, or AI-adjacent... ...progress reviews with real feedback, not...DatabricksAmazon Web ServicePipelineFull timeWork at office
Biz First
New York, NY2 days ago - ...proprietary AI Studio and AI Engines, the company helps drive... ...with AI. Senior Data Engineer Are you an experienced... ...optimizing end-to-end real-time and batch pipelines, and developing cloud-... ...technologies such as AWS, GCP, Azure, Databricks, and Snowflake? We are...DatabricksAmazon Web ServicePipelineH1bLocal area
Fusemachines
New York, NY5 days ago - Overview Job Title: Senior Data Engineer (Banking) Location:... ...engineer, Snowflake, Databricks, Python, PySpark, SQL,... ...robust ETL/ELT pipelines using Snowflake, Databricks... ...warehouses, data marts, and real-time data solutions for... ...with cloud platforms (AWS/Azure) and CI/CD tools...DatabricksAmazon Web ServicePipelineContract work
$100k - $143k
...team of analytics engineers focused on delivering... ..., and reliable data to power various business... ...maintain dbt data pipelines to deliver clean,... .... • Implement real-time data ingestion and... ...using PySpark and AWS Glue. • Apply... ...knowledge of Databricks, including SQL, notebooks...DatabricksAmazon Web ServicePipelineLocal area3 days per weekNew York Life
New York, NY2 days ago- ...Talascend is currently seeking an AWS Data Engineer for a remote, contract... ...large-scale data pipelines and analytics solutions on... ...streaming data pipelines for real-time analytics. Build solutions... ...frameworks. Working knowledge of Databricks on AWS. Working knowledge of...DatabricksAmazon Web ServicePipelineContract workRemote work
Talascend
New York, NY1 day ago - ...solutions that deliver real results. Our... ...seeking an experienced Data Engineer to support the... ...and scaling of data pipelines and infrastructure... ...Microsoft Fabric, Databricks, and/or Azure... ...scalable batch and real-time data pipelines across Azure and AWS. ~ Build and...DatabricksAmazon Web ServicePipelineWorldwide
IBM
New York, NY4 days ago - .... About the Team OKX data team is responsible for... ...science. We are data engineers, data analysts and... ...Kafka, etc. Expert in real-time big data processing using... ...and real-time data pipelines to ensure data... ...DataWorks/MaxCompute, AWS Spark, Databricks, and StarRocks/Doris....DatabricksAmazon Web ServicePipelineWork experience placement
Framework Ventures
New York, NY1 day ago - ...located in New York is looking for an experienced Senior Data Engineer to design and develop ETL/ELT pipelines for banking applications. Candidates should have... ...the banking sector, with expertise in Snowflake, Databricks, and Python. The role entails optimizing data...DatabricksPipeline
$140k - $240k
...Overview The Data Engineer on the Nebula team plays... ...production-ready data pipelines and platforms.... ..., including batch and real-time pipelines, operational... ...cloud platforms such as AWS, GCP, or Azure... ..., BigQuery, Redshift, Databricks, or equivalent technologies...DatabricksAmazon Web ServicePipelineLocal areaRemote workFlexible hoursBayview Asset Management
New York, NY1 day ago- ...Mindfire is seeking a skilled Data Engineer to architect and build scalable data platforms on... ...involves developing and optimising ETL/ELT pipelines and containerised applications, as well... ...with strong skills in Python, SQL, and AWS services. Join us to build high-performance...Amazon Web ServicePipeline
MINDFIRE INTERNET SOLUTIONS
New York, NY1 day ago $125k - $163.8k
...We are looking for a Data Engineer to join our growing data... ...delivery of scalable data pipelines and platform components built on Databricks and Airflow - from... ...production batch and near-real-time pipelines: build and maintain... ...in cloud environments (AWS, Azure, or GCP),...DatabricksAmazon Web ServicePipelineTemporary workLocal areaWorldwideFanDuel
New York, NY4 days ago$150k - $170k
...Senior Data Engineer At Magnite, we cultivate an environment... ...and streaming data pipelines. The Core Data... ...Spark jobs, Java-based real-time event processing, and... ...currently based both on Databricks and on-prem Spark infrastructure... ...for job orchestration AWS-based cloud...DatabricksAmazon Web ServicePipelineHourly payWork experience placementSummer workSeasonal workLocal areaShift workMagnite
New York, NY5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer: Real-Time Pipelines on AWS & Databricks. Be the first to apply!
Related searches
- bi data engineer New York, NY
- staff data engineer New York, NY
- data visualization developer New York, NY
- data science developer New York, NY
- senior data center engineer New York, NY
- sr information security engineer New York, NY
- IT data engineer New York, NY
- junior big data engineer New York, NY
- entry level big data engineer New York, NY
- data engineer contract New York, NY