Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer (Databricks)

Rearc

As a Lead Data Engineer at Rearc, you'll play a pivotal role in establishing and maintaining technical excellence within our data engineering team. Your deep expertise in data architecture, ETL processes, and data modeling will be instrumental in optimizing data workflows for efficiency, scalability, and reliability. You’ll work closely with cross‑functional teams to design and implement robust data solutions that align with business objectives and industry best practices. Building strong partnerships with technical teams and stakeholders will be essential as you drive data‑driven initiatives and ensure their successful delivery. What You Bring 10+ years of experience in data engineering, data architecture, or related technical fields, with a proven ability to design, build, and optimize large‑scale data ecosystems. Leading complex data engineering initiatives , architecting end‑to‑end data solutions that are scalable, reliable, and aligned with business strategy. Deep hands‑on expertise in ETL/ELT design, data warehousing, and data modeling, enabling the creation of efficient, high‑quality data pipelines and analytical foundations. Extensive experience with data integration frameworks and best practices , ensuring seamless, performant data movement across diverse systems and platforms. Advanced knowledge of cloud‑based data services and architectures , including platforms such as AWS Redshift, Azure Synapse Analytics, Google BigQuery, or equivalent technologies. Strategic and analytical thinking , allowing you to tackle complex data challenges, drive architectural decisions, and influence long‑term data strategy. Proficiency with modern data engineering frameworks, including Databricks, Spark and lakehouse technologies like Delta Lake for managing and processing large‐scale datasets. Exceptional communication and interpersonal skills , enabling effective collaboration with engineers, product teams, executives, and customers—while influencing technical direction and decision‑making. What You’ll Do As a Lead Data Engineer, you will help shape, deliver, and elevate Rearc’s data engineering practice. You’ll lead by example—combining strong technical execution with mentorship, strategic thinking, and a culture‑first approach. Engage deeply with stakeholders to understand data needs, business challenges, and technical constraints, translating them into scalable, high‑quality data solutions. Implement with a DataOps mindset , applying modern engineering practices and leveraging tools such as Apache Airflow, Databricks/Spark, Kafka, or similar technologies to build reliable, automated, and efficient data pipelines and architectures. Lead and execute complex projects , providing technical direction, setting engineering standards, and ensuring alignment with customer goals and Rearc’s principles. Mentor and develop data engineers , offering guidance, code reviews, and hands‑on support to help them grow technically and professionally. Promote knowledge sharing and thought leadership by contributing to internal and external content—writing technical blogs, sharing best practices, and fostering a culture of continuous learning and innovation. Benefits Generous time away and flexible PTO Maternity and paternity leave Access to educational resources with reimbursement for continued learning #J-18808-Ljbffr Rearc

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer (Databricks) in New York, NY vacancy
  •  ...Lead Data Engineer - Palantir & PySpark (Lead Data engineer role where experience should be on both hands on and leading the team.)...  ...Azure, or GCP) and related data services (e.g., S3, Snowflake, Databricks). Should be familiar with CI/CD pipelines, version... 
    Databricks
    Remote work

    VBeyond

    Jersey City, NJ
    3 days ago
  • A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate...  ...collaborate with AI/ML teams. The position requires strong skills in Databricks, Python, and SQL, along with relevant experience in cloud... 
    Databricks
    Remote job

    INSPYR Solutions

    New York, NY
    2 days ago
  •  ...within the newly consolidated Data and Analytics (D&A)...  ...infrastructure, data governance, data engineering, data modeling, data analysis...  ...data science, and AI. The Lead Data Engineer is crucial to D...  ...utilizing Big Data and Azure/Databricks for Eligibility Program data... 
    Databricks

    MetLife

    New York, NY
    22 hours ago
  • $156.8k - $235.2k

    Paramount Pictures is seeking a Senior Lead Data Engineer in New York to build and scale data foundations for personalization systems powering...  ...systems, and deep experience with tools like Spark, Kafka, Databricks, and AWS. The position offers a salary range of $156,800 - $... 
    Databricks

    Paramount Pictures

    New York, NY
    22 hours ago
  • $220k - $300k

     ...Overview The Lead Data Engineer on the Nebula team plays a significant technical leadership role in shaping and scaling the data foundation...  ...lakehouse platforms such as Snowflake, BigQuery, Redshift, Databricks, or equivalent technologies ~ Experience with... 
    Databricks
    Local area
    Remote work
    Flexible hours

    Bayview Asset Management

    New York, NY
    2 days ago
  • $220k - $230k

     ...leader in digital assets and data center infrastructure, delivering...  ...'re a senior, hands-on data engineer with 8+ years of experience designing...  ...scale - and you're ready to lead. You set the technical...  ...the modern data stack - AWS, Databricks/Spark, dbt, and Airflow. You care... 
    Databricks
    Local area
    Flexible hours

    Galaxy USA

    New York, NY
    4 days ago
  • $73.01k - $170.64k

     ...Databricks Lead Data Engineer As a Databricks Lead Data Engineer, you are expected to lead the development team and have strong development and programming skills in Python, SQL, Pyspark, Databricks, Notebook Repositories, Medallion Architecture, CI/CD, and ETL platforms... 
    Databricks
    Work at office
    Local area
    Flexible hours

    Perficient

    New York, NY
    1 day ago
  • Maes Solutions is actively recruiting Lead data engineer for full-time roles with direct clients across multiple US locations, emphasizing...  ...optimization Experience designing and operating data pipelines on Databricks using orchestration tools such as Apache Airflow... 
    Databricks
    Full time
    Relocation

    MAES Solutions

    New York, NY
    1 day ago
  • $220k

    Specialist Recruiter | Databricks Data Engineer Recruitment | Connecting Top Talent with Leading US Opportunities Job Title: Lead Data Engineer Location: Remote Employment Type: Full-time Compensation: Up to $220k + benefits About the Role We’re hiring experienced... 
    Databricks
    Full time
    Remote work
    Flexible hours

    KDR Talent Solutions USA

    New York, NY
    2 days ago
  • EPAM Systems, Inc. is seeking a Lead Data Software Engineer (Python+AWS) to drive the design and development of scalable data solutions. In this...  ...proficiency in Python and SQL, and expertise in AWS and Databricks. You will work in a dynamic environment while... 
    Databricks

    EPAM Systems, Inc.

    New York, NY
    2 days ago
  • The position will lead technical and business analysis that is critical in discovery process requirement for additional data loads - this would include new and existing datasets supporting...  ...AWS (Redshift, PostgreSQL, S3), Azure Databricks, Oracle, SQLSERVER, etc.) and ETL... 
    Databricks
    Work experience placement
    Interim role

    Element Technologies

    New York, NY
    1 day ago
  •  ...democratize the benefits of data and AI . Since 2019, we have...  ...personalization and decisioning for leading brands and agencies while...  ...culture from the ground up. Engineers here own major surface areas...  ...Partnerships with Microsoft and Databricks Fully remote or hybrid from... 
    Databricks
    Remote work
    Flexible hours

    Yobi

    New York, NY
    2 days ago
  • $70 - $85 per hour

    Lead Data Engineer Location: Remote Duration: 6+ months Compensation: $70.00 - 85.00/hr Work Requirements: US Citizen, GC Holders or Authorized...  ...and maintain scalable data pipelines using Apache Spark on Databricks. Build and manage Delta Lake architectures for efficient... 
    Databricks
    Contract work
    Local area
    Remote work
    Flexible hours

    INSPYR Solutions

    New York, NY
    2 days ago
  • $164.64k - $246.96k

     ...positive mark on culture. Overview We are hiring a Senior Lead Data Engineer to build and scale the data foundations that power Paramount...  ...and maintain batch/streaming pipelines (Spark, Flink, Databricks, Airflow) producing ML features for ranking models. Ensure... 
    Databricks

    Paramount

    New York, NY
    1 day ago
  • A leading live entertainment company based in Pennsylvania is seeking a Data Engineer with a strong background in Data Engineering and leadership skills. The role requires experience with Databricks, Spark, Python, and SQL, along with the ability to oversee data quality... 
    Databricks

    Live Nation International

    New York, NY
    22 hours ago
  • $112k - $135k

    Publicis Media is seeking an experienced Data Engineer in New York. The ideal candidate will...  ...with ETL/ELT processes, Amazon Redshift, Databricks, and SQL. Responsibilities include...  ...data pipelines, managing datasets, and leading data platform performance efforts. The... 
    Databricks

    Publicis Media

    New York, NY
    4 days ago
  • Humana Inc is seeking a Data Engineering Lead to architect and operationalize its centralized Finance reporting data platform. This remote role involves leading design efforts for scalable Databricks lakehouse solutions and optimizing data transformation workflows to support... 
    Databricks
    Remote job

    Humana Inc

    New York, NY
    3 days ago
  •  ...Employer & Broker pillar of Data and Analytics (D&A) organization...  ...of the project (technical leads, build team, project managers...  ...platforms such as Azure Synapse, Databricks, Bigdata, Oracle, SQL Server...  ...the data loaded by data engineering team. Work with IT, business... 
    Databricks
    Work experience placement

    MetLife

    New York, NY
    4 days ago
  •  ...right candidate 10 months duration Need a strong Sr Lead Data Engineer/ Architect for clients Azure Digital Data platform Core...  ...Data platform Supply Chain and distribution company databricks Azure datalake ADF Unity Catalog Responsibilities... 
    Databricks
    Freelance
    Local area
    Remote work
    Flexible hours

    SysMind Tech

    Jersey City, NJ
    22 hours ago
  • $175k - $200k

     ...Thacher & Bartlett LLP is seeking a Senior Data Engineer to design, build, and optimize enterprise-scale data pipelines. You will lead development for data solutions using...  ...years of experience with Azure Data Factory, Databricks, SQL Server, and Python. A Bachelor's... 
    Databricks

    Simpson Thacher & Bartlett LLP

    New York, NY
    1 day ago
  •  ...): 11 Job Summary: As an ML Lead at S&P Global, you will utilize...  ..., and maintain our evolving data infrastructure, which is...  ...Management, ETL tools, Data Engineering. Required languages: English....  ...Unified analytics platforms like Databricks. Experience in managing and... 
    Databricks
    Flexible hours

    S&P Global

    New York, NY
    22 hours ago
  • $220k - $230k

    Crypto Pro Network is seeking a Senior Data Engineer in New York City. The ideal candidate has over 8 years of experience in data engineering...  ...in Python or Java, and a deep understanding of AWS and Databricks/Spark. You will be responsible for architecting and building... 
    Databricks

    Crypto Pro Network

    New York, NY
    1 day ago
  • Elevate and drive innovation, data engineering solutions, and serve as a decision maker for Firmwide...  ...Experience data products. As a Senior Lead Software Engineering at JPMorgan Chase...  ..., automation Key Technical Skills: Databricks, DBX, PySpark, Spark, Terraform, Kafka,... 
    Databricks

    Aumni

    New York, NY
    2 days ago
  • $152.9k - $221.1k

     ...Lead Data Engineer By joining Chubb as Lead Data Engineer for our North America Finance & Actuarial data platform, you'll set the engineering...  ...initiatives, including cloud migration (Azure Synapse / Databricks / Snowflake), ETL modernization, and replatforming efforts.... 
    Databricks
    Permanent employment
    Full time
    Temporary work
    Work experience placement
    Work at office
    Local area

    Chubb

    Passaic, NJ
    3 days ago
  • $170k - $210k

     ...diverse, and we are therefore committed to embracing diversity of thought and experience within our team. We are looking for a Lead Data Engineer to join our team. This is a high-impact, strategic role focused equally on technical leadership and hands-on execution. You... 
    Remote work
    Work visa
    Shift work

    Mark43

    New York, NY
    13 days ago
  •  ...Job Title- Data Engineer Location- New York, NY 10112 Reporting Type- Onsite...  ...is a plus. Responsibilities Lead end-to-end data science projects using agile...  ...sets. Loading of data into S3 as well as tabular schemas. Databricks experience preferred.
    Databricks

    campus4tech

    New York, NY
    2 days ago
  • $250.8k - $286.2k

     ...Senior Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers... 
    Full time
    Part time
    Internship
    Local area

    Capital One Financial Corp

    New York, NY
    4 days ago
  •  ...Role: Sr. Data Engineer Location: New York,NY Longterm Client: Mphasis | JPMC Rate: $60/Hr on C2C...  ...data lakehouse platforms (preferably AWS data lake services or Databricks, alternatively Hadoop), atleast one relational data stores (Postgres... 
    Databricks

    Concord IT Systems

    New York, NY
    4 days ago
  •  ...Data Engineer – Gen AI & RAG We are seeking an experienced Data Engineer with strong expertise in Generative AI, RAG architecture, and...  ...workflows using Python and PySpark Work extensively with Databricks and Snowflake for enterprise data processing Collaborate with... 
    Databricks

    Group Nine LLC

    New York, NY
    22 hours ago
  •  ...We’re hiring a hands-on Data Engineer to build scalable data platforms across key corporate functions including Finance, Compliance, Facilities...  ...’ll Do Design and optimize data pipelines using Azure + Databricks Build ETL/ELT workflows for structured and unstructured... 
    Databricks
    Contract work

    GQR

    New York, NY
    3 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer (Databricks). Be the first to apply!