Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer - Databricks & Data Architecture

Rearc

As a Lead Data Engineer at Rearc, you'll play a pivotal role in establishing and maintaining technical excellence within our data engineering team. Your deep expertise in data architecture, ETL processes, and data modeling will be instrumental in optimizing data workflows for efficiency, scalability, and reliability. You’ll work closely with cross-functional teams to design and implement robust data solutions that align with business objectives and industry best practices. Building strong partnerships with technical teams and stakeholders will be essential as you drive data-driven initiatives and ensure their successful delivery. What You Bring 10+ years of experience in data engineering, data architecture, or related technical fields, with a proven ability to design, build, and optimize large-scale data ecosystems. A strong track record of leading complex data engineering initiatives , architecting end-to-end data solutions that are scalable, reliable, and aligned with business strategy. Deep hands-on expertise in ETL/ELT design, data warehousing, and data modeling, enabling the creation of efficient, high-quality data pipelines and analytical foundations. Extensive experience with data integration frameworks and best practices , ensuring seamless, performant data movement across diverse systems and platforms. Advanced knowledge of cloud-based data services and architectures , including platforms such as AWS Redshift, Azure Synapse Analytics, Google BigQuery, or equivalent technologies. Strong strategic and analytical thinking , allowing you to tackle complex data challenges, drive architectural decisions, and influence long-term data strategy. Proficiency with modern data engineering frameworks, including Databricks, Spark , and lakehouse technologies like Delta Lake for managing and processing large-scale datasets. Exceptional communication and interpersonal skills , enabling effective collaboration with engineers, product teams, executives, and customers—while influencing technical direction and decision-making. What You’ll Do As a Lead Data Engineer, you will help shape, deliver, and elevate Rearc’s data engineering practice. You’ll lead by example—combining strong technical execution with mentorship, strategic thinking, and a culture-first approach. Engage deeply with stakeholders to understand data needs, business challenges, and technical constraints, translating them into scalable, high-quality data solutions. Implement with a DataOps mindset , applying modern engineering practices and leveraging tools such as Apache Airflow, Databricks/Spark, Kafka, or similar technologies to build reliable, automated, and efficient data pipelines and architectures. Lead and execute complex projects , providing technical direction, setting engineering standards, and ensuring alignment with customer goals and Rearc’s principles. Mentor and develop data engineers , offering guidance, code reviews, and hands-on support to help them grow technically and professionally. Promote knowledge sharing and thought leadership by contributing to internal and external content—writing technical blogs, sharing best practices, and fostering a culture of continuous learning and innovation. Some More About Us Founded in 2016, Rearc is built on the belief that great engineering happens when creativity is encouraged, bureaucracy is eliminated, and people are empowered to challenge the status quo. We’re not just a consultancy—we’re a community of problem-solvers who care deeply about improving the lives of fellow software engineers. We’re committed to empowering engineers to build awesome products and experiences. Our success hinges on our people’s ability to think freely, speak up with alternative approaches, and push the boundaries of what’s possible. If you're the kind of engineer driven by curiosity, ownership, and the desire to make a real impact—you’ll feel right at home here. Our approach is simple: give engineers the tools, autonomy, and support they need to create meaningful change —whether that’s for our clients, our team, or the industry at large. We look for engineers who thrive on responsibility and freedom, who bring both technical excellence and strong leadership, and who are eager to get hands-on. Our ideal team members don’t just talk the talk—they walk the walk, designing and building solutions that raise the bar for cloud, data, and modern engineering. Join us, and let’s build great things together. Generous time away and flexible PTO Maternity and paternity leave Access to educational resources with reimbursement for continued learning #J-18808-Ljbffr Rearc

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer - Databricks & Data Architecture in New York, NY vacancy
  •  ...Data Foundations Engineer The Data Foundations Engineer designs and scales modern data architectures powering Wallet, Payments, and Commerce products. This role focuses on building...  ...-call. ~ Experience with Snowflake, Databricks, Trino, OLAP/NRT systems, Superset or... 
    Databricks

    Apolis

    New York, NY
    4 days ago
  •  ...Lead Data Engineer - Palantir & PySpark (Lead Data engineer role where experience should be on both hands on and leading the team.)...  ...Azure, or GCP) and related data services (e.g., S3, Snowflake, Databricks). Should be familiar with CI/CD pipelines, version... 
    Databricks
    Remote work

    VBeyond

    Jersey City, NJ
    3 days ago
  • Overview Unlock your potential as a Data Engineer with a focus on innovation and impact in Venus's Team. This permanent...  ...to work with cutting-edge technologies like DataBricks, aiming to enhance data architectures and drive analytics capabilities. This role suits candidates... 
    Databricks
    Permanent employment

    FRG Technology Consulting

    New York, NY
    3 days ago
  • A data solutions company in Georgia is seeking a qualified candidate with strong expertise in Databricks Lakehouse architecture and advanced experience with PySpark and Spark SQL. Ideal applicants will possess a deep knowledge of AWS services and data orchestration tools... 
    Databricks

    VySystems

    New York, NY
    21 hours ago
  • Rearc is seeking a Senior Data Engineer in New York, New York. In this role, you'll engage directly with clients to tackle complex data...  ...technical standards for our data platforms. Your expertise in Databricks and strong code-building skills will be critical as you work... 
    Databricks

    Rearc

    New York, NY
    1 day ago
  •  ...the newly consolidated Data and Analytics (D&A)...  ...data governance, data engineering, data modeling, data analysis...  ..., and AI. The Lead Data Engineer is crucial...  ...and optimizing data architecture, solutions, and operations...  ...Big Data and Azure/Databricks for Eligibility... 
    Databricks

    MetLife

    New York, NY
    21 hours ago
  • Build modern data pipelines, lakehouse architectures, and real-time streaming infrastructure for enterprise clients...  ...the Role We're looking for a Data Engineer to help our clients build the data...  ...on cloud platforms (Snowflake, Databricks, BigQuery) Develop data quality frameworks... 
    Databricks
    Full time
    Remote work

    Mashbot, Inc.

    New York, NY
    2 days ago
  • A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate...  ...collaborate with AI/ML teams. The position requires strong skills in Databricks, Python, and SQL, along with relevant experience in cloud... 
    Databricks
    Remote job

    INSPYR Solutions

    New York, NY
    2 days ago
  • $140k - $150k

    Piper Companies is seeking a Lead Data Engineer to support a company focused on enterprise digital transformation and advanced cloud‑based...  ...will provide strategic direction for enterprise‑wide data architecture and cloud infrastructure. This role enables the organization... 

    Piper Companies

    New York, NY
    1 day ago
  • $156.8k - $235.2k

    Paramount Pictures is seeking a Senior Lead Data Engineer in New York to build and scale data foundations for personalization systems powering...  ...systems, and deep experience with tools like Spark, Kafka, Databricks, and AWS. The position offers a salary range of $156,800 - $... 
    Databricks

    Paramount Pictures

    New York, NY
    21 hours ago
  • $220k - $300k

     ...Overview The Lead Data Engineer on the Nebula team plays a significant technical leadership...  ...environment. Success requires strong architectural judgment, operational discipline, and...  ...as Snowflake, BigQuery, Redshift, Databricks, or equivalent technologies... 
    Databricks
    Local area
    Remote work
    Flexible hours

    Bayview Asset Management

    New York, NY
    2 days ago
  • $220k - $230k

     ...in digital assets and data center infrastructure,...  ...senior, hands-on data engineer with 8+ years of experience...  ...- and you're ready to lead. You set the technical...  ...data stack - AWS, Databricks/Spark, dbt, and...  ...systems end-to-end from architecture through production. You... 
    Databricks
    Local area
    Flexible hours

    Galaxy USA

    New York, NY
    4 days ago
  • A global cybersecurity leader is seeking a Lead Data Engineer to steer their data engineering efforts. This position involves designing robust...  ...guiding junior engineers, and setting the vision for data architecture. The ideal candidate should have over 5 years of experience... 

    OPSWAT

    New York, NY
    2 days ago
  • $73.01k - $170.64k

     ...Databricks Lead Data Engineer As a Databricks Lead Data Engineer, you are expected to lead the development team and have strong development...  ..., Pyspark, Databricks, Notebook Repositories, Medallion Architecture, CI/CD, and ETL platforms associated with Data Lakehouse... 
    Databricks
    Work at office
    Local area
    Flexible hours

    Perficient

    New York, NY
    1 day ago
  • $146.6k - $150k

    Tech & AI Data Engineer II - QuantumBlack, AI by McKinsey Job ID: 1...  ...workflows, and emerging agentic architectures. You will design and...  ...AI while collaborating with leading AI and data experts in the...  ...commonly used data platforms (Databricks, Snowflake, BigQuery, PSQL,... 
    Databricks
    Apprenticeship
    Work at office
    Local area
    Easy work

    McKinsey & Company

    New York, NY
    21 hours ago
  • $220k

    Specialist Recruiter | Databricks Data Engineer Recruitment | Connecting Top Talent with Leading US Opportunities Job Title: Lead Data Engineer Location: Remote Employment...  ...is a hands-on engineering role focused on architecture, implementation, and optimization of robust... 
    Databricks
    Full time
    Remote work
    Flexible hours

    KDR Talent Solutions USA

    New York, NY
    2 days ago
  • Maes Solutions is actively recruiting Lead data engineer for full-time roles with direct...  ...to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability...  ...and operating data pipelines on Databricks using orchestration tools such as... 
    Databricks
    Full time
    Relocation

    MAES Solutions

    New York, NY
    1 day ago
  • EPAM Systems, Inc. is seeking a Lead Data Software Engineer (Python+AWS) to drive the design and development of scalable data solutions. In this...  ...proficiency in Python and SQL, and expertise in AWS and Databricks. You will work in a dynamic environment while... 
    Databricks

    EPAM Systems, Inc.

    New York, NY
    2 days ago
  • The position will lead technical and business analysis that is critical in discovery process requirement for additional data loads - this would include new and existing datasets supporting...  ...AWS (Redshift, PostgreSQL, S3), Azure Databricks, Oracle, SQLSERVER, etc.) and ETL... 
    Databricks
    Work experience placement
    Interim role

    Element Technologies

    New York, NY
    1 day ago
  • Overview OKX is a leading crypto exchange and developer of OKX...  ...and related products. The Data Engineering team handles the full data...  ...designing and building data architectures and pipelines to empower evidence...  ...DataWorks, MaxCompute, AWS DataBricks, Hadoop, Hive, Spark and... 
    Databricks

    Framework Ventures

    New York, NY
    2 days ago
  • $200k - $300k

     ...IT infrastructure and engineering capabilities, positioning...  ..., modern cloud architectures, and sophisticated Artificial...  ...you'll do Lead the development and deployment...  ...that turn complex data into actionable...  ...frameworks, such as Spark or Databricks ~ Strong skills in... 
    Databricks
    Work experience placement

    Point72

    New York, NY
    4 days ago
  •  ...democratize the benefits of data and AI . Since 2019,...  ...and decisioning for leading brands and agencies while...  ...room to shape the architecture and culture from the ground up. Engineers here own major surface...  ...Partnerships with Microsoft and Databricks Fully remote or hybrid... 
    Databricks
    Remote work
    Flexible hours

    Yobi

    New York, NY
    2 days ago
  • $70 - $85 per hour

    Lead Data Engineer Location: Remote Duration: 6+ months Compensation: $70.00 - 85.00/hr Work Requirements: US Citizen...  ...maintain scalable data pipelines using Apache Spark on Databricks. Build and manage Delta Lake architectures for efficient data storage and retrieval.... 
    Databricks
    Contract work
    Local area
    Remote work
    Flexible hours

    INSPYR Solutions

    New York, NY
    2 days ago
  • $164.64k - $246.96k

     ...positive mark on culture. Overview We are hiring a Senior Lead Data Engineer to build and scale the data foundations that power Paramount...  ...and maintain batch/streaming pipelines (Spark, Flink, Databricks, Airflow) producing ML features for ranking models. Ensure... 
    Databricks

    Paramount

    New York, NY
    1 day ago
  • $170k - $281k

     ...Principal Software Engineer, Data Architecture Mastercard's Data & Analytics organization is seeking...  ...Architecture Review Board. Lead the modernization of legacy data platforms...  ...modern data technologies, including Databricks, Snowflake, Delta Lake, and streaming... 
    Databricks
    Worldwide

    Dynamic Yield

    New York, NY
    1 day ago
  • $210k - $230k

     ...Evercore Inc. is seeking an Experienced Director, Lead, Enterprise Data Architecture in New York. This high-impact role requires extensive expertise in data engineering, specifically using Databricks and Apache Spark. Responsibilities include defining enterprise architecture... 
    Databricks

    Evercore

    New York, NY
    3 days ago
  • A leading live entertainment company based in Pennsylvania is seeking a Data Engineer with a strong background in Data Engineering and leadership skills. The role requires experience with Databricks, Spark, Python, and SQL, along with the ability to oversee data quality... 
    Databricks

    Live Nation International

    New York, NY
    21 hours ago
  •  ...Data And Analytics Engineering Lead As Data And Analytics Engineering Lead within our Reinsurance Data...  ...and implementation of enterprise BI architecture, including data models, dashboards,...  ...including Azure Data Lake Storage, Databricks, and other cloud or on-premise data... 
    Databricks

    Oliver Wyman

    New York, NY
    1 day ago
  • $65 - $70 per hour

    ECLARO is seeking a Senior Data Engineer for a remote opportunity based in New York. The role...  ..., mentoring peers, and driving architectural decisions. Candidates should have 5-9...  ...with hands-on experience in Azure and Databricks. Benefits include a 401k plan, medical... 
    Databricks
    Hourly pay
    Remote work

    ECLARO

    New York, NY
    2 days ago
  • $220k - $300k

    Bayview Asset Management, LLC is looking for a Lead Data Engineer to join their Nebula team. This fully remote role involves shaping and...  ...benefits, this position offers a great opportunity to drive data architecture and quality. #J-18808-Ljbffr Bayview Asset Management, LLC
    Remote job

    Bayview Asset Management, LLC

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer - Databricks & Data Architecture. Be the first to apply!