Lead Data Engineer - Databricks & Data Architecture
Rearc
As a Lead Data Engineer at Rearc, you'll play a pivotal role in establishing and maintaining technical excellence within our data engineering team. Your deep expertise in data architecture, ETL processes, and data modeling will be instrumental in optimizing data workflows for efficiency, scalability, and reliability. You’ll work closely with cross-functional teams to design and implement robust data solutions that align with business objectives and industry best practices. Building strong partnerships with technical teams and stakeholders will be essential as you drive data-driven initiatives and ensure their successful delivery. What You Bring 10+ years of experience in data engineering, data architecture, or related technical fields, with a proven ability to design, build, and optimize large-scale data ecosystems. A strong track record of leading complex data engineering initiatives , architecting end-to-end data solutions that are scalable, reliable, and aligned with business strategy. Deep hands-on expertise in ETL/ELT design, data warehousing, and data modeling, enabling the creation of efficient, high-quality data pipelines and analytical foundations. Extensive experience with data integration frameworks and best practices , ensuring seamless, performant data movement across diverse systems and platforms. Advanced knowledge of cloud-based data services and architectures , including platforms such as AWS Redshift, Azure Synapse Analytics, Google BigQuery, or equivalent technologies. Strong strategic and analytical thinking , allowing you to tackle complex data challenges, drive architectural decisions, and influence long-term data strategy. Proficiency with modern data engineering frameworks, including Databricks, Spark , and lakehouse technologies like Delta Lake for managing and processing large-scale datasets. Exceptional communication and interpersonal skills , enabling effective collaboration with engineers, product teams, executives, and customers—while influencing technical direction and decision-making. What You’ll Do As a Lead Data Engineer, you will help shape, deliver, and elevate Rearc’s data engineering practice. You’ll lead by example—combining strong technical execution with mentorship, strategic thinking, and a culture-first approach. Engage deeply with stakeholders to understand data needs, business challenges, and technical constraints, translating them into scalable, high-quality data solutions. Implement with a DataOps mindset , applying modern engineering practices and leveraging tools such as Apache Airflow, Databricks/Spark, Kafka, or similar technologies to build reliable, automated, and efficient data pipelines and architectures. Lead and execute complex projects , providing technical direction, setting engineering standards, and ensuring alignment with customer goals and Rearc’s principles. Mentor and develop data engineers , offering guidance, code reviews, and hands-on support to help them grow technically and professionally. Promote knowledge sharing and thought leadership by contributing to internal and external content—writing technical blogs, sharing best practices, and fostering a culture of continuous learning and innovation. Some More About Us Founded in 2016, Rearc is built on the belief that great engineering happens when creativity is encouraged, bureaucracy is eliminated, and people are empowered to challenge the status quo. We’re not just a consultancy—we’re a community of problem-solvers who care deeply about improving the lives of fellow software engineers. We’re committed to empowering engineers to build awesome products and experiences. Our success hinges on our people’s ability to think freely, speak up with alternative approaches, and push the boundaries of what’s possible. If you're the kind of engineer driven by curiosity, ownership, and the desire to make a real impact—you’ll feel right at home here. Our approach is simple: give engineers the tools, autonomy, and support they need to create meaningful change —whether that’s for our clients, our team, or the industry at large. We look for engineers who thrive on responsibility and freedom, who bring both technical excellence and strong leadership, and who are eager to get hands-on. Our ideal team members don’t just talk the talk—they walk the walk, designing and building solutions that raise the bar for cloud, data, and modern engineering. Join us, and let’s build great things together. Generous time away and flexible PTO Maternity and paternity leave Access to educational resources with reimbursement for continued learning #J-18808-Ljbffr Rearc
- ...Data Foundations Engineer The Data Foundations Engineer designs and scales modern data architectures powering Wallet, Payments, and Commerce products. This role focuses on building... ...-call. ~ Experience with Snowflake, Databricks, Trino, OLAP/NRT systems, Superset or...Databricks
- ...Lead Data Engineer - Palantir & PySpark (Lead Data engineer role where experience should be on both hands on and leading the team.)... ...Azure, or GCP) and related data services (e.g., S3, Snowflake, Databricks). Should be familiar with CI/CD pipelines, version...DatabricksRemote work
- Overview Unlock your potential as a Data Engineer with a focus on innovation and impact in Venus's Team. This permanent... ...to work with cutting-edge technologies like DataBricks, aiming to enhance data architectures and drive analytics capabilities. This role suits candidates...DatabricksPermanent employment
- A data solutions company in Georgia is seeking a qualified candidate with strong expertise in Databricks Lakehouse architecture and advanced experience with PySpark and Spark SQL. Ideal applicants will possess a deep knowledge of AWS services and data orchestration tools...Databricks
- Rearc is seeking a Senior Data Engineer in New York, New York. In this role, you'll engage directly with clients to tackle complex data... ...technical standards for our data platforms. Your expertise in Databricks and strong code-building skills will be critical as you work...Databricks
- ...the newly consolidated Data and Analytics (D&A)... ...data governance, data engineering, data modeling, data analysis... ..., and AI. The Lead Data Engineer is crucial... ...and optimizing data architecture, solutions, and operations... ...Big Data and Azure/Databricks for Eligibility...Databricks
- Build modern data pipelines, lakehouse architectures, and real-time streaming infrastructure for enterprise clients... ...the Role We're looking for a Data Engineer to help our clients build the data... ...on cloud platforms (Snowflake, Databricks, BigQuery) Develop data quality frameworks...DatabricksFull timeRemote work
- A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate... ...collaborate with AI/ML teams. The position requires strong skills in Databricks, Python, and SQL, along with relevant experience in cloud...DatabricksRemote job
$140k - $150k
Piper Companies is seeking a Lead Data Engineer to support a company focused on enterprise digital transformation and advanced cloud‑based... ...will provide strategic direction for enterprise‑wide data architecture and cloud infrastructure. This role enables the organization...$156.8k - $235.2k
Paramount Pictures is seeking a Senior Lead Data Engineer in New York to build and scale data foundations for personalization systems powering... ...systems, and deep experience with tools like Spark, Kafka, Databricks, and AWS. The position offers a salary range of $156,800 - $...Databricks$220k - $300k
...Overview The Lead Data Engineer on the Nebula team plays a significant technical leadership... ...environment. Success requires strong architectural judgment, operational discipline, and... ...as Snowflake, BigQuery, Redshift, Databricks, or equivalent technologies...DatabricksLocal areaRemote workFlexible hours$220k - $230k
...in digital assets and data center infrastructure,... ...senior, hands-on data engineer with 8+ years of experience... ...- and you're ready to lead. You set the technical... ...data stack - AWS, Databricks/Spark, dbt, and... ...systems end-to-end from architecture through production. You...DatabricksLocal areaFlexible hours- A global cybersecurity leader is seeking a Lead Data Engineer to steer their data engineering efforts. This position involves designing robust... ...guiding junior engineers, and setting the vision for data architecture. The ideal candidate should have over 5 years of experience...
$73.01k - $170.64k
...Databricks Lead Data Engineer As a Databricks Lead Data Engineer, you are expected to lead the development team and have strong development... ..., Pyspark, Databricks, Notebook Repositories, Medallion Architecture, CI/CD, and ETL platforms associated with Data Lakehouse...DatabricksWork at officeLocal areaFlexible hours$146.6k - $150k
Tech & AI Data Engineer II - QuantumBlack, AI by McKinsey Job ID: 1... ...workflows, and emerging agentic architectures. You will design and... ...AI while collaborating with leading AI and data experts in the... ...commonly used data platforms (Databricks, Snowflake, BigQuery, PSQL,...DatabricksApprenticeshipWork at officeLocal areaEasy work$220k
Specialist Recruiter | Databricks Data Engineer Recruitment | Connecting Top Talent with Leading US Opportunities Job Title: Lead Data Engineer Location: Remote Employment... ...is a hands-on engineering role focused on architecture, implementation, and optimization of robust...DatabricksFull timeRemote workFlexible hours- Maes Solutions is actively recruiting Lead data engineer for full-time roles with direct... ...to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability... ...and operating data pipelines on Databricks using orchestration tools such as...DatabricksFull timeRelocation
- EPAM Systems, Inc. is seeking a Lead Data Software Engineer (Python+AWS) to drive the design and development of scalable data solutions. In this... ...proficiency in Python and SQL, and expertise in AWS and Databricks. You will work in a dynamic environment while...Databricks
- The position will lead technical and business analysis that is critical in discovery process requirement for additional data loads - this would include new and existing datasets supporting... ...AWS (Redshift, PostgreSQL, S3), Azure Databricks, Oracle, SQLSERVER, etc.) and ETL...DatabricksWork experience placementInterim role
- Overview OKX is a leading crypto exchange and developer of OKX... ...and related products. The Data Engineering team handles the full data... ...designing and building data architectures and pipelines to empower evidence... ...DataWorks, MaxCompute, AWS DataBricks, Hadoop, Hive, Spark and...Databricks
$200k - $300k
...IT infrastructure and engineering capabilities, positioning... ..., modern cloud architectures, and sophisticated Artificial... ...you'll do Lead the development and deployment... ...that turn complex data into actionable... ...frameworks, such as Spark or Databricks ~ Strong skills in...DatabricksWork experience placement- ...democratize the benefits of data and AI . Since 2019,... ...and decisioning for leading brands and agencies while... ...room to shape the architecture and culture from the ground up. Engineers here own major surface... ...Partnerships with Microsoft and Databricks Fully remote or hybrid...DatabricksRemote workFlexible hours
$70 - $85 per hour
Lead Data Engineer Location: Remote Duration: 6+ months Compensation: $70.00 - 85.00/hr Work Requirements: US Citizen... ...maintain scalable data pipelines using Apache Spark on Databricks. Build and manage Delta Lake architectures for efficient data storage and retrieval....DatabricksContract workLocal areaRemote workFlexible hours$164.64k - $246.96k
...positive mark on culture. Overview We are hiring a Senior Lead Data Engineer to build and scale the data foundations that power Paramount... ...and maintain batch/streaming pipelines (Spark, Flink, Databricks, Airflow) producing ML features for ranking models. Ensure...Databricks$170k - $281k
...Principal Software Engineer, Data Architecture Mastercard's Data & Analytics organization is seeking... ...Architecture Review Board. Lead the modernization of legacy data platforms... ...modern data technologies, including Databricks, Snowflake, Delta Lake, and streaming...DatabricksWorldwide$210k - $230k
...Evercore Inc. is seeking an Experienced Director, Lead, Enterprise Data Architecture in New York. This high-impact role requires extensive expertise in data engineering, specifically using Databricks and Apache Spark. Responsibilities include defining enterprise architecture...Databricks- A leading live entertainment company based in Pennsylvania is seeking a Data Engineer with a strong background in Data Engineering and leadership skills. The role requires experience with Databricks, Spark, Python, and SQL, along with the ability to oversee data quality...Databricks
- ...Data And Analytics Engineering Lead As Data And Analytics Engineering Lead within our Reinsurance Data... ...and implementation of enterprise BI architecture, including data models, dashboards,... ...including Azure Data Lake Storage, Databricks, and other cloud or on-premise data...Databricks
$65 - $70 per hour
ECLARO is seeking a Senior Data Engineer for a remote opportunity based in New York. The role... ..., mentoring peers, and driving architectural decisions. Candidates should have 5-9... ...with hands-on experience in Azure and Databricks. Benefits include a 401k plan, medical...DatabricksHourly payRemote work$220k - $300k
Bayview Asset Management, LLC is looking for a Lead Data Engineer to join their Nebula team. This fully remote role involves shaping and... ...benefits, this position offers a great opportunity to drive data architecture and quality. #J-18808-Ljbffr Bayview Asset Management, LLCRemote job
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer - Databricks & Data Architecture. Be the first to apply!
- senior data center engineer New York, NY
- bi data engineer New York, NY
- data engineer manager New York, NY
- data science developer New York, NY
- junior data developer New York, NY
- etl data engineer New York, NY
- entry level big data engineer New York, NY
- data engineer New York, NY
- big data cloud engineer New York, NY
- junior big data engineer New York, NY

