Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer

$70 - $85 per hour

INSPYR Solutions

Lead Data Engineer Location: Remote Duration: 6+ months Compensation: $70.00 - 85.00/hr Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S. Summary We are looking for a Lead Data Engineer who excels in data modeling and can build end-to-end data solutions. The ideal candidate will bring strong leadership skills, with proven experience driving projects from initiation through successful completion. Responsibilities Design, develop, and maintain scalable data pipelines using Apache Spark on Databricks. Build and manage Delta Lake architectures for efficient data storage and retrieval. Implement robust ETL/ELT workflows using Databricks notebooks, SQL, and Python. Collaborate with AI/ML teams to operationalize models within the Databricks environment. Optimize data workflows for performance, reliability, and cost-efficiency in cloud platforms (AWS, Azure, or GCP). Ensure data quality, lineage, and governance using tools like Unity Catalog and MLflow. Develop CI/CD pipelines for data and ML workflows using Databricks Repos and Git integrations. Monitor and troubleshoot production data pipelines and model deployments. Key Responsibilities Strong hands‑on experience with Databricks, including Spark, Delta Lake, and MLflow. Proficiency in Python, SQL, and distributed data processing. Experience with cloud‑native data services (e.g., AWS Glue, Azure Data Factory, GCP Dataflow). Familiarity with machine learning lifecycle and integration of models into data pipelines. Understanding of data warehousing, data lakehouse architecture, and real‑time streaming (Kafka, Spark Structured Streaming). Experience with version control, CI/CD, and infrastructure‑as‑code tools. Excellent communication and collaboration skills. Certifications in Databricks (e.g., Databricks Certified Data Engineer Associate/Professional). Experience with feature engineering and feature stores in Databricks. Exposure to MLOps practices and tools. Bachelor's or Master's degree in Computer Science, Data Engineering, or related field. Leveraged Databricks for scalable AI and BI solutions, integrating well‑known large language models (Anthropic, LLaMA, Gemini) to enhance data‑driven insights. Developed agentic AI agents to automate complex decision‑making workflows. Technology Stack Databricks (Spark, Delta Lake, MLflow, Notebooks) Python & SQL Apache Spark (via Databricks) Delta Lake (for lakehouse architecture) Cloud Platforms: Azure, AWS, or GCP Cloud Storage: ADLS, S3, GCS Data Integration: Kafka or Event Hubs, Auto Loader (Databricks file ingestion) REST APIs AI/ML: MLflow (model tracking/deployment), Hugging Face Transformers, LangChain / LlamaIndex (LLM integration), LLMs (Anthropic Claude, Meta LLaMA, Google Gemini) DevOps: Git (GitHub, GitLab, Azure Repos), Databricks Repos, CI/CD (GitHub Actions, Azure DevOps) Security & Governance: Unity Catalog, RBAC Our Benefits Package Includes Comprehensive medical benefits Competitive pay 401(k) retirement plan ...and much more! About INSPYR Solutions Technology is our focus and quality is our commitment. As a national expert in delivering flexible technology and talent solutions, we strategically align industry and technical expertise with our clients' business objectives and cultural needs. Our solutions are tailored to each client and include a wide variety of professional services, projects, and talent solutions. By always striving for excellence and focusing on the human aspect of our business, we work seamlessly with our talent and clients to match the right solutions to the right opportunities. Learn more about us at inspyrsolutions.com. INSPYR Solutions provides Equal Employment Opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, or genetics. In addition to federal law requirements, INSPYR Solutions complies with applicable state and local laws governing nondiscrimination in employment in every location in which the company has facilities. Seniority Level Mid‑Senior level Employment Type Contract Job Function Information Technology Industries Wholesale Building Materials and Construction #J-18808-Ljbffr INSPYR Solutions

Vacancy posted 11 hours ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in New York, NY vacancy
  • EPAM Systems, Inc. is seeking a skilled individual to join their team, focusing on the development of a data platform. The ideal candidate will oversee a team, manage complex data workflows, and contribute to architectural choices. Qualified applicants should have strong... 
    Suggested
    Remote job
    Flexible hours

    EPAM Systems, Inc.

    New York, NY
    3 days ago
  • $125k - $140k

     ...- TMX Careers. The TMX group of companies includes leading global exchanges such as the Toronto Stock Exchange...  ...lines of business: Indexing, Digital Distribution, and Data & Analytics. Made up of developers, data engineers, designers, and project managers, the platform team... 
    Suggested

    VettaFi

    New York, NY
    2 days ago
  • A leading financial firm in New York is seeking a Lead Market Data Engineer to architect and optimize platforms delivering high-quality market data. This hands-on role involves designing scalable data pipelines and collaborating with teams to ensure smooth data integration... 
    Suggested
    Full time
    Contract work

    Balyasny Asset Management

    New York, NY
    11 hours ago
  • $150k - $195k

    Enercon Services, Inc. is looking for a Principal Data Center MEP Engineer to lead design and project direction in the power industry. You will drive impactful solutions, manage teams, develop detailed designs and ensure project execution. This role requires a Bachelor... 
    Suggested
    Remote job

    Enercon Services, Inc.

    Brooklyn, NY
    2 days ago
  •  ...Senior Python and Snowflake Developer to design, build, and optimize scalable data pipelines, develop robust backend services, implement Snowflake data models, and drive end-to-end data engineering solutions. The role involves collaborating with cross‑functional teams,... 
    Suggested

    TheStaffed

    New York, NY
    4 days ago
  • A leading Behavioral AI company in New York is seeking a Lead Data Engineer to oversee data workflows and ensure the reliability of their products. This hands-on role involves working on schema modeling and orchestration techniques while collaborating with cross-functional... 
    Remote job

    Yobi AI

    New York, NY
    1 day ago
  •  ...collaborate directly with the client's team, make business trips.... Digis islooking for anexperienced, proactive, and self-driven Lead Data Engineer tojoin our fully remote team. About the Project You’ll bepartofalarge-scale platform focused on performance management and... 
    Remote work

    Digis Corp

    New York, NY
    11 hours ago
  • $180k - $250k

     ...experience — talk with your recruiter to learn more. Base pay range $180,000.00/yr - $250,000.00/yr A world-leading trading firm is seeking a Lead Data Engineer to architect, scale, and optimize the data infrastructure powering its multi-billion-dollar systematic trading... 
    Full time

    Evolve Group, Inc.

    New York, NY
    11 hours ago
  • $156.8k - $235.2k

    Paramount Pictures is seeking a Senior Lead Data Engineer in New York to build and scale data foundations for personalization systems powering Home, Search/Browse, Notifications, and Artwork. This high-impact role requires expertise in large-scale ETL/streaming, distributed... 

    Paramount Pictures

    New York, NY
    3 days ago
  • $149.3k - $200.2k

     ...audiences experience sports, entertainment & news. The Product & Data Engineering team is responsible for end to end development for Disney’s...  ...personalization, search, messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team, you will be partnering... 
    Work experience placement
    Worldwide

    The Walt Disney Company

    New York, NY
    11 hours ago
  • Lead Data Engineer in NY job at Vertex Elite LLC. New York, NY. Location: New York City, NY; (Onsite 5 days/week) Duration: Long Term Contract Tech skill sets: Data Architecture, Data Modelling, Expert level SQL, Python, AWS Technologies (Glue), Snowflake and data engineering... 
    Long term contract

    Vertex Elite LLC

    New York, NY
    1 day ago
  • $197.3k - $225.1k

    Lead Data Engineer (Bank Tech) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers... 
    Full time
    Part time
    Internship
    Local area

    Capital One

    New York, NY
    3 days ago
  • W Talent is partnering with a leading organisation within the Financial Services sector to appoint a Lead Data Engineer as part of a significant investment in its Data & Analytics capability. This is an excellent opportunity to join a growing team responsible for delivering... 

    W Talent

    New York, NY
    2 days ago
  •  ...to ethically democratize the benefits of data and AI . Since 2019, we have built one of...  ...‑art personalization and decisioning for leading brands and agencies while protecting privacy...  ...and culture from the ground up. Engineers here own major surface areas , build 0→1... 
    Remote work
    Flexible hours

    Yobi AI

    New York, NY
    11 hours ago
  • $170k - $220k

     ...platform. About The Role We are looking for a leader of the Coast data engineering team to shape our company’s data culture and underlying...  ...the Head of Engineering from our NYC office. What You’ll Do Lead design and implementation of all aspects of our data ecosystem... 
    Work at office
    Local area
    Relocation package
    Flexible hours

    Better Tomorrow Ventures

    New York, NY
    11 hours ago
  • Position Overview We are seeking a Lead Data Engineer to join a rapidly growing Data & Analytics organization responsible for delivering enterprise-scale data products, analytics solutions, and cloud-based data platforms. This role will lead the design, development, and... 

    Burtch Works

    New York, NY
    3 days ago
  • Lead Market Data Engineer - Systematic Data Join to apply for the Lead Market Data Engineer - Systematic Data role at Balyasny Asset Management L.P. The Lead Market Data Engineer will architect, build, and optimize the platforms that deliver high‑quality market data to... 
    Full time

    Balyasny Asset Management

    New York, NY
    11 hours ago
  • Carlsbad Tech is looking for a Lead Data Engineer to develop and optimize data pipelines in New York, NY. This role requires expertise in building data solutions using technologies like Kafka and Snowflake, ensuring scalable and reliable data processing. Join a team that... 

    Carlsbad Tech

    New York, NY
    11 hours ago
  • $135k - $150k

    New York, United States | Posted on 08/30/2023 The position of Lead Data Engineer is open in New YorkCity, New York, within the Insurance industry. The role offers flexiblework-from-home days while requiring occasional on-site presence for clientmeetings. The Lead Data... 

    Career-Mover

    New York, NY
    1 day ago
  • $220k

    Specialist Recruiter | Databricks Data Engineer Recruitment | Connecting Top Talent with Leading US Opportunities Job Title: Lead Data Engineer Location: Remote Employment Type: Full-time Compensation: Up to $220k + benefits About the Role We’re hiring experienced... 
    Full time
    Remote work
    Flexible hours

    KDR Talent Solutions USA

    New York, NY
    11 hours ago
  • The position will lead technical and business analysis that is critical in discovery process requirement for additional data loads - this would include new and existing datasets supporting the taxonomy for JJMD - (Core & Non-core taxonomy subject areas) Strong collaboration... 
    Work experience placement
    Interim role

    Element Technologies

    New York, NY
    4 days ago
  • Lead Data Engineer - Front Office | Global Multi-Strategy Firm | Up to $600k TC My client is a global multi-strategy firm operating across public and private markets. The data platform sits at the centre of capital allocation, risk, and PnL. Golden-source datasets, security... 
    Immediate start

    Mondrian Alpha

    New York, NY
    4 days ago
  • A leading athletic apparel company is seeking a Lead Data Engineer to establish and maintain database systems while building efficient data pipelines. The role requires strong skills in Python and SQL, along with 8+ years of data engineering experience. This remote position... 
    Remote job
    Contract work

    NIKE, Inc.

    New York, NY
    11 hours ago
  • A leading technology firm is seeking a Mid-Senior level Data Engineering Lead to manage a team for complex data integration projects. Your responsibilities will include designing data models, developing ETL/ELT pipelines, and ensuring data accuracy. The ideal candidate... 
    Remote job

    Largeton Group

    New York, NY
    11 hours ago
  • $160k - $180k

    Metrics Recruitment is looking for a Lead/Senior Data Engineer to build critical data infrastructure for a B2B SaaS company in New York. You will collaborate directly with the CTO to create and scale a modern data architecture, owning projects from inception through deployment... 

    Metrics Recruitment

    New York, NY
    1 day ago
  • $220k - $230k

    Who You Are You're a senior, hands‑on data engineer with 8+ years of experience designing, building, and operating production data platforms at scale — and you're ready to lead. You set the technical direction for data infrastructure, mentor other engineers, and raise... 
    Flexible hours

    Unchain Data

    New York, NY
    2 days ago
  • Microsoft Fabric Developer / Data Engineer Overview strong expertise in architecture design and implementation. proven experience in customer interaction and stakeholder management. Experience in job scheduling , monitoring and maintaining jobs in on-premise using MS SQL... 

    Career Steer Services

    New York, NY
    1 day ago
  •  ...technology. Join this dynamic team at an early stage and be part of the exciting journey to make a difference to the industry. Lead Data & Analytics Engineer As Lead Data and Analytics Engineer at Secturasoft, you will work with the engineers, product owners, and the business... 
    Immediate start

    O'Neal Industries, Inc.

    New York, NY
    11 hours ago
  • $145.2k - $217.88k

    Mount Sinai Medical Center is seeking a Lead Data Warehouse Engineer to integrate multi-modal clinical data into the AIR.MS data warehouse. This senior role involves designing data pipelines, mentoring engineers, and ensuring compliance with privacy regulations. Candidates... 

    Mount Sinai Medical Center

    New York, NY
    4 days ago
  • SwiftCruit is seeking a Feature Lead Data Engineer based in New York, NY, responsible for building data pipelines to handle large structured and unstructured data volumes. You will design and engineer high-performance platforms that meet GenAI requirements. The role involves... 

    SwiftCruit

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!