Lead Data Engineer
The Judge Group, LLC
Lead Data Engineer
New York, NY (10001)
6+ months Contract to Perm About the Role
We are seeking a Lead Software Engineer - Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence. Responsibilities
Design and develop large-scale data pipelines and data platforms.
Build cloud-native solutions using AWS services and Spark.
Lead data modeling, warehousing, migration, and analytics initiatives.
Improve system performance, resilience, monitoring, and automation.
Collaborate with cross-functional teams and mentor junior engineers. Required Qualifications
9-12+ Experience in Data Engineering.
Strong proficiency in Python and/or Java with Spark/PySpark.
Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration.
Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).
Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles.
Experience with data formats such as Parquet, Iceberg, Avro, or JSON.
Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager. Preferred Qualifications
Experience building REST APIs using Java.
Knowledge of Hybrid Cloud architectures.
Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.
Strong leadership, problem-solving, and communication skills. Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.
New York, NY (10001)
6+ months Contract to Perm About the Role
We are seeking a Lead Software Engineer - Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence. Responsibilities
Design and develop large-scale data pipelines and data platforms.
Build cloud-native solutions using AWS services and Spark.
Lead data modeling, warehousing, migration, and analytics initiatives.
Improve system performance, resilience, monitoring, and automation.
Collaborate with cross-functional teams and mentor junior engineers. Required Qualifications
9-12+ Experience in Data Engineering.
Strong proficiency in Python and/or Java with Spark/PySpark.
Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration.
Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).
Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles.
Experience with data formats such as Parquet, Iceberg, Avro, or JSON.
Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager. Preferred Qualifications
Experience building REST APIs using Java.
Knowledge of Hybrid Cloud architectures.
Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.
Strong leadership, problem-solving, and communication skills. Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in New York, NY vacancy
- ...Lead Data Engineer – Big Data & Cloud (AWS)(Python) Location: New York, NY (Onsite) Contract: 6 Months Visa Status: ONLY GC & USC Strong Python and Big Data expertise JOB DESCRIPTION: We are seeking a Senior Data Engineer / Technology Lead with 10–15...SuggestedContract work
Argyle Infotech
New York, NY4 days ago $179.4k - $204.7k
...Lead Data Engineer (Bank Tech) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of...SuggestedFull timePart timeInternshipH1bLocal areaCapital One Financial Corp
New York, NY22 hours ago$220k - $260k
...Overview The Lead Data Engineer on the Nebula team plays a significant technical leadership role in shaping and scaling the data foundation that powers analytics, reporting, AI development, and operational decision-making across the organization. This role combines...SuggestedLocal areaRemote workFlexible hoursBayview Asset Management
New York, NY22 hours ago- ...Job Title: Snowflake Lead Data Engineering Duration- Fulltime Permanent Location: New York, NY (Onsite From Day 1) Job Description: Must Have Technical/Functional Skills Snowflake, Cortex AI, Python and...SuggestedPermanent employmentFull time
Q1 Technologies
New York, NY3 days ago $115k - $150k
...difference our hard work makes, and continue on our own paths of lifelong learning. How can you make an impact? The Lead Data Engineer in Data and Analytics is responsible for enhancing McGraw-Hill Education's (MHE) business intelligence and data services capabilities...SuggestedRemote workWorldwideMcGraw-Hill Education
New York, NY2 days ago$170k - $220k
...Lead Data Engineer New York, NY What Is Coast Coast is re-imagining the trillion-dollar U.S. B2B card payments infrastructure, with a focus on the country's 500,000 commercial fleets, 40 million commercial vehicles, and many millions of commercial drivers. The...Work at officeLocal areaRelocation packageFlexible hoursCoast
New York, NY3 days ago- Data Engineering Manager Location: Remote (NY/NJ residents only) Direct Hire Overview Seeking a hands-on engineering leader to guide... ...modernization of a cloud-based data platform. Responsibilities Lead, mentor, and perform code reviews for a team of engineers....For contractorsWork at officeLocal areaRemote work
Vaco
New York, NY22 hours ago - ...through innovative product design and industry-leading technology. Our portfolio of brands––... ...entertainment. Powered by vast data and insights, TTG helps theatres, producers... ...the Role: We’re looking for a Lead Data Engineer to join TodayTix Group and own how we scale...Full timeContract workWork at officeFlexible hoursShift work2 days per week
TodayTix Group
New York, NY16 days ago $135k - $180k
...million patients, Medidata offers industry-leading expertise, analytics-powered insights, and one of the largest clinical trial data sets in the industry. More than 1 million... ...including product, delivery, data science, engineering, partnerships, and biostatistics....Ongoing contractWork at officeLocal areaFlexible hoursDassault Systèmes
New York, NY12 days ago- ...Snowflake Lead – Data Engineering San Antonio, TX / New York, NY (Onsite) Local to San Antonio, TX / New York, NY only! P&C / Specialty Insurance exp Must Must-Have Skills – Snowflake Lead (Data Engineering) * 12+ years of Data Engineering / Analytics experience...Temporary workLocal area
Delan Associates, Inc
New York, NYa month ago $197.3k - $225.1k
...Overview Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery...Full timePart timeInternshipH1bLocal areaCapital One
New York, NY2 days ago$130k - $155k
...BI/Tableau Report Developer to join our data team, specializing in the Media and Entertainment... ...Media Data Visualization Leadership: Lead the entire lifecycle (design, development... .... Collaborate extensively with Data Engineers and Scientists to define and refine data...Full timeTemporary workPart timeLocal areaTegna
New York, NY22 hours ago- ...TechnologyExperience Level: Experience ProfessionalsContact: Paul KrugJob ID: REQ8505Role OverviewWe are seeking a hands-on Senior / Lead Data Engineer to provide technical leadership for the platforms, pipelines, and data products that power analytics, applications, and...
$120k - $180k
...position sits within the newly consolidated Data and Analytics (D&A) organization... ...data infrastructure, data governance, data engineering, data modeling, data analysis to business intelligence, data science, and AI.The Lead Data Engineer is crucial to D&A USB’s Eligibility...Full timeTemporary workWork at officeLocal areaRelocation package3 days per week- ...Description Build What Matters at RevStar Role Title: Data & AI Engineer (Databricks Specialist) Reports To: Data & AI Practice Lead Location: Remote (US-Based / Eastern or Central Time Zone Preferred) Employment Type: Full-Time W2 Ready to build greenfield...Remote jobFull time
Revstar
New York, NY14 hours ago - ...connect. With deep industry expertise and a data-driven platform, we deliver subscriptions... .... We're looking for a Data & AI Engineer to help accelerate adoption of AI and shape... ...current. Support audience enrichment, lead and contact enrichment, and internal/external...Full time
Arizent
New York, NY22 hours ago $130k - $155k
...Job Title: Lead BI/Data/Web Developer Location-Type: Hybrid (3 days/week onsite: Mon/Tue/Thu, in NYC or Tysons, VA) Start Date... ...sit on the front-end/user-facing side of the business (data engineers handle the backend), working closely with a strong India-based...Permanent employmentFull timeWork experience placementImmediate start3 days per weekMondo
New York, NY11 days ago$197.3k - $225.1k
...Overview Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery...Full timePart timeInternshipH1bLocal area- ...Responsibilities: Build and maintain data pipelines using Python and SQL. Create and manage workflows in Databricks and... ...and fix issues to meet service goals. Work with analytics, engineering, and business teams to deliver clean, ready-to-use datasets....Work at office
Trinity Technology Solutions
New York, NY1 day ago - ...contributing author on any and all project deliverables associated with their assigned areas of responsibility * Participate in data conversion and data maintenance * Provide best practice and industry specific solutions * Advise on and provide...
United IT Solutions
New York, NY3 days ago - ...Data Engineer Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable...Flexible hours
BaseTen
New York, NY1 day ago - ...AWS Data Engineer Location: NYC, NY – Onsite Position Summary The Senior Data Engineer will design, build, and support data pipelines and analytics datasets that power enterprise reporting across Finance, Technology, and Operations. This role focuses on ingesting...
United IT
New York, NY4 days ago $140k - $155k
...Link Logistics Real Estate ("Link") is a leading operator of warehouses and business parks... ...feet. We leverage our scale, proprietary data and insights, and foundational focus on sustainability... .... We're hiring a hybrid ML/Backend Engineer to own the intelligence layer of Link's...Local areaLink Logistics Corp
New York, NY2 days ago- ...our unparalleled access to a wide range of publicly available data sources. About our Team: KEPL is a fast-growing team at... ...Requirements ~ Master/PhD degree in math, computer science, engineering, or other related fields. ~1-3 years of professional...
Point72
New York, NY3 days ago - ...Data Engineer Employment Type: Full-Time, Mid-level CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence platform focused on providing solutions that empower our federal customers with the tools...Full timeFlexible hours
Contact Government Services LLC
New York, NY4 days ago - ...Big Data Engineer Job Type: Full Time Employment Location: Phoenix, AZ or New York Experience Required: 6+ Years Responsibilities: Minimum 6 years of experience on Big Data Hadoop technologies & software development Hands on experience on Big Data platforms...Full time
Syntricate Technologies
New York, NY4 days ago $115k - $135k
...Data Engineer Within Healthcare Design, build, and maintain scalable batch data pipelines, transformation workflows, dimensional models, and curated analytical data models using SQL, Python, Databricks, Apache Spark/PySpark, and distributed data processing technologies...Permanent employmentFull timeLocal areaMichael Page
New York, NY22 hours ago$73.15k - $112.55k
...including content, commerce, multicultural and future-proofed data capabilities. Responsibilities Design, build, and maintain... ...optimizing data pipelines and feeds. Collaborate with the data engineering team on performance tuning, query optimization, and code...Temporary workFreelanceFlexible hoursPublicis Groupe Holdings B.V
New York, NY4 days ago$160k - $200k
...crypto industry. We transform raw, complex data and facts into actionable research,... ...asset class. Who You Are: Software engineering is your passion, and you have special... ...and develop our data pipelines as well as leading our data-related projects and strategies...Full timeWork at officeRemote workFlexible hoursBlockworks
New York, NY4 days ago$110k - $190k
...Senior Data Management Professional - Data Engineering - Private Funds Location New York Business Area Data Ref # 10052928 Description &... ...applications with our product teams. You will be expected to lead projects, collaborate with Engineering and Product,...Temporary workFor contractorsWork experience placementBloomberg
New York, NY3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
