Lead Data Engineer
SO Digital Recruitment
We are partnered with an established Financial Services firm based in New York. They are hiring a Lead Data Engineer to lead the design and delivery of scalable data pipelines and cloud-based transformation frameworks across its Azure data platform. This is a hands‑on senior role for someone who can both build and lead — owning end‑to‑end pipeline architecture, mentoring on best practice, and working directly with business stakeholders to translate requirements into production‑grade data solutions. What you'll be doing: Design and build scalable ETL/ELT pipelines using Azure Data Factory, Databricks, and PySpark Architect and optimise data lake structures, including partitioning strategies for efficient querying at scale Integrate modern data tooling to support incremental updates, real‑time querying, and pipeline orchestration Own CI/CD for data solutions via Azure DevOps, covering ADF, Databricks, and Synapse deployments Partner with business owners and stakeholders on data governance, quality, and compliance Build dashboards and reporting layers in Power BI to support decision‑making Drive reliability improvements across pipeline monitoring, error handling, and retry logic What we're looking for: Strong hands‑on experience with Azure Databricks, Azure Data Factory, and Spark (Python/Scala) Proven experience architecting data lake and warehousing solutions at scale Experience with modern data tooling — DBT, Apache Airflow, Apache Hudi SQL and NoSQL proficiency, with experience across high‑volume, timeseries, or event‑driven data Comfortable operating in a stakeholder‑facing capacity — translating business needs into technical delivery #J-18808-Ljbffr
Vacancy posted 5 hours ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in New York, NY vacancy
- ...Position Overview We are seeking a Lead Data Engineer to join a rapidly growing Data & Analytics organization responsible for delivering enterprise-scale data products, analytics solutions, and cloud-based data platforms. This role will lead the design, development, and...Suggested
Burtch Works
New York, NY1 day ago - ...We are seeking a highly motivated Engineering Consultant with experience in the Pharmaceutical or Life Sciences domain to work on data and analytics-driven initiatives. This role requires a blend of engineering expertise, domain understanding, and consulting capabilities...Suggested
MathCo
New York, NY1 day ago $215.2k - $245.6k
...and disruptors, who solve real problems and meet real customer needs. We are seeking Data Engineers who are passionate about marrying data with emerging technologies. As a Capital One Lead Data Engineer, you’ll have the opportunity to be on the forefront of driving a...SuggestedFull timePart timeInternshipH1bLocal areaCapital One
New York, NY3 days ago- ...Lead Market Data Engineer - Systematic Data Join to apply for the Lead Market Data Engineer - Systematic Data role at Balyasny Asset Management L.P. The Lead Market Data Engineer will architect, build, and optimize the platforms that deliver high‑quality market data to...SuggestedFull time
Balyasny Asset Management
New York, NY22 hours ago - ...Title: AWS Lead Data Engineer Location: Newark, NJ (Hybrid) Duration: Long Term Contract Associate need to go to in-person client interview. Job Summary We are seeking an experienced AWS Lead Data Engineer to join our Data Engineering team. As a technical leader, you...SuggestedLong term contractWork at office
Innovim Technology Solutions
New York, NY4 days ago $180k - $250k
...experience — talk with your recruiter to learn more. Base pay range $180,000.00/yr - $250,000.00/yr A world-leading trading firm is seeking a Lead Data Engineer to architect, scale, and optimize the data infrastructure powering its multi-billion-dollar systematic trading...Full timeEvolve Group, Inc.
New York, NY22 hours ago- ...collaborate directly with the client's team, make business trips.... Digis islooking for anexperienced, proactive, and self-driven Lead Data Engineer tojoin our fully remote team. About the Project You’ll bepartofalarge-scale platform focused on performance management and...Remote work
DIGIS Corporation
New York, NY4 days ago - ...The position will lead technical and business analysis that is critical in discovery process requirement for additional data loads – this would include new and existing datasets supporting the taxonomy for JJMD – (Core & Non-core taxonomy subject areas) Strong collaboration...Work experience placementInterim role
Element Technologies
New York, NY1 day ago $125k - $140k
...– TMX Careers. The TMX group of companies includes leading global exchanges such as the Toronto Stock Exchange... ...lines of business: Indexing, Digital Distribution, and Data & Analytics. Made up of developers, data engineers, designers, and project managers, the platform team...VettaFi
New York, NY5 hours ago- ...A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate will design and maintain scalable data pipelines and collaborate with AI/ML teams. The position requires strong skills in Databricks...Remote work
INSPYR Solutions
New York, NY5 hours ago - ...Type – C2C Job Description We are seeking an experienced Senior Data Engineer to support complex data engineering initiatives within our... ...Data Vault, star and snowflake schemas Demonstrated experience leading or contributing to cloud migration and legacy modernization programs...Work at office
Inizio Partners Corp
Jersey City, NJ5 hours ago $220k - $230k
...Overview Galaxy is a global leader in digital assets and data center infrastructure, delivering solutions that... ...well-tested code, following best practices in software engineering Mentor and develop other engineers, lead code reviews, and raise the bar for engineering...Local areaFlexible hours(Brand Shop) Ttl Trading Private Limited- Thoppumpady
New York, NY2 days ago$140k - $160k
...Fitch Group is currently seeking a Associate Director/Lead Data Engineer based out of our Chicago office. As a leading, global financial information services provider, Fitch Group delivers vital credit and risk insights, robust data, and dynamic tools to champion more...Temporary workWork at officeImmediate startWorldwideShift work2 days per weekFitch Ratings
New York, NY22 hours ago$170k - $220k
...platform. About The Role We are looking for a leader of the Coast data engineering team to shape our company’s data culture and underlying... ...to the Head of Engineering from our NYC office. What You’ll Do Lead design and implementation of all aspects of our data ecosystem...Work at officeLocal areaRelocation packageFlexible hoursBetter Tomorrow Ventures
New York, NY3 days ago- ...Role: US Data Engineer Data Platforms Location : Minneapolis, MN 4 days onsite (Only Local applications will be considered) Required skills: This is a hands‑on delivery lead role specialized in data engineering. Strong communication skill to work with senior executives...Local area
Openkyber
Brooklyn, NY1 day ago $135k - $150k
...New York, United States | Posted on 08/30/2023 The position of Lead Data Engineer is open in New YorkCity, New York, within the Insurance industry. The role offers flexiblework-from-home days while requiring occasional on-site presence for clientmeetings. The Lead Data...Career-Mover
New York, NY5 hours ago- ...W Talent is partnering with a leading organisation within the Financial Services sector to appoint a Lead Data Engineer as part of a significant investment in its Data & Analytics capability. This is an excellent opportunity to join a growing team responsible for delivering...
W Talent
New York, NY22 hours ago - ...We are seeking a Senior Analytic Data Engineer to join our technical team, with primary responsibilities spanning the design, development, and maintenance of our organizational data and programming infrastructure for Hedis. This role is critical for establishing and maintaining...
Openkyber
New York, NY4 days ago $149.3k - $200.2k
...audiences experience sports, entertainment & news. The Product & Data Engineering team is responsible for end to end development for Disney’s... ...personalization, search, messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team, you will be partnering...Work experience placementWorldwideThe Walt Disney Company
New York, NY1 day ago$160k - $220k
...healthcare settings across four continents — and we're hiring the engineering team to build what comes next. About the Role A patient wears... ...all flow into the same clinical picture alongside the wearable data. Your job is to make sure every one of those signals — from raw...Permanent employmentFull timeWork experience placementH1bWorldwideRelocationFlexible hoursShift workNight shiftDay shiftDeliberate Solutions, Inc.
New York, NY1 day ago- ...to ethically democratize the benefits of data and AI . Since 2019, we have built one of... ...‑art personalization and decisioning for leading brands and agencies while protecting privacy... ...and culture from the ground up. Engineers here own major surface areas , build 0→1...Remote workFlexible hours
Yobi AI
New York, NY1 day ago $140k - $150k
...Piper Companies is seeking a Lead Data Engineer to support a company focused on enterprise digital transformation and advanced cloud‑based data modernization initiatives. This position is hybrid in Ft. Washington, PA . The Sr Data Engineer will provide strategic direction...Piper Companies
New York, NY4 days ago- ...Qualifications 10+ years of hands‑on data engineering experience, including technical leadership of data platform initiatives Strong, hands... ...Azure, or GCP) and its core data services Proven experience leading large‑scale data migrations from on‑premise (e.g. Hadoop) and...
Altraize
New York, NY2 days ago - ...publicly traded P&C insurance company and a leading commercial lines insurer in the United... ...and casualty insurance and risk engineering services; affluent and high net worth individuals... ...a highly skilled and experienced lead data engineer to join our dynamic team. The...Work experience placementLocal area
Chubb
Jersey City, NJ22 hours ago - ...position sits within the newly consolidated Data and Analytics (D&A) organization... ...data infrastructure, data governance, data engineering, data modeling, data analysis to business... ...intelligence, data science, and AI. The Lead Data Engineer is crucial to D&A USB’s Eligibility...Temporary workWork at officeLocal area3 days per week
MetLife
New York, NY2 days ago - ...Job Title: Lead Data Engineer Location: Jersey City Type: Contract Key Skills: Snowflake, SQL, Python, Spark, AWS- Glue, Big Data Concepts Responsibilities: ~ Lead the design, development, and implementation of data solutions using AWS and Snowflake...Contract workWork experience placement
VBeyond
Jersey City, NJ4 days ago - ...Lead Data Engineer Locations: Richmond - VA / McLean - VA / Plano - TX / Chicago - IL / NYC - NY / Wilmington - DE (Preferred Location; Hybrid Role; Needs to work 3 days from Office in a week) Job Type: Long Term Contract Responsibilities: Lead the design...Long term contractWork at office
InterSources
New York, NY3 days ago - Job Description Must Have: DBT Core, Snowflake and Python, AWS Airflow, Container services (Docker), Terraform Good to Have: BI tooling experience (Power BI or Tableau), understanding of Agile processes (Kanban or Jira) We need strong person as this position. ...
Echo IT Solutions
New York, NY3 days ago - ..., build, and deliver top-notch technology products. As a Lead Software Engineer at JPMorganChase within the Commercial & Investment Bank (CIB... ...solutions or break down technical problems, with a focus on data engineering and Spark-based ETL/ELT Develops secure high-...
J.P. Morgan
Jersey City, NJ4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
Related searches
- data New York, NY
- data collection researcher New York, NY
- data reviewer New York, NY
- data technician New York, NY
- provider data management New York, NY
- data cabling New York, NY
- data intern New York, NY
- clinical data New York, NY
- voice and data technician New York, NY
- clinical data coordinator remote New York, NY