Data Engineer / Lead Data Engineer
Merican Inc
Job Title: Senior Data Engineer / Lead Data Engineer
Location: Hybrid , NJ
Employment Type: Full-Time
NEED LOCAL CANDIDATE, NY OR NJ
EXP 12+
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for large-volume structured and unstructured data.
- Develop data processing solutions using Python, PySpark, Apache Spark, and SQL .
- Build and optimize data pipelines using Databricks, Azure Data Factory, AWS Glue, and Snowflake .
- Work with both Azure and AWS cloud platforms to implement modern data engineering solutions.
- Work with Delta Lake, Azure Data Lake, Amazon S3, Azure Blob Storage, Azure Synapse, and Snowflake .
- Develop pipeline orchestration and scheduling using Airflow, Control-M, Databricks Workflows, Azure Data Factory, and AWS services .
- Implement monitoring, logging, alerting, and troubleshooting processes for production data pipelines.
- Work with CI/CD processes using GitHub, Bitbucket, Jenkins, Docker, and Azure DevOps .
Required Skills
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Python, PySpark, Apache Spark, and SQL .
- Strong experience with Azure and/or AWS cloud environments .
- Experience with Databricks and Delta Lake .
- Strong knowledge of Azure Data Factory, Azure Synapse, Azure Data Lake/Blob Storage .
- Experience with AWS services such as S3, Glue, Athena, DMS, Lambda, SNS, SQS, and EventBridge .
- Strong experience with Snowflake and cloud data warehousing.
- Strong understanding of ETL/ELT, data warehousing, data modeling, and data integration .
- Experience working with Oracle, SQL Server, MySQL, Hive, and other relational databases .
- Strong programming and scripting experience using Python, SQL, and Shell scripting .
- Experience with Git, Bitbucket/GitHub, Jenkins, Docker, and Azure DevOps .
- .
Preferred Skills
- Experience with Kafka and real-time streaming .
- Experience with Airflow and Control-M .
- Knowledge of MLOps and machine-learning data pipelines .
- Experience with Kubernetes and containerized data workloads .
- Experience with API data integration and modernization .
- Knowledge of LLM/AI-assisted data engineering solutions .
- Experience with Power BI or Tableau for data reporting and analytics.
- Experience with data migration and reverse engineering of legacy data models.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Engineer / Lead Data Engineer in New York, NY vacancy
$137.7k - $229.5k
Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make a significant... ..., and fuel growth through innovation.Work You’ll DoAs a Lead Data Engineer II on the team, you will be responsible for:Support...SuggestedLocal areaDeloitte
New York, NY4 days ago- ...Lead Data EngineerLocations: Richmond - VA / McLean - VA/ Plano - TX / Chicago - IL / NYC - NY / Wilmington - DE (Preferred Location;... ...Lead the design, development, and deployment of scalable data engineering solutions that deliver operational and analytical data to third...SuggestedWork at office
InterSources
New York, NY4 days ago - ...Summary We are looking for a Lead Data Engineer who can own more than pipelines. Beyond building and maintaining ETL and integrations, you'll own and implement data engineering best practices that make a platform trustworthy at scale: orchestration, observability,...SuggestedContract workInterim roleFlexible hours
K2 Integrity
New York, NY1 day ago $94k - $152.66k
...Job Description We’re looking for an Identity Data Engineer who is passionate about data quality, intellectually curious about how real-world identities get resolved, and ready to get deep into the details. You'll work directly with PII-class data at a low level — examining...SuggestedPermanent employmentFull timeContract workTemporary workWork at officeLocal areaRemote workFlexible hours2 days per week3 days per weekDentsu International Americas, LLC
New York, NY1 day ago$175k - $225k
...A leading global investment management firm is hiring a hands-on Senior/Lead Data Engineer to provide technical leadership for the platforms, pipelines, and data products that power analytics, applications, and investment decision-making across the firm. You'll architect...SuggestedLong Ridge Partners
New York, NY2 days ago- ...BDIPlus is seeking a Senior Data Engineer to support a Fortune 100 financial services client’s real-time intelligence initiatives. In this role, You’ll work in a virtualization-first architecture using Denodo , building both virtual and physical data products that...
BDIPlus
New York, NY4 days ago - ...Logicbroker is the Agentic Commerce Orchestration Engine helping enterprise retailers, brands,... ...Job Summary We are hiring aLead Data Engineerto drive delivery across our analytics... ...initiatives. This role is a hands-on technical lead responsible for translating well-defined...For contractorsWork at officeRemote work
Logicbroker
New York, NY3 days ago $142k - $166k
...or the Brooklyn, NY office. Hybrid work expectations are subject to business needs and company policy. Job Purpose The Lead Data Engineer is a collaborative, hands-on role within the Digital Platform Data Engineering team. This individual is responsible for designing...Work at officeLocal areaNational Grid plc
New York, NY1 day ago$170k - $220k
...About the Role We are looking for a leader of the Coast data engineering team to shape our company's data culture and underlying infrastructure... ...our modern, cloud-native data platform. This is a chance to lead a small, high-leverage team and make a definitive impact,...Work at officeLocal areaRelocation packageFlexible hoursBetter Tomorrow Ventures
New York, NY1 day ago$170k - $220k
...with the Coast Fleet and Fuel Card built on a cutting-edge spend management platform. About the Role We’re looking for a Lead Data Engineer who can not just get their hands dirty building pipelines and also act as a thought leader for the company. Data sits at the...Work at officeLocal areaFlexible hours3 days per week- ...thinking about — in real time, at scale, across every screen. Our data exists with the consent of over a billion people, organized... ...it’s the most culturally relevant. We are looking for a Lead Data Engineer to join our Revenue Operations team.This role sits at the...
Samba
New York, NY15 days ago $135k - $150k
New York, United States | Posted on 08/30/2023 The position of Lead Data Engineer is open in New YorkCity, New York, within the Insurance industry. The role offers flexiblework-from-home days while requiring occasional on-site presence for clientmeetings. The Lead Data...- Job Title:Lead Data Engineer - Big Data & Cloud (AWS)(Python) Location: New York, NY (Onsite) Contract: 6 Months Visa Status: ONLY GC & USC Strong Python and Big Data expertise JOB DESCRIPTION: We are seeking a Senior Data Engineer / Technology Lead with 10-15 years...Contract work
$147.25k - $190k
...and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference. As a Lead Data Engineer at JPMorganChase within the Commercial & Investment Bank Operational Resiliency team, you are an integral part of an agile team...- A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate will design and maintain scalable data pipelines and collaborate with AI/ML teams. The position requires strong skills in Databricks...Remote job
$70 - $85 per hour
Lead Data Engineer Location: Remote Duration: 6+ months Compensation: $70.00 - 85.00/hr Work Requirements: US Citizen, GC Holders or Authorized to Work in the U.S. Summary We are looking for a Lead Data Engineer who excels in data modeling and can build end-to-end data...Contract workLocal areaRemote workFlexible hours- Lead AWS Data Engineer Contract-to-hire 100% Remote Profile Summary: Lead Data and Analytics Engineer will be a senior member of data and analytics engineering chapter. This is a hands-on technology engineering leadership role focused on building AWS data & analytics engineering...Contract workRemote work
$148.14k - $214.76k
...retirement package to secure your future. Responsibilities Lead data design, prototype, and development of data pipeline architecture... .... Be a senior Individual contributor of the Data or Software Engineering teams. Be part of Technical Review Board along with Manager and...Remote jobFull timeFlexible hoursWeekend work$149.3k - $200.2k
...audiences experience sports, entertainment & news. The Product & Data Engineering team is responsible for end to end development for Disney’s... ...personalization, search, messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team, you will be partnering...Work experience placementWorldwide$111.8k
Overview Lead Data Engineer | Build the Future of Data at GATX Remote (Chicago-area preferred) Founded in 1898 and headquartered in Chicago, IL, GATX Corporation (NYSE: GATX) is an industry leader with 125+ years of success - success that is powered by our people. We...Temporary workWork at officeRemote work- We are seeking a highly motivated Engineering Consultant with experience in the Pharmaceutical or Life Sciences domain to work on data and analytics-driven initiatives. This role requires a blend of engineering expertise, domain understanding, and consulting capabilities...
- Lead Market Data Engineer - Systematic Data Join to apply for the Lead Market Data Engineer - Systematic Data role at Balyasny Asset Management L.P. The Lead Market Data Engineer will architect, build, and optimize the platforms that deliver high‑quality market data to...Full time
- Lead Data Engineer Blend is a premier AI services provider, committed to co-creating meaningful impact for its clients through the power of data science, AI, technology, and people. With a mission to fuel bold visions, Blend tackles significant challenges by seamlessly...Flexible hours
- ...position sits within the newly consolidated Data and Analytics (D&A) organization... ...data infrastructure, data governance, data engineering, data modeling, data analysis to business... ...intelligence, data science, and AI. The Lead Data Engineer is crucial to D&A USB’s Eligibility...Temporary workWork at officeLocal area3 days per week
- Lead Data Privacy Engineer We're building a world of health around every individual — shaping a more connected, convenient and compassionate health experience. At CVS Health®, you'll be surrounded by passionate colleagues who care deeply, innovate with purpose, hold ourselves...Full timeWork experience placementLocal area
- Senior / Lead Data Engineer We are seeking a hands-on Senior / Lead Data Engineer to provide technical leadership for the platforms, pipelines, and data products that power analytics, applications, and investment decision-making across the firm. You will architect scalable...
- Job Title: Lead Data Engineer Location: Dallas, TX Client: Prodapt/ AT&T Project Duration: 12 months Job Type: Contract Work Arrangement: Onsite role - 5 days a week Interview: In person Job Summary We are seeking a highly skilled Technical Lead - ETL & Python...Contract work
- Lead Data Engineer Are you an experienced Lead Data Engineer looking for the right company to call "home"? Look no further! Our client is seeking an experienced Lead Data Engineer, ready to bring their skills, experience, and energy to a great, growing team. What you...Contract work
- Description The Lead Data Engineer owns Awayday's ingestion and pipeline layer end to end, from each source system through to the landing and staging layers in Snowflake that every domain builds on. It is a foundational mandate. You will build and run the pipelines that...Contract workFor contractors
- Lead Data Engineer Join Brooksource as a Lead Engineer, where you will combine hands-on application development with technical leadership and solution design influence. In this fully remote role, you will support end-to-end delivery across SaaS and custom applications,...Remote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer / Lead Data Engineer. Be the first to apply!
Related searches
- senior data center engineer New York, NY
- senior cloud data engineer New York, NY
- senior data integration developer New York, NY
- data developer New York, NY
- entry level data engineer New York, NY
- data engineer New York, NY
- big data cloud engineer New York, NY
- data platform engineer New York, NY
- data engineer intern New York, NY
- data engineering intern summer New York, NY