Data Engineer / Lead Data Engineer
Merican Inc
Job Title: Senior Data Engineer / Lead Data Engineer
Location: Hybrid , NJ
Employment Type: Full-Time
NEED LOCAL CANDIDATE, NY OR NJ
EXP 12+
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for large-volume structured and unstructured data.
- Develop data processing solutions using Python, PySpark, Apache Spark, and SQL .
- Build and optimize data pipelines using Databricks, Azure Data Factory, AWS Glue, and Snowflake .
- Work with both Azure and AWS cloud platforms to implement modern data engineering solutions.
- Work with Delta Lake, Azure Data Lake, Amazon S3, Azure Blob Storage, Azure Synapse, and Snowflake .
- Develop pipeline orchestration and scheduling using Airflow, Control-M, Databricks Workflows, Azure Data Factory, and AWS services .
- Implement monitoring, logging, alerting, and troubleshooting processes for production data pipelines.
- Work with CI/CD processes using GitHub, Bitbucket, Jenkins, Docker, and Azure DevOps .
Required Skills
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Python, PySpark, Apache Spark, and SQL .
- Strong experience with Azure and/or AWS cloud environments .
- Experience with Databricks and Delta Lake .
- Strong knowledge of Azure Data Factory, Azure Synapse, Azure Data Lake/Blob Storage .
- Experience with AWS services such as S3, Glue, Athena, DMS, Lambda, SNS, SQS, and EventBridge .
- Strong experience with Snowflake and cloud data warehousing.
- Strong understanding of ETL/ELT, data warehousing, data modeling, and data integration .
- Experience working with Oracle, SQL Server, MySQL, Hive, and other relational databases .
- Strong programming and scripting experience using Python, SQL, and Shell scripting .
- Experience with Git, Bitbucket/GitHub, Jenkins, Docker, and Azure DevOps .
- .
Preferred Skills
- Experience with Kafka and real-time streaming .
- Experience with Airflow and Control-M .
- Knowledge of MLOps and machine-learning data pipelines .
- Experience with Kubernetes and containerized data workloads .
- Experience with API data integration and modernization .
- Knowledge of LLM/AI-assisted data engineering solutions .
- Experience with Power BI or Tableau for data reporting and analytics.
- Experience with data migration and reverse engineering of legacy data models.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Engineer / Lead Data Engineer in New York, NY vacancy
- ...and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference. As a Lead Data Engineer at JPMorganChase within the Commercial & Investment Bank Operational Resiliency team, you are an integral part of an agile team...Suggested
JP Morgan Chase
New York, NY3 days ago $137.7k - $229.5k
Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make a significant... ..., and fuel growth through innovation.Work You’ll DoAs a Lead Data Engineer II on the team, you will be responsible for:Support...SuggestedLocal areaDeloitte
New York, NY1 day ago- ...assets including Fandango, Rotten Tomatoes, GolfNow and GolfPass. Job Description We are seeking a hands-on Data Product Engineering Lead to lead engineering resources to deliver trusted, scalable, and reusable data products from source systems through the Silver...SuggestedFull timeLocal area
Versant Media
New York, NY7 days ago $135k - $180k
...million patients, Medidata offers industry-leading expertise, analytics-powered insights, and one of the largest clinical trial data sets in the industry. More than 1 million... ...including product, delivery, data science, engineering, partnerships, and biostatistics....SuggestedOngoing contractWork at officeLocal areaFlexible hoursDassault Systèmes
New York, NYa month ago- ...thinking about — in real time, at scale, across every screen. Our data exists with the consent of over a billion people, organized... ...it’s the most culturally relevant. We are looking for a Lead Data Engineer to join our Revenue Operations team.This role sits at the...Suggested
Samba
New York, NY12 days ago $61k - $101k
...Requirements: At least 5 years of relevant experience in data engineering, analytics engineering, or data platform engineering, with delivery... ...Spark TOGAF microservices More: We are hiring a Lead Data Engineer for our Commercial & Investment Bank Operational...Full timeJ.P. Morgan
New York, NY2 days ago- ...Lead Data Engineer – Big Data & Cloud (AWS)(Python) Location: New York, NY (Onsite) Contract: 6 Months Visa Status: ONLY GC & USC Strong Python and Big Data expertise JOB DESCRIPTION: We are seeking a Senior Data Engineer / Technology Lead with 10–15...Contract work
Argyle Infotech
New York, NY4 days ago $197.3k - $225.1k
...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers...Full timePart timeInternshipH1bLocal areaCapital One
New York, NY2 days ago$170k - $220k
...Lead Data Engineer New York, NY What Is Coast Coast is re-imagining the trillion-dollar U.S. B2B card payments infrastructure, with a focus on the country's 500,000 commercial fleets, 40 million commercial vehicles, and many millions of commercial drivers. The...Work at officeLocal areaRelocation packageFlexible hoursCoast
New York, NY3 days ago- Lead Data Engineer Locations: Richmond - VA / McLean - VA / Plano - TX / Chicago - IL / NYC - NY / Wilmington - DE (Preferred Location; Hybrid Role; Needs to work 3 days from Office in a week) Job Type: Long Term Contract Responsibilities: Lead the design, development,...Long term contractWork at office
$147.25k - $190k
...and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference. As a Lead Data Engineer at JPMorganChase within the Commercial & Investment Bank Operational Resiliency team, you are an integral part of an agile team...- Summary We are looking for a Lead Data Engineer who can own more than pipelines. Beyond building and maintaining ETL and integrations, you'll own and implement data engineering best practices that make a platform trustworthy at scale: orchestration, observability, lineage...Contract workInterim roleFlexible hours
$115k - $150k
...see the difference our hard work makes, and continue on our own paths of lifelong learning. How can you make an impact? The Lead Data Engineer in Data and Analytics is responsible for enhancing McGraw-Hill Education’s (MHE) business intelligence and data services capabilities...Remote workWorldwide$135k - $150k
New York, United States | Posted on 08/30/2023 The position of Lead Data Engineer is open in New YorkCity, New York, within the Insurance industry. The role offers flexiblework-from-home days while requiring occasional on-site presence for clientmeetings. The Lead Data...- ...work to create a thriving ecosystem that delivers accessible, high-quality, and sustainable healthcare for all. Business Title Lead Data Engineer, Data Imports and Migrations Role summary: You will help move healthcare data into athenahealth systems with accuracy,...Full timeTemporary workWork at officeRemote work
$120.9k - $187k
...together. Make it better. We create fulfilling purpose-driven careers by learning from the world and each other. POSITION TITLE Lead Data Engineer POSITION LOCATION This position is available to Virginia residents as Richmond or Lynchburg, Virginia in-office applicants *A...Work at officeLocal areaRemote work- Providing technical leadership for data portfolios, the full-time Lead Data Engineer will drive the modernization of data ecosystems, overseeing the design and implementation of scalable data solutions while collaborating with various stakeholders in a remote work environment...Full timeWork experience placementRemote work
$109.5k - $167.83k
...company, you’ll be helping provide protection and peace of mind when our customers need it most. Protective is looking for a Lead Data Engineer to set the technical direction for a delivery pod building data products on Voyager, our Databricks lakehouse on Azure. You will...Contract workTemporary work- A technology solutions firm is looking for a Lead Data Engineer to work remotely on building end-to-end data solutions. The ideal candidate will design and maintain scalable data pipelines and collaborate with AI/ML teams. The position requires strong skills in Databricks...Remote job
- ...team - a group of collaborative and highly skilled technology professionals, all of whom are leading experts in real estate data, data science, and technology. WDTech Engineering builds solutions that impact not only our products but also the people and processes across...Contract workTemporary workWork experience placementLocal areaRemote workFlexible hours
$175k - $225k
A leading global investment management firm is hiring a hands-on Senior/Lead Data Engineer to provide technical leadership for the platforms, pipelines, and data products that power analytics, applications, and investment decision-making across the firm. You'll architect...- Lead Data Engineer Join Brooksource as a Lead Engineer, where you will combine hands-on application development with technical leadership and solution design influence. In this fully remote role, you will support end-to-end delivery across SaaS and custom applications,...Remote work
- Description The Lead Data Engineer owns Awayday's ingestion and pipeline layer end to end, from each source system through to the landing and staging layers in Snowflake that every domain builds on. It is a foundational mandate. You will build and run the pipelines that...Contract workFor contractors
- Lead Data Engineer Are you an experienced Lead Data Engineer looking for the right company to call "home"? Look no further! Our client is seeking an experienced Lead Data Engineer, ready to bring their skills, experience, and energy to a great, growing team. What you...Contract work
- Senior/Lead Data Engineer Remote - CST Hours, Within US only Max Vendor Rate - *** Senior/Lead Data Engineer Position Summary Seeking a hands-on Senior/Lead Data Engineer with 8+ years of experience to design, build, and support scalable cloud data solutions for healthcare...Remote work
- ...Consulting firm, delivers intelligent solutions across AI, data, analytics, and product engineering. Since 2000, we have partnered with some of the largest... ..., we have an exciting opportunity for a highly skilled Lead Data Engineer to join our innovative and dynamic team....Worldwide
- BDIPlus is seeking a Senior Data Engineer to support a Fortune 100 financial services client’s real-time intelligence initiatives. In this role, You’ll work in a virtualization-first architecture using Denodo , building both virtual and physical data products that are governed...
$180k - $250k
...experience — talk with your recruiter to learn more. Base pay range $180,000.00/yr - $250,000.00/yr A world-leading trading firm is seeking a Lead Data Engineer to architect, scale, and optimize the data infrastructure powering its multi-billion-dollar systematic trading...Full time- ...Logicbroker, Inc. Logicbroker is the Agentic Commerce Orchestration Engine helping enterprise retailers, brands, suppliers, and... ...customers, and run smarter operations. Job Summary We are hiring a Lead Data Engineer to drive delivery across our analytics and data platform...For contractorsWork at officeRemote work
$142k - $166k
...MA, or the Brooklyn, NY office. Hybrid work expectations are subject to business needs and company policy. Job Purpose The Lead Data Engineer is a collaborative, hands-on role within the Digital Platform Data Engineering team. This individual is responsible for designing...Work at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer / Lead Data Engineer. Be the first to apply!
Related searches
- senior data center engineer New York, NY
- senior cloud data engineer New York, NY
- senior data integration developer New York, NY
- data developer New York, NY
- entry level data engineer New York, NY
- data engineer New York, NY
- big data cloud engineer New York, NY
- data platform engineer New York, NY
- data engineer intern New York, NY
- data engineering intern summer New York, NY