Big Data Engineer

Purple Drive

New York, NY

Purple Drive

Big Data Engineer

Share New York, NY PDT

Overview:

Coding expert in Java/Python, Hadoop Ecosystem(HDFS), Spark, Hive,

Strong work experience - Agile environment preferred Data Engineer (Big Data – Hadoop, Green Plum, etc., Data Owner) Designs and builds scalable data pipelines, integrates diverse sources, and optimizes storage/processing using Hadoop ecosystem and Greenplum.

Ensures data quality, security, and compliance through governance frameworks.

Implements orchestration, monitoring, and performance tuning for reliable, cost-efficient operations. Expertise in Hadoop ecosystem (HDFS, Hive, Spark, Kafka) and MPP databases like Greenplum for large-scale data processing and optimization. Collaborates with Data Owners and stakeholders to translate business rules into technical solutions. Delivers curated datasets, lineage, and documentation aligned with SLAs and regulatory standards. Subject matter expert having experience of interacting with client, understanding the requirement, and guiding the team. Documenting the requirements clearly with defined scope and must play an anchor role in setting the right expectations and delivering as per the schedule

Apply

Vacancy posted 5 days ago

Similar jobs that could be interesting for youBased on the Big Data Engineer in New York, NY vacancy

Lead Data Engineer
...Lead Data Engineer – Big Data & Cloud (AWS)(Python) Location: New York, NY (Onsite) Contract: 6 Months Visa Status: ONLY GC & USC Strong Python and Big Data expertise JOB DESCRIPTION: We are seeking a Senior Data Engineer / Technology Lead with 10–15...
Big data
Contract work
Argyle Infotech
New York, NY
3 days ago
Lead Data Engineer
...a brighter future and make a meaningful difference. As a Lead Data Engineer at JPMorganChase within the Commercial & Investment Bank Operational... ...optimization and operational managementFamiliar with big data and analytics engines/platforms such as Apache Spark and Hadoop...
Big data
JP Morgan Chase
New York, NY
2 days ago
Data Engineer
...Description Bustle Digital Group (BDG) is looking for a talented Data Engineer to join our growing data team. You will work alongside the... ...experience with Python for scripting and automation. Cloud & Big Data Savvy: Hands-on experience with cloud platforms (AWS, GCP...
Big data
Work at office
Remote work
Flexible hours
BDG
New York, NY
a month ago
Data Engineer III
$166k - $205k
...else can. About the role: We are seeking an exceptional Data Engineer III to join our Engineering team, with a high sense of ownership... ...distributed data processing, data warehousing, data governance, Big Data, data variance, data privacy, and data quality). ~...
Big data
Work at office
Work visa
Flexible hours
3 days per week
Garner Health
New York, NY
29 days ago
Sr. Data Engineer
...: Business Intelligence CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence... ...to manipulate data (Python, R, SQL, SAS) Strong knowledge of big data analysis and storage tools and technologies Strong...
Big data
Full time
Flexible hours
CGS Federal (Contact Government Services)
New York, NY
4 days ago
Sr. Data Engineer
$115k - $145k
...embrace authenticity and inspire people to thrive. As our Senior Data Engineer, you’ll join the Content Delivery Engineering team (CDE) within... ...requirements including KPI and acceptance criteria Use big data, relational and non-relational data sources, to access data...
Big data
Full time
Work at office
Local area
NBCUniversal
New York, NY
11 days ago
Lead Data Engineer
...a brighter future and make a meaningful difference. As a Lead Data Engineer at JPMorganChase within the Commercial & Investment Bank Operational... ...optimization and operational managementFamiliar with big data and analytics engines/platforms such as Apache Spark and Hadoop...
Big data
JP Morgan Chase
New York, NY
2 days ago
Senior Azure Databricks Data Engineer
...SummaryWe are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in... ...can design, develop, and optimize scalable data platforms and analytics solutions using the... ...cloud technologies preferably Azure, and big data processing to support enterprise analytics...
Big data
EXL Service
New York, NY
3 days ago
Python - Big Data Engineer
...Job Title: Lead Consultant | Analytics - Packages | Python - Big Data Work Location: New YorkNY10004(Prefer local candidate due to... ...experience working in Python, Spark ETL projects Build the AI query engine on Cortex Analyst: map business questions to certified semantic...
Big data
Local area
Tech3pillars Technologies
New York, NY
2 days ago
Lead Data Engineer
$215.2k - $245.6k
...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast... ...iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers, doers and disruptors, who solve real...
Big data
Full time
Part time
Internship
H1b
Local area
Capital One National Association
New York, NY
4 days ago
Senior Data Engineer
$10k
...a relationship, for all. About the Role: As a Senior Data Engineer at Hinge, you will create essential data processes and contribute... ...the rest of the organization. You will be working on an actual big data architecture while concentrating on real-world problems....
Big data
Full time
Work experience placement
Hinge
New York, NY
29 days ago
Data Engineer - Manager - Consulting - Location OPEN
$125.5k - $230.2k
...intelligent digital future. Our elite team of product leaders, data scientists, designers, and software engineers enable our clients to solve their most complex... ..., data pipelines, streaming architectures, and big data ecosystems. Cloud Computing Architecture...
Big data
Summer holiday
Flexible hours
Ernst & Young
New York, NY
a month ago
Azure Data Engineer
...degree in computer science, Information Technology, Engineering, or a related discipline, with 6-10 years of experience in Data Engineering, Application Development, System... ...to stay current with evolving cloud and big data technologies and contribute effectively to...
Big data
TechWish
New York, NY
3 days ago
Senior Manager, Data Engineering
$121.5k - $200k
...more than 60 countries. We are seeking an experienced Senior Data Engineering Manager – Enterprise Data Products to join our Global Data & Analytics... ...~ Experience with cloud ecosystems (AWS, Azure or GCP) and big data technologies (Spark, Snowflake, Databricks, etc.). GCP is...
Big data
Summer work
Immediate start
Flexible hours
600 UMG Recordings Inc
New York, NY
1 day ago
Senior Quantitative Data Engineer
$200k - $220k
...force behind Schonfeld’s next-generation data platforms, you will architect, operate, and... ...with quants, data scientists, and fellow engineers to achieve sub-minute SLAs, shape firm-... ...with Python and SQL ~ Deep expertise in big-data architecture: partitioning, sharding...
Big data
Full time
Schonfeld
New York, NY
10 days ago
Staff / Principal Data Engineer
$180 per hour
...the Role We are building an AI-native data platform that powers fraud detection and response... .... We are hiring a Staff or Principal Data Engineer to own the data platform and data lake at... ...Apache Spark, Apache Flink and modern big-data systems. Proven command of best...
Big data
For contractors
Work at office
3 days per week
AppGate Cybersecurity, Inc.
New York, NY
a month ago
Big Data Engineer Lead
$149.3k - $202.27k
...contributor within the Ratings Organization's Data Services Team. Our team brings deep... ...Collaborate with data scientists and software engineers to design, build, and deploy scalable... ...incentive awards—small perks can make a big difference. For more information on benefits...
Big data
Full time
Second job
Live in
Worldwide
Flexible hours
S&P Global
New York, NY
3 days ago
Data Architecture & Engineering - Manager - Financial Services - Consulting - NYC
$125.5k - $230.2k
...better working world. Manager – Financial Services – Data and Analytics – Data Architecture and Engineering EY is the only professional services firm with... ...and Engineering, Cloud Data Modernization, Big Data initiatives and Event-driven architecture, preferably...
Big data
Work experience placement
Summer holiday
Flexible hours
Ernst & Young
New York, NY
more than 2 months ago
Senior Software Engineer - Data
...define the future of finance? Are you a strong software engineer who's passionate about data, distributed systems, and delivering impact at scale? Are... ...in a cloud-based environment. Hands-on experience with big data frameworks such as Spark or Databricks. Knowledge...
Big data
Full time
Remote work
Home office
Elliptic
New York, NY
4 days ago
Big Data Engineer - Remote
...Job Description Job Description Job Title: Big Data Engineer Job Type: Contractor Location: Remote Job Summary: In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform...
Big data
Remote job
For contractors
YO AI Labs
New York, NY
17 days ago
AWS Data Engineer
...Description Role Overview We are seeking a skilled AWS Data Engineer to design, build, and optimize scalable data pipelines and architectures... .... Solid understanding of data modeling, warehousing, and big data concepts. Familiarity with distributed processing...
Big data
Permanent employment
Full time
Local area
Capgemini
New York, NY
13 days ago
AWS Data Engineer
...inclusive world. Job Description We are seeking a skilled AWS Data Engineer to design, build, and optimize scalable data pipelines and... ...frameworks Solid understanding of data modeling, warehousing, and big data concepts Familiarity with distributed processing...
Big data
Permanent employment
Full time
Local area
Capgemini
New York, NY
13 days ago
Software Engineer, Data Platform (All Teams)
About the TeamDoorDash is a data driven organization and relies on timely, accurate and... ...and tools that simplify data workflows for engineers, analysts, and ML practitioners. In... ...releasing softwareExperience working on big data domains (streaming, batch, data lake...
Big data
Hourly pay
Work at office
Local area
Remote work
Relocation
Flexible hours
Doordash
New York, NY
4 hours ago
Data Scientist III
...Job Responsibilities: Experienced in either programming languages such as Python and/or R, big data tools such as Hadoop, or data visualization tools such as Tableau. The ability to communicate effectively in writing, including conveying complex information and...
Big data
SGS Consulting
New York, NY
more than 2 months ago
Senior Data Engineer, Fleet Monitoring & Analysis
$153k - $204k
...and high-reliability operations. About the role: As a Senior Data Engineer you will own and evolve the data lake and analytics stack that... ...data pipeline orchestration tools (e.g., Apache Airflow) and big data technologies (e.g., Apache Spark). ~ Experience designing...
Big data
Permanent employment
Full time
Temporary work
Casual work
Work at office
Flexible hours
CoreWeave
New York, NY
26 days ago
Back-end Engineer - Data Platforms
$155k - $215k
...markets and shape the future of our communities. This is a Back-end Engineer - Data Platforms position at the Vice President level, which is part... .... This role requires strong expertise in Python, PySpark, Big Data technologies, Snowflake, Graph Technologies, System...
Big data
Full time
Temporary work
Morgan Stanley
New York, NY
9 days ago
Data Engineer
...TimeWorking Type On SiteJob Reference 0000017586Salary Type AnnuallyIndustry Hedge Fund;Private EquitySelling Points Drive impactful data engineering initiatives in a fast-paced financial environment. Collaborate on innovative projects utilizing cutting-edge technologies....
Green Key Resources
New York, NY
3 days ago
Senior Data Engineer I - QuantumBlack, AI by McKinsey (Critical Industries)
...technical owner, designing and leading the implementation of scalable data architectures for cutting-edge AI and agentic systems. You will... .... You will also spearhead the design of complex feature engineering workstreams, ensuring our data assets are not only robust but also...
Apprenticeship
Work at office
Local area
Easy work
McKinsey & Company
New York, NY
23 hours ago
Senior Data Scientist
$175.5k - $180k
...sitting on enormous clinical and operational data assets, but turning those assets into... ...with QuantumBlack Labs data scientists, ML engineers, and product managers, as well as Social,... ...SQL; experience with PySpark or another big-data framework is a plusExperience with...
Big data
Apprenticeship
Easy work
McKinsey & Company
New York, NY
23 hours ago
Data Scientist
$80 - $95 per hour
Job Overview We're seeking an experienced Data Scientist to join our team. As a Data Scientist, you will be responsible for analyzing... ...data visualization tools (Tableau, Power BI) · Experience with big data technologies (Hadoop, Spark) · Strong understanding of data...
Big data
Hourly pay
Full time
Remote work
Flexible hours
Franklin Barbecue
New York, NY
6 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!