Senior Big Data Engineer
Veriipro
We are seeking a Senior Big Data Engineer with 8+ years of IT experience and strong expertise in Hadoop, Spark, Python, and Snowflake. The ideal candidate will have hands‑on experience building scalable data pipelines, migrating data from Hadoop to Snowflake using PySpark, and developing enterprise ETL solutions. Roles and Responsibilities Design, develop, and maintain scalable data pipelines using Hadoop, Spark, Python, and Snowflake. Perform data migration from Hadoop to Snowflake using PySpark frameworks and libraries. Build and optimize ETL workflows for large-scale data ingestion, transformation, and validation. Develop and optimize complex SQL queries for data processing and analytics. Ensure data quality, performance, and reliability across enterprise data platforms. Collaborate with cross‑functional teams to gather requirements, troubleshoot issues, and deliver data solutions. Required Qualifications 8+ years of overall IT experience in data engineering or big data development. Strong hands‑on experience with Hadoop, Apache Spark, Python, and Snowflake. Experience migrating enterprise data from Hadoop to Snowflake using PySpark. Solid understanding of ETL processes, data integration, and data transformation techniques. Strong SQL skills with experience writing and optimizing complex queries. Excellent analytical, problem‑solving, and communication skills. Experience working in Unix/Linux environments. (Preferred) #J-18808-Ljbffr Veriipro
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Senior Big Data Engineer in New York, NY vacancy
$250k - $300k
...deep fundamental research and applying data science and engineering techniques to alternative data sets.... ...structured and unstructured data sets. As a senior member of our Data Engineering team,... ...role, you will:Develop and support big data processing pipelines including but...SeniorBig dataWork experience placement- ...Senior Analytics Engineer As a Senior Analytics Engineer, you will support the BI lead data pipeline and ecosystem from end to end. You will be partnering with multiple cross... ...Director, Business Intelligence. Required Skills: SQL Additional Skills: Big Data...SeniorBig data
Samprasoft
New York, NY5 days ago - ...candidate is an inquisitive learner with a passion for security, data and analytics.The candidate must effectively collaborate with stakeholders... ...) as well as advanced designer experience in encapsulation of big data displayed via visualization toolkits aimed to drive action...SeniorBig data
- ...Position Summary Lead complex data engineering projects; design data architectures aligned with business requirements; optimize data workflows... ...with Python scripting and PySpark. Proficiency in SQL and big data processing. Experience with data lake processing in Iceberg...SeniorBig data
Saransh
New York, NY2 days ago - A leading performance marketing engine in New York is seeking a Senior Data Engineer to contribute to the development of Micro Batch/Real Time streaming pipelines... ...have a degree in Computer Science and experience with big data tools like Bigtable and Apache Spark. This...SeniorBig data
$60 - $70 per hour
Sciata is looking for a Data Engineer in Atlanta, GA. You will design, implement, and maintain the data pipeline architecture for our business... ...in data engineering, strong SQL knowledge, and is skilled with big data tools like Hadoop and Spark. This role involves...SeniorBig dataHourly pay- Vortalsoft Inc is seeking a skilled data engineer to develop and maintain data pipelines for their Big Data financial crime solution. The ideal candidate will have at least 4 years of experience in Big Data solutions and strong skills in programming, data modeling, and...SeniorBig data
£75k per year
...Data Engineering Lead - AI focused - required for an AI-First SaaS business, offering Remote Working, paying up to circa £75K + Benefits Working... ...and impact in the business; an ideal role for an experienced Senior or Lead Data Engineer who wants to work on far more than just...SeniorBig dataFull timeRemote workApply Recruitment
New York, NY1 day ago- Overview Job Title: Senior Data Engineer Department: DigitalReporting To: Manager, Data EngineeringEmployment Type: Full-TimeOverview:Audacy... ...platform, you’ll play a key role in shaping how Audacy utilizes big data on the magnitude of billions of rows to unlock insights,...SeniorBig dataFull timeLocal area
$165k - $230k
...Position Summary We're hiring a Senior Data Engineer to own data at truly massive scale. You'll design and run pipelines that clean, enrich... ...data partnerships. Nice to have ~ Experience with big data, columnar storage formats, vector indexes, and privacy/compliance...SeniorBig dataPermanent employmentWork at officeMonday to FridayRaylu, Inc
New York, NY5 days ago- Veriipro in New York City is seeking a Senior Big Data Engineer with 8+ years of IT experience and hands-on expertise in Hadoop, Spark, Python, and Snowflake. You will design scalable data pipelines, migrate data from Hadoop to Snowflake using PySpark, and develop enterprise...SeniorBig data
$140k - $175k
...talented people in the industry. Rockstar Games is on the lookout for a skilled Senior Analytics Engineer with strong software development skills who is passionate about games and big data. This is a full-time, in-office position based out of Rockstar’s NYC headquarters...SeniorBig dataFull timeWork at office- Mount Sinai Health System is seeking an experienced Information Technology Specialist II (Data Architect) to lead data collection, movement, storage, and transformation for Big Data initiatives. The role covers ETL, data warehousing, and upcoming streaming/pipeline architectures...SeniorBig data
- ...Senior Data EngineerWe are looking for a Senior Data Engineer to help design, build, and operate the core data platform that powers analytics, machine learning, and... ...Redshift, BigQuery, or Snowflake.Familiarity with big data or distributed processing frameworks such as Spark...SeniorBig dataWork at office
Nelo Mobile
New York, NY9 hours ago - A technology solutions company is seeking a skilled Java developer in New York to implement solutions for large data processing using NoSQL and Hadoop. The ideal candidate will have strong server-side Java experience and be involved in architectural discussions. Responsibilities...SeniorBig data
$215.2k - $244.7k
...make an impact?West Monroe is seeking a Senior Architect to join our thriving Technology... ...to work with clients developing Big Data strategy and roadmaps, while gaining exposure... ...with Large Language Models (LLMs), prompt engineering, embeddings, vector databases, and Retrieval...SeniorBig dataLocal areaImmediate startFlexible hours- ..., we'd love to meet you! ABOUT THE ROLE We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse... ..., Polars , PySpark or DuckDB ; 2+ years of experience with Big Data technologies ( Spark , Snowflake ); Expert-level...SeniorBig dataWork at officeRemote workVisa sponsorshipWork visaFlexible hours
AgileEngine
New York, NY3 days ago - Mount Sinai Health System in New York, NY, is seeking a senior data engineer to design scalable data pipelines and architectures in a hybrid/remote... ...and govern large datasets, leveraging Azure, Databricks, and big data tools to support research initiatives. #J-18808-Ljbffr...SeniorBig dataRemote job
$153k - $204k
...provisioning and high-reliability operations.About the role:As a Senior Data Engineer you will own and evolve the data lake and analytics stack... ...data pipeline orchestration tools (e.g., Apache Airflow) and big data technologies (e.g., Apache Spark).Experience designing,...SeniorBig dataPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hours- ...technology company in New Jersey is looking for an experienced Data Engineer to build and maintain scalable data pipelines. Candidates should... ...experience in Python and AWS, with a strong understanding of big data technologies. The role involves collaboration with cross-functional...SeniorBig data
$160k - $220k
..., and problem solvers to help us create it. Who you are As a Senior Data Engineer at Metropolis, you will play a key role in shaping data products... ...of experience with Python, and experience building scalable Big Data solutions and ETL ecosystems Proficiency in SQL, ETL/ELT...SeniorBig dataTemporary workWork at officeLocal areaAfternoon shiftMetropolis Inc
New York, NY3 days ago$250.8k - $286.2k
Capital One is actively seeking a Senior Lead Data Engineer in New York, NY. This position involves collaborating within Agile teams to design and develop innovative solutions using cloud and big data technologies. As a leader, you will guide fellow engineers in tackling...SeniorBig data- ...Senior Lead Data EngineerDo you love building and pioneering in the technology space? Do you enjoy... ...? At Capital One, you'll be part of a big group of makers, breakers, doers and... ...real customer needs. We are seeking Data Engineers who are passionate about marrying data...SeniorBig dataInternship
Capital One
New York, NY2 days ago $120k - $140k
...Opportunity Omni Platform Group (OPG) is currently seeking a senior data engineer to join our technology team. In this role you will build Omni... ...with Python, C++, or other popular language. Experience with big data and/or infrastructure. Bonus for having experience in setting...SeniorBig dataWork at officeLocal areaImmediate startFlywheel
New York, NY1 day ago$185k
...Senior Data EngineerNew York, NY (This role is based in Trepp's NYC office and follows a hybrid... ...with multiple product teams and engineering teams to solve complex problems in our financial... ...least 3 years of experience in big data technologiesDemonstrated experience...SeniorBig dataTemporary workWork at officeWork from homeMonday to FridayTrepp, LLC
New York, NY10 hours ago$250.8k - $286.2k
...Senior Lead Data Engineer What You’ll Do: Collaborate with and across Agile teams to design, develop, test, implement, and support technical solutions... ...does not apply) At least 2 years of experience in big data technologies At least 1 year experience with cloud computing...SeniorBig dataFull timePart timeInternshipLocal areaCapital One National Association
New York, NY1 day ago- We're transforming our customer support data from static, disconnected reports into a live... .... We're looking for an experienced Data Engineer to build the pipelines, storage, and logic... ...Skills 5+ years in data engineering with big data tools (Hadoop, Hudi, Spark, Presto, Pinot...SeniorBig data
$136.04k - $182.49k
Role Overview Engineering Services designs, builds, and sustains technology that powers DE&E Technology... ..., including Quality Assurance, Data Analytics, Software Development, Broadcast... ...model development. Experience working in big data environments (Hadoop, S3, HBase)....SeniorBig dataWorldwide$176k - $210k
Komodo Health Inc is seeking a Senior Data Engineer based in San Francisco to develop and maintain AWS cloud infrastructure and enhance data quality... ...AWS services, along with experience in CI/CD pipelines and big-data tools. This role offers a salary range of $176,000 - $21...SeniorBig data$220k - $245k
...benefits using clear incentives and powerful, data-driven insights. Our approach guides... ...the role: We are seeking an exceptional Senior Data Engineer with a high sense of ownership who is... ...processing, data warehousing, data governance, Big Data, data variance, data privacy, and...SeniorBig dataWork at officeFlexible hours3 days per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Big Data Engineer. Be the first to apply!
Related searches
- data center engineer New York, NY
- senior cloud data engineer New York, NY
- data infrastructure engineer New York, NY
- sr information security engineer New York, NY
- hadoop big data developer New York, NY
- data visualization developer New York, NY
- associate data engineer New York, NY
- remote data engineer New York, NY
- etl data engineer New York, NY
- data engineer analytics New York, NY