Senior Big Data Engineer
PB consulting
We are seeking a Senior Big Data Engineer with 8+ years of IT experience and strong expertise in Hadoop, Spark, Python, and Snowflake. The ideal candidate will have hands-on experience building scalable data pipelines, migrating data from Hadoop to Snowflake using PySpark, and developing enterprise ETL solutions.
Roles and Responsibilities
- Design, develop, and maintain scalable data pipelines using Hadoop, Spark, Python, and Snowflake.
- Perform data migration from Hadoop to Snowflake using PySpark frameworks and libraries.
- Build and optimize ETL workflows for large-scale data ingestion, transformation, and validation.
- Develop and optimize complex SQL queries for data processing and analytics.
- Ensure data quality, performance, and reliability across enterprise data platforms.
- Collaborate with cross-functional teams to gather requirements, troubleshoot issues, and deliver data solutions.
Required Qualifications
- 8+ years of overall IT experience in data engineering or big data development.
- Strong hands-on experience with Hadoop, Apache Spark, Python, and Snowflake.
- Experience migrating enterprise data from Hadoop to Snowflake using PySpark.
- Solid understanding of ETL processes, data integration, and data transformation techniques.
- Strong SQL skills with experience writing and optimizing complex queries.
- Excellent analytical, problem-solving, and communication skills.
- Experience working in Unix/Linux environments. (Preferred)
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Big Data Engineer in Kingston, NY vacancy
$110k - $167k
Aon is seeking a leader for the Construction & Infrastructure Claims Team in Kingston, NY. You will provide operational direction and support, directly manage the team, and oversee complex claims. This role requires exceptional interpersonal and project management skills...SeniorAon
Kingston, NY5 days ago$17.25 per hour
...A Senior Retail Stocking Associate (full-time) is a valued member of a high performing team who is empowered & equipped to do their job. You will know why your work matters and be able to take pride in what you do! This role is perfect for you if you have the desire to...SeniorHourly payFull timeTemporary workHarbor Freight Tools
Kingston, NY2 days ago- HealthAlliance Hospital · AccountingKingston, NYFinance/Info SystemsFull TimeDayM-F 8a-4pReporting to the Director of Finance, the Senior Accountant is responsible for the preparation, review and analysis of monthly and annual financial statements for multiple facilities...SeniorTemporary work
- What We're Looking For You will be working on a wide variety of projects, from branding, advertising, web design, and everything in between. Our style is very collaborative and informal so versatility and communication skills are key. You are a Designer with well-rounded...Senior
- Courier II LabCorp is seeking a Courier II to join our team in Kingston, New York. This position includes a $1,000 Sign-On Bonus! (External candidates only) Work Schedule: Monday Friday 2:00pm 10:30pm; ability to cover 1st and 3rd shift and weekends as needed Job...SeniorRelocation packageMonday to FridayShift workNight shiftWeekend workDay shift
- ...Senior Inspectors are responsible for inspecting, testing, and maintaining fire sprinkler systems, fire pumps, and backflows with minimal supervision. Essential Job Duties Inspect, test, and maintain fire protection systems in accordance with NFPA codes with working knowledge...SeniorFull timeTemporary workTraineeshipWork at officeLocal areaFlexible hours
Summit Fire Protection
Kingston, NY2 days ago - ...of high-performance AC and brushless DC Fans, Blowers, and Cooling Systems for the aerospace and defense industries The Senior Design Engineer supports the design, development, and testing of various types of electric motors, including both brushless DC and AC induction...Senior
NPAworldwide
Woodstock, NY5 days ago - As the Senior PCB Designer with AMETEK Rotron, you will play a key part in supporting engineering efforts by assisting in the design and development of detailed layouts of printed circuit boards. You will develop printed circuit board layouts based on input from both electrical...Senior
- ...Position Summary The Senior Project Manager is responsible for the overall leadership... ..., Assistant Project Managers, Project Engineers, and field leadership. Ensure effective... ...build, negotiated, industrial, healthcare, data center, mission-critical, or large...SeniorContract workFor contractorsFor subcontractorWork at officeLocal areaWeekend workAfternoon shiftEarly shift
Dynamic Systems
Bloomington, NY3 days ago - ...Huck's Market Senior Associate This is so much more than just a job! Come join a company where you have an opportunity to be an owner. We are a 100% Employee-Owned Company. It takes a special person to be an Associate with Huck's. Our standards for customer service...SeniorWork at officeLocal areaImmediate startRelocationAll shiftsShift workAfternoon shift
- As the Senior Motor Design Engineer with AMETEK Rotron, you will provide critical support for the design and development of engineering projects. In this role you will support the design, development, and testing of various types of electric motors, including both brushless...SeniorWork at office
- The Senior Buyer represents the organization in the acquisition and uninterrupted supply... ...Collaborates closely with Sales, Manufacturing, Engineering, and Finance to support the organization’... ...self‑management, ensuring decisions are data‑driven and aligned with business...SeniorWork at officeLocal area
$85k - $120k
...types of projects including site development, drinking water, sanitary sewer, storm sewer, stormwater management, and other civil engineering projects. You will have the opportunity to oversee large scale projects, while also being able to develop work within the...SeniorWeekly payFull timeWork at officeC.T. Male Associates
Ulster Park, NY3 days ago$16 - $17.5 per hour
ShopRite - Produce Clerk (SRS NY) Salary Range $16.00 - $17.50/hr Location: Kingston, NY (ShopRite of Kingston) Job Description Title: ShopRite - Produce Clerk Description: We are living our Purpose - To Care Deeply about People, Helping them to Eat Well and...SeniorLocal areaImmediate startShift work$147.14k - $193.12k
...Senior Project Manager – Water or Wastewater We are committed to making a meaningful impact through our work. As a Senior Project... ...professional involvement. What You’ll Need Bachelor’s degree in Civil Engineering, Environmental Engineering, or a related field. Substantial...SeniorFull timeSummer workWork at officeLocal areaFlexible hoursTighe & Bond
Rhinebeck, NY5 days ago$97.66k - $104.49k
Description Are you an Education professional interested in making a positive difference in the lives of children with developmental disabilities? If you answered YES to either of these questions, consider joining our Devereux Advanced Behavioral Health team! Being Principal...Devereux Advanced Behavioral Health
Red Hook, NY1 day ago$16 - $17.5 per hour
ShopRite - Bakery Clerk (SRS NY) Salary Range $16.00 - $17.50/hr Location: Kingston, NY (ShopRite of Kingston) Job Description Title: ShopRite - Bakery Clerk We are living our Purpose - To Care Deeply about People, Helping them to Eat Well and Be Happy. This...SeniorLocal areaImmediate startShift work$16 - $17.5 per hour
ShopRite - Online Shopping Clerk (SRS NY) Salary Range $16.00 - $17.50/hr Location: Kingston, NY (ShopRite of Kingston) Job Description We are living our Purpose To Care Deeply about People, Helping them to Eat Well and Be Happy. This Purpose guides everything...SeniorLocal areaImmediate startShift work$56.8k
...University of New York located ninety miles north of New York City in the Catskill Mountains, is currently accepting applications for a Senior Program Coordinator: Advanced Technology, Clean Technology, and Advanced Manufacturing. The Program Coordinator/Director for...SeniorApprenticeshipWork at officeLocal area$80k - $95k
...(On-site) ~Job Schedule Full time ~Export-Controlled Data No , Responsibilities Howmet Fastening Systems is looking for a Senior Quality Engineer to support and provide direction in the implementation of product quality systems...SeniorPermanent employmentFull timeHowmet Corporation
Kingston, NY3 days ago- ATS Company: SP IndustriesSenior
$96k
Assistant Principal Rhinebeck, NY ( Job Type Full-time Description If you're passionate about working with children and making a difference in their lives, we want to hear from you! The Astor Learning Center (ALC), a therapeutic special education school for...Full timeLocal areaAstor Services for Children & Families
Rhinebeck, NY3 days ago- ...Job Description Job Description SENIOR BUILDING INSPECTOR **YOU MUST COMPLETE THE CITY OF KINGSTON NY EMPLOYMENT APPLICATION... ...building division staff and developers, contractors, architects, engineers, and public. Coordinate final inspections with other...SeniorFull timeFor contractorsFor subcontractorLocal area
- Job Description Job Description CPA firm located in Rhinebeck, NY is seeking a talented auditor/ tax preparer, with 2-5 plus years overall public accounting experience in the areas of attestation services and tax services . We are looking for an individual to join...SeniorWork at officeFlexible hours
- ...Job Description Job Description Senior Scheduler - Data Center - New Paltz, NY Our client is a Commissioning Company that has a national... ..., ETN, MMNs, Seabees; Army – Power Generation, Corps of Engineers; Air Force – Power Production; Generator Techs; Maritime,...SeniorRemote work
- ...deepening our investment in product and engineering. We’ve assembled an in-house engineering... ...of continued development. This is a senior individual-contributor role. Beyond product... ...and track product success metrics, using data to validate outcomes and inform prioritization...SeniorTemporary workWork at officeRemote workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Big Data Engineer. Be the first to apply!
Related searches
- senior performance engineer Kingston, NY
- senior Kingston, NY
- senior operations technician Kingston, NY
- senior manager diversity & inclusion Kingston, NY
- senior application security Kingston, NY
- senior application administrator Kingston, NY
- srs Kingston, NY
- software data engineer
- entry level big data engineer
- big data developer