Big Data Developer
Techvilla Solutions
We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.
Key Responsibilities
- Develop, maintain, and optimize big data applications and data processing pipelines.
- Work with Kafka and Apache Spark for real-time and batch data processing.
- Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
- Write and optimize complex SQL queries using Hive and Impala.
- Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
- Use Splunk for log analysis, monitoring, and troubleshooting.
- Manage and monitor batch workflows and job scheduling using AutoSys.
- Work in Linux-based environments for development, deployment, and operational support.
- Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
- Ensure data quality, reliability, scalability, and performance of big data applications.
Required Technical Skills
- Strong hands-on experience with Kafka, Spark, Scala, and SQL.
- Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
- Proficiency in data analysis and data wrangling for large datasets.
- Experience in performance analysis, tuning, troubleshooting, and issue resolution.
- Hands-on experience with Splunk and AutoSys.
- Strong knowledge of Linux-based infrastructures.
- Experience in distributed data processing and big data technologies.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Big Data Developer in Hollis, NY vacancy
- ...work together.What’s the role?As a Senior Data Scientist on the Ads Marketplace team, you... ...Recommendations, and Search), the organization that develops the machine learning capabilities powering... ...have independently extracted meaning from big data sets with little engineering support....Big dataFull timeWork at officeLocal areaRemote workVisa sponsorship
Etsy
Brooklyn, NY4 days ago - ...Job Description Perform record-by-record reconciliation, aggregate totals validation, and exception identification to ensure no data loss or corruption during the PeopleSoft-to-Workday migration Refine data crosswalks showing PeopleSoft field equivalents, data type...Suggested
The Dignify Solutions, LLC
Plainview, NY7 days ago $105.4k - $207.8k
...accelerate new digital ventures, and fuel growth through innovation.Work you'll do As a Data Engineer III on the AI & Data team, you will be responsible for… Designing, developing, testing, and maintaining batch and near-real-time data pipelines using modern data engineering...SuggestedLocal areaDeloitte
Jericho, NY17 hours ago- ...Data AnalystVisa status: U.S. Citizens and those authorized to work in the U.S. are encouraged to apply. Tax Terms: W2, 1099 Corp-Corp... ...programming language (Python, C,C++, Java, SQL)Familiarity with Big Data frameworks and visualization tools (Cassandra, Hadoop, Spark...Big data
Keylent Inc
Long Island City, NY3 days ago - ...dashboards, scorecards and presentations.Develops and maintains OLAP structures (cubes) to support... ....Acts as the first line of support for data questions and issues and strives to become... ....Familiar with accessing data stored on Big Data platforms (e.g. Hadoop, NOSQL) for data...Big dataPermanent employmentH1b
Sonsoft
Bethpage, NY17 hours ago $110.7k - $218.3k
...marketing, commerce, sales, and service. We are a team of strategists, data scientists, operators, creatives, designers, engineers, and... ...at ****@*****.***. Recruiting tips From developing a stand out resume to putting your best foot forward in the...Local areaDeloitte
Jericho, NY16 hours ago$100k - $130k
...responsible for protecting WSI associates, customers, systems, and data. The Security Operations Center (SOC) monitors enterprise... ...office, closet, laundry room and even outdoor spaces. We've seen some big changes since our first brick-and-mortar store opened more than half...Full timeH1bWork at officeLocal areaHome officeRelocation packageMonday to ThursdayShift workWilliams-Sonoma
Brooklyn, NY4 days ago- ...dashboards, scorecards and presentations.Develops and maintains OLAP structures (cubes) to support... ....Acts as the first line of support for data questions and issues and strives to become... ...with accessing data stored on Big Data platforms (e.g. Hadoop, NOSQL) for data...Big dataPermanent employmentFull timeH1b
Sonsoft
Bethpage, NY17 hours ago - We are seeking a highly motivated and technically skilled IT Support Specialist to join our growing team. This role is ideal for someone early in their IT career with solid hands-on experience in Microsoft 365, Azure, and Windows networking. The successful candidate will...Full timeRemote work
TBG | The Bachrach Group
Garden City, NY1 day ago - DomainTools is seeking a Principal Product Manager to grow and refine data products powering threat intelligence. You will own the product... ...market-leading data products driven by customer insights and big-data analysis. You will collaborate with engineering, marketing,...Big dataRemote job
$120k - $150k
...application, infrastructure, and security owners.Mentor team members, develop reusable engagement methodologies, and identify safe... ..., closet, laundry room and even outdoor spaces. We've seen some big changes since our first brick-and-mortar store opened more than...Full timeH1bWork at officeLocal areaHome officeRelocation packageMonday to ThursdayWilliams-Sonoma
Brooklyn, NY4 days ago- ...is looking for a Support Engineer to own the complete incident lifecycle—from alert to root cause and permanent fix—covering our Big Data platform and data products. You will work across cloud and on‑prem environments, ensuring reliability and rapid response to issues...Big dataPermanent employment
- A leading telecom company is seeking a Business Analyst to develop and support Big Data solutions across various business units. In this role, you will gather and analyze business requirements and ensure that solutions are clearly specified for implementation. The ideal...Big dataRemote job
$110.7k - $218.3k
...innovative and emergent technologies, including cloud computing, big data and analytics, process automation, artificial intelligence /... ...****@*****.***. Recruiting tips From developing a stand out resume to putting your best foot forward in the interview...Big dataWork at officeDeloitte
Jericho, NY4 days ago- ...that help people do their best work together.What’s the role?Etsy’s Data Platform organization builds systems and infrastructure for... ...marketplace. We believe that small, empowered, self-motivated teams can do big things. We believe in measuring everything, taking advantage of...Full timeWork at officeLocal areaRemote workVisa sponsorship
Etsy
Brooklyn, NY4 days ago - ...Medien basieren. Mit einem Netzwerk hoch spezialisierter Unternehmen definiert und entwickelt Reply Geschäftsmodelle, die durch KI, Big Data, Cloud Computing, digitale Medien und dem Internet der Dinge ermöglicht werden. Reply bietet Beratung, Systemintegration und...Big data
$95k - $105k
Job DescriptionWe are seeking a Senior Data Solutions Engineer to join our growing Data Products & Platforms team. This role is designed... ...in Snowflake using curated ELT pipelines (e.g., Coalesce.io)Develop and maintain Snowflake semantic views to standardize business logic...Full time1-800-Flowers.com
Jericho, NY4 days ago$135k - $158k
...business needs and company policy. Job Purpose As a Staff Data Engineer, you will be part of a Data Engineering team that is... ...scalability, and speed. A Staff Data Engineer will design and develop highly scalable and extensible data pipelines which enable...Work at officeLocal areaImmediate startFlexible hoursNational Grid USA
Brooklyn, NY4 days ago- We are looking for a Support Engineer to keep Kyivstar's Big Data platform and data products running reliably every day. You will own the full incident lifecycle - from the first alert and triage to root-cause analysis, permanent fix and prevention - across our data pipelines...Big dataPermanent employment
$80k - $100k
...Engineering The Visualization Imaging and Data Analysis Center (VIDA) at NYU's Tandon... ...research hub focused on developing innovative solutions to complex data challenges... ...following areas: Database systems and big data management Information integration...Big dataFull timeImmediate startNew York University
Brooklyn, NY1 day ago- Position Summary Data Engineer-Consultant AI&EAre you an experienced, passionate pioneer in technology who wants to work in a... ...stakeholder collaboration.Technical lead, responsible for Design, develop, and operationalize scalable healthcare data platformsDesign,...
Deloitte
Jericho, NY5 days ago $80k - $100k
...InfinityRx turns raw claims and financial data into actionable intelligence. The ideal... ...forecasting and decision-making. Collaborate with developers to improve data integration, quality, and... ...predictive modeling. Familiarity with big data technologies and tools. Excellent...Big data- Datacenter Operations Technician - Join Our Dynamic Tech Team! Are you someone who enjoys working hands‑on with servers, cables, and network gear in a fast‑paced environment? We are looking for a Datacenter Operations Technician with solid experience to join our 24/7 infrastructure...Full timeShift workNight shift
- implementation and initial stabilization Support the September ramp-up and October implementation of inbound and outbound data processes. Ingest, transform, validate, and reconcile Gainwell, Arcadia, DHCS, and other approved data feeds within Acentra's standard claims...
- Job Overview We are looking for a Data Center Engineer to cover the Solon, Ohio area and assist with managing the architecture and operations of our data centers in that region. This person will also occasionally provide support for other regions. Responsibilities Maintaining...Local area
- ProArch is seeking a data integration specialist to support the September ramp-up and October implementation of inbound and outbound data processes. The role involves ingesting, transforming, validating, and reconciling Medicaid claims and eligibility data in ProArch's...
- Altera Digital Health is seeking a Report Developer to design, develop, and support reports for clinical and business users. You will build stored procedures and data extracts aligned with client requirements and provide on-call support for production issues. You will...
- Data is pivotal to our goal of frequent launch and rapid iteration. We’re recruiting a Data Engineer at iRocket to build pipelines, analytics... ...machines, launch telemetry, and operations systems. Develop dashboards, real-time monitoring, data-driven anomaly detection,...Temporary work
- ...Senior Associate You'll join the Card Data & Analytics team, a strategic partner to the... ...complex datasets from multiple sources to develop actionable business strategies and... ...such as Python or SAS, with experience in Big Data technologies including Snowflake, Databricks...Big data
Chase
Brooklyn, NY2 days ago $18 per hour
...services designed for specific industry verticals. iShare was founded to fill a real market need - with so much noise about cloud, big data, and information security, companies today are adopting software product after product and losing sight of which they need and why...Big dataHourly payContract workImmediate startRemote workShift workNight shiftWeekend workAfternoon shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Developer. Be the first to apply!
