Big Data Developer
Techvilla Solutions
We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.
Key Responsibilities
- Develop, maintain, and optimize big data applications and data processing pipelines.
- Work with Kafka and Apache Spark for real-time and batch data processing.
- Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
- Write and optimize complex SQL queries using Hive and Impala.
- Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
- Use Splunk for log analysis, monitoring, and troubleshooting.
- Manage and monitor batch workflows and job scheduling using AutoSys.
- Work in Linux-based environments for development, deployment, and operational support.
- Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
- Ensure data quality, reliability, scalability, and performance of big data applications.
Required Technical Skills
- Strong hands-on experience with Kafka, Spark, Scala, and SQL.
- Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
- Proficiency in data analysis and data wrangling for large datasets.
- Experience in performance analysis, tuning, troubleshooting, and issue resolution.
- Hands-on experience with Splunk and AutoSys.
- Strong knowledge of Linux-based infrastructures.
- Experience in distributed data processing and big data technologies.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Big Data Developer in Shrewsbury, NJ vacancy
- Position: Senior Data Engineer Location: 99 Jefferson Rd, Parsippany, NJ 07054 (Onsite... ...engineering teams. Experience in building Big Data/ML/AI applications and optimizing... ...IAM, Security Group, VPC etc. Experience developing in Python and Notebooks/IDE Experience with...Big data
$146k - $234k
ResponsibilitiesWe are seeking a Data Scientist to build and curate the knowledge base for a generative AI/ML research effort, and to develop the data and knowledge extraction pipelines for machine learning models. The role is responsible for extracting, structuring, and...SuggestedContract workFor subcontractorShift workPeraton Corporation
Red Bank, NJ5 days ago- ...Job Description Job Description Description: Develop, maintain, and enhance Power BI reports, dashboards, data models, and Microsoft Fabric solutions to support business intelligence and data-driven decision-making. Troubleshoot and resolve issues related...SuggestedWork experience placement
Mindlance
Belmar, NJ3 days ago $146k - $234k
...tracking, model versioning, dataset loaders for synthetic and real data, and the objective and interfacePackage trained models for... ...containerized code that runs unattended in an environment the developer does not controlSome working knowledge of multi-objective and constrained...SuggestedContract workFor subcontractorShift workPeraton Corporation
Red Bank, NJ5 days ago- ...Description The Account Executive will identify, develop, and close new business opportunities for... ...50 years, Verisk has been the leading data analytics and technology partner to the... ...for people like you to help us translate big data into big ideas. Join us and create an...Big data
Verisk Analytics
Holmdel, NJ1 day ago - ...Recycling & Refining Sibanye-Stillwater is seeking an Associate Data Engineer to join our Business Intelligence Center of Excellence... ...building and maintaining data ingestion and transformation pipelines. Develop clean, reliable, and well‑modeled datasets aligned with business...Monday to Friday
- Sibanye-Stillwater is seeking an Associate Data Engineer to join the Business Intelligence Center of Excellence for the Recycling & Refining organization. This on-site role in Langhorne, PA supports data pipelines, datasets, and analytics solutions, with a focus on automation...Remote work
- Peraton is seeking a Communications/RF Data Collection and Software Defined Radio (SDR) Engineer in Red Bank, NJ to lead large-scale... ...role spans building automated data collection infrastructure, developing channel emulation pipelines, and integrating signal processing...
- ...Tracking: Manage Google Tag Manager and similar tools to ensure accurate tracking of website features and user interactions, resolving data collection issues as needed.Data Analysis & Insights: Analyze multi-channel traffic and campaign data to uncover trends and deliver...
E-Solutions
Holmdel, NJ2 days ago - ...performance and learning needs and designs, develops, delivers, deploys, and continuously... ...learner feedback, assessment results, LMS data, and business-performance indicators to evaluate... ...for people like you to help us translate big data into big ideas. Join us and create an...Big data
Verisk Analytics
Holmdel, NJ2 days ago $154.1k - $231.1k
...Business Unit Overview Consumer, network, and data center applications are evolving at an... ...formulate structured research plans, and develop innovative engineering solutions — including... ...noise coupling) Develop and apply AI and big data analytics for predictive PI modeling...Big dataLocal areaTE Connectivity
Middletown, NJ1 day ago- ...for Spring 2027. This fully onsite program invites students to develop analytical and engineering skills in a real-world supply chain environment... ..., warehousing, and material flow initiatives, analyzing data and supporting continuous improvement across our distribution...Internship
- ...levels with machine learning experience. These are hybrid positions in Monmouth County. If you have solid computer science fundamentals (data structures, algorithms, etc.) and experience with C or Python and Linux you may qualify for one of these exceptional opportunities....
- ...in the arsenal of a Machine Learning engineer. Computer Science Fundamentals and Programming. NLP and Text Mining Required. Pandas Data Analysis Required Experience on Cleansing Data with Python. Education Requirement Bachelors, Masters in Computer Science/ Computer Engineering...Full timeImmediate start
- ...#LI-MC1 About Us For over 50 years, Verisk has been the leading data analytics and technology partner to the global insurance industry... ...innovation. And we are looking for people like you to help us translate big data into big ideas. Join us and create an exceptional experience...Big dataFull timeTemporary workWork at officeRemote workFlexible hours
- TE Connectivity is seeking a Power Integrity Engineer to lead power integrity design and testing for advanced data center solutions. You will simulate, validate, and deploy power delivery systems, working with a multidisciplinary team to meet strict performance targets...
- iCIMS Middleware Test Company seeks an associate developer to compile and maintain product specifications, revise nutritional data using Genesis R&D, and review label information for buy-out products and ingredients. You will perform shelf life testing, collaborate with...
- TE Connectivity is seeking a results-driven Project Manager Engineer I to coordinate supplier-driven projects for data center and connectivity applications. You will define and track prototype requirements, manage purchasing activities in SAP, and report milestones and...
$156k - $234k
...to be a subject matter expert in power integrity, leading the technical direction for power distribution products in next-generation data centers. You will be responsible for technical roadmaps, mentoring engineers, and collaborating with internal and external...- ...3, United StatesContact: Not ApplicableOnsite Support Engineer (Data & ETL Platform) - InvestmentsRole SummaryWe are seeking an experienced... ...-M.• Perform SQL-based data analysis and troubleshooting.• Develop and maintain Unix Shell scripts to automate support activities.•...
VDart
Holmdel, NJ1 day ago $100k
...Cloud World/Oracle Java one (Las Vegas) -2023/2022 and at Gartner Data Analytics Summit (Florida)-2023. All positions are open for... ...looking for entry-level software programmers, Java full-stack developers, Python/Java developers, data analysts/data scientists, and...Full timeH1bSynergisticIT
Lincroft, NJ3 days ago- ...performance, document results, and use evaluation data and stakeholder feedback to improve... ...platforms. Preferred Qualifications and Developing Capabilities Experience with... ...for people like you to help us translate big data into big ideas. Join us and create an...Big dataFull timeTemporary workFlexible hours
- iCIMS Middleware Test Company in Matawan, NJ is seeking an associate developer to design and implement features per a functional specification. The role emphasizes data and specification work across product lines. Responsibilities include compiling product specs, revising...
- Data Scientist Location: Hybrid 3 days in New York, NY, Holmdel, NJ, Bethlehem, PA, or Boston, MA Industry: Insurance Domain Term:... ...disciplinary team including data engineers, business analysts, software developers and functional business experts and business leaders. You Have:...Full time
- Senior Data Scientist We are seeking a Senior Data Scientist, Agentic AI—an experienced individual contributor with deep expertise... ...the Data Science Lab by establishing repeatable frameworks for developing, testing, and deploying agentic AI solutions. Mentor junior data...
- Senior Data Architect About the job Our client (Large Insurance company) is seeking a Senior... ..., reliability, scalability, and security. Develop, document, and maintain logical and... ...solutions and provide guidance across storage, big data platform services, Big Data ecosystem...Big dataRemote workFlexible hours
- TE Connectivity is seeking a reliability engineer for its Digital Data Networks (DDN) team. You will design and drive reliability for Data and Device Network products from concept through production, applying DFSS, HALT/HAST, and robust design principles. Collaborate with...
$112k - $179k
ResponsibilitiesPeraton Labs is seeking a Senior AI Integration Developer to lead the design and implementation of an AI assistant... ...and tooling infrastructure that connects it to the application’s data, workflows, and services. Given the sensitive nature of the operational...Contract workLocal areaShift workPeraton Corporation
Red Bank, NJ5 days ago- ...Monmouth County. If you have solid computer science fundamentals (data structures, algorithms, etc.) and experience with C, Python and... ...opportunities. The Software Engineering - ML roles focus on developing and maintaining high-performance software with a strong foundation...
- Thermal Works LLC is seeking a Commercial Cost Engineer to lead pricing and commercial evaluation for large-scale data center cooling systems. You will build scalable cost models covering equipment, logistics, and services for U.S. and international projects, partnering...Remote job
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Developer. Be the first to apply!
