Big Data Developer
Techvilla Solutions
We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala, along with performance tuning and troubleshooting in the Cloudera Big Data ecosystem.
Key Responsibilities
- Develop, maintain, and optimize big data applications and data processing pipelines.
- Work with Kafka and Apache Spark for real-time and batch data processing.
- Perform data analysis, data wrangling, and processing of large volumes of structured and unstructured data.
- Write and optimize complex SQL queries using Hive and Impala.
- Monitor, troubleshoot, and resolve performance issues in the Cloudera Big Data ecosystem.
- Use Splunk for log analysis, monitoring, and troubleshooting.
- Manage and monitor batch workflows and job scheduling using AutoSys.
- Work in Linux-based environments for development, deployment, and operational support.
- Collaborate with business users and cross-functional teams to deliver high-quality data solutions.
- Ensure data quality, reliability, scalability, and performance of big data applications.
Required Technical Skills
- Strong hands-on experience with Kafka, Spark, Scala, and SQL.
- Experience with Hive, Impala, Hadoop, and the Cloudera ecosystem.
- Proficiency in data analysis and data wrangling for large datasets.
- Experience in performance analysis, tuning, troubleshooting, and issue resolution.
- Hands-on experience with Splunk and AutoSys.
- Strong knowledge of Linux-based infrastructures.
- Experience in distributed data processing and big data technologies.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Big Data Developer in Cranford, NJ vacancy
- ...We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala...Big data
Techvilla Solutions
Linden, NJ2 days ago - ...Big Data DeveloperSoftstandard Solutions LLC, a Green Brook Township, NJ based IT Solutions and Services Company has multiple openings for Big Data Developer. Job duties include:Installed and configured Hadoop clusters for application development and Hadoop tools like...Big dataRelocation
SoftStandard Solutions
Dunellen, NJ1 day ago - ...permInterview process: screening then one and done with the manager and leadRequirements3+ years of total experience, primarily as a Big Data DeveloperExperience with Big Data platforms/tools (Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, Kafka, etc.)...Big dataWork at office3 days per week
E-Solutions
Iselin, NJ3 days ago - ...process: screening then one and done with the manager and lead Requirements ~3+ years of total experience, primarily as a Big Data Developer ~ Experience with Big Data platforms/tools (Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, Kafka, etc.) ~...Big dataPermanent employmentContract workWork at office3 days per week
E-Solutions
Iselin, NJ17 hours ago - ...Hadoop Developer / Data Engineer / Big Data Developer / Big Data EngineerIselin, NJ 08830 - Onsite 1 Day a Week! 12 + Months At least 1 day onsite a weekRequirements:Under the leadership of the Data Management and Analytics organization, Data Engineer is responsible for...Big data1 day per week
Samprasoft
Iselin, NJ3 days ago - ...e.g. TensorFlow, H2O These are in addition to the experience of data management on Hadoop, which is a foundational element. Role Lead... ...; 3 years of experience in the development of open source big data components like Hadoop, Flume, Hive, Spark, Sqoop, Pig, HBase...Big data
- ...Senior Big Data Engineer Location: New Jersey (hybrid) Duration: 6+ Months We are seeking a highly skilled and experienced Senior... ...of our data initiatives. Responsibilities: Design, develop, and implement scalable data pipelines and ETL processes using...Big data
Zortech Solutions
Elizabeth, NJ1 day ago - ...contract Role Summary: We are seeking an experienced AWS Lead Data Engineer to join our Data Engineering team. As a technical leader... ..., and data ingestion pipelines on AWS. Proven experience with ETL/ELT processes, data modeling, and big data frameworks....Big dataFull timeContract workWork at office
Arkhya Tech Inc.
Newark, NJ2 days ago - ...Job Title : Data Engineer Ab initio Location: Berkeley Height, NJ We are seeking a... ...reliability and troubleshooting data issues • Develop ETL pipelines for batch and real-time data... ..., integrity, and compliance across big data platforms. • Monitor and troubleshoot...Big dataFull time
Envision Technology Solutions
Berkeley Heights, NJ2 days ago - ...to work one-on-one with application and infrastructure developers to build and enhance the data infrastructure and application patterns that power mission... ...Hands‑on experience with ETL, distributed compute, and big data frameworks such as Spark, Glue, Hadoop/EMR, Impala,...Big dataFull time
EXL
Newark, NJ2 days ago - ...Job title: AWS Data Engineer Location: Hybrid 2 days onsite (Newark, NJ) Duration... ...for Data Journey USBIE ~ Experience developing and deploying application code using SQL,... ...scripts ~ Developer experience in big data or warehouse (Hadoop, DB2 LUW) environment...Big data
Damco Solutions
Newark, NJ1 day ago - ...Position title: AWS data engineer Location: Chandler, AZ Onsite: 3 days a week Interview process: 2 video interviews to... ...Analytics, AWS Certified Solutions Architect) Experience with big data technologies like Hadoop, Spark or Kafka Knowledge of...Big dataContract work3 days per week
RIT Solutions, Inc.
Summit, NJ2 days ago - ...Job Title: Senior AWS Data Engineer Location: Newark, NJ / Remote Duration... ...work with application and infrastructure developers to enhance AI/ML infrastructure, ensuring... ...IAM expertise ~ Familiarity with ETL/big data (Glue, Spark, EMR), relational DBs,...Big dataRemote work
3B Staffing LLC
Newark, NJ1 day ago - Talent Lend Inc is seeking a skilled professional to lead the development of innovative technology solutions aimed at solving various business challenges. The ideal candidate will have experience in implementing machine learning models, particularly on platforms like Hadoop...Big data
- ...Microsoft Fabric Data EngineerNewark, New Jersey, United StatesAbout the JobMicrosoft Fabric... ...data management, ETL/ELT processes, and big data technologies. You will be... ...Synapse Analytics, and Azure SQL Database.Develop Data Pipelines: Build, maintain, and orchestrate...Big dataFull timeLocal area
Fortis Hayes
Newark, NJ3 days ago - ...Cloud Data Engineer This position is for a Cloud Data engineer with a background in Python... ...data quality and integrity. Design, develop, and deploy Spark program in databricks... ...for performance and scalability to handle big data workloads. Monitor and troubleshoot...Big data
United IT
Iselin, NJ1 day ago $70 - $75 per hour
...opportunity to work one-on-one with application and infrastructure developers to build and enhance the AI/ML infrastructure and application... ...Groups, KMS Keys, VPC NACLs, and SCPs. Familiar with ETL and big data tool-chains such as those provided by Hadoop/EMR, Glue, Spark,...Big dataFull timeContract workRemote work$32.36 - $50.56 per hour
We are seeking a talented AWS Data Engineer to join our dynamic Data Engineering team The... ...will be responsible for designing developing and maintaining scalable data pipelines and... ...AWS Knowledge of ETLELT data modeling and big data technologies Familiarity with data...Big data- ...required in a relevant field. • 8+ years of experience working in a data engineering role supporting an on-prem Data Lake; experience... ...skills and working knowledge of data structures, algorithms and big data tools (Spark, Hadoop, Python, SQL, NoSQL, Hive); Must...Big data
PALNAR
Newark, NJ3 days ago $79.2k - $124.7k
...dual focus on both operational DLP engineering to protect company data, and other DLP practices. The candidate will be responsible for... ...and align with the company's data protection strategy. • Develop, implement, and maintain automation scripts and processes (e.g.,...Full timeFor contractorsLocal areaRelocationVisa sponsorshipFlexible hoursShift workMerck & Co.
Rahway, NJ3 days ago- ...Job Description Looking for data engineers with 9+ years of experience. 1. Experience developing and deploying application code using SQL, Python, spark. 2.... ...shell scripts. 6. Developer experience in big data or warehouse (Hadoop, DB2 LUW) environment....Big data
- ...Week) Contract: 6-24 months to perm Must have: Python, RDS (must have), Spark, and SQL Nice to have : Pyspark Experience with Big Data platforms/tools (Linux, HDFS, Hive, Hortonworks, Spark, Scala, Python, Java, Kafka, etc.) Experience with API’s, reading from and...Big dataPermanent employmentContract work3 days per week
- ...Job Description Job Description Role: Sr. Data Engineer Hybrid model( Client location, Newark, NJ) Visa: GC, Citizen Required... ...pipelines in AWS. Knowledge of ETL/ELT, data modeling, and big data technologies. Familiarity with data warehousing concepts...Big data
$57 per hour
...version of your resume to ****@*****.*** . Title: Data Scientist – Formulation & Product Development Location:... ...Scientist to transform complex data into actionable insights by developing both predictive machine learning models and GenAI-based solutions...Hourly payFull timeContract workImmediate start3 days per weekKelly
Clark, NJ1 day ago$132.9k - $182.7k
...Senior Structural Engineer professional based within our Advanced Manufacturing team, you’ll work on a variety of projects related to Data Centers, including new builds, retrofits, colocations, etc. You will be performing various aspects of structural engineering...Full timeRemote workJacobs
Clark, NJ3 days ago- ...Job Title: Data Engineer Reference ID: 24-04727 Location: Beaverton, MA... ...prototypes. Participates in quality assurance and develops test application code in client server environment... ...either in Databricks/AWS for building big data pipelines. ~ Experience with...Big dataContract workWork experience placementLocal areaWork from homeVisa sponsorshipWork visa
$100k
...walmart lab s etc to name a few. Currently We are looking for entry-level software programmers Java Full stack developers Python/Java developers Data analysts/ Data Scientists Machine Learning engineers for full time positions with clients. Who Should Apply...Full timeH1bRemote work$117k - $184.2k
...to save and improve lives around the world. Within Small Molecule Science & Technology (SM S&T), we are strengthening the use of data and analytics to improve technical operations, process understanding, and business performance. The Senior Specialist, Data Engineering...Full timeFor contractorsMSD
Rahway, NJ2 days ago- ...know if you are looking for a job. Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJ Work Type:... ...Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations...Big dataFull timeWork at office
Envision Technology Solutions
Berkeley Heights, NJ17 hours ago - ...monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data... ...Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations...Big dataFull time
SysTechCorp Inc
Berkeley Heights, NJ4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Developer. Be the first to apply!
