Apache Spark / Big Data Administrator
Full-time
SysTechCorp Inc
Job Description
- 10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.
- Focuses on cluster health, performance tuning, and production data pipelines.
- Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).
- OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).
- Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).
- Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loads
- Excellent coordination with stakeholders and communication skills
Must have experience with below responsibilities
- Cluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.
- Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.
- Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads.
- Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.
- Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster.
- Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.
Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Apache Spark / Big Data Administrator in Berkeley Heights, NJ vacancy
- ...well. We have the below requirement with my client, please let me know if you are looking for a job. Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJ Work Type: 5 days a week at client office Job Description ~10+ years...Big dataFull timeWork at office
Envision Technology Solutions
Berkeley Heights, NJ2 days ago - ...Company: HAN Staffing Job Title: Sr. Core Java Developer w/d Strong spark experience (3 positions)Location: Jersey City, NJ - Hybrid... ...solutions and integrate with distributed systems.Strong expertise in Big Data technologies, including hands-on experience with Hadoop...Big data
HAN Staffing
Iselin, NJ5 days ago - ...difference at Fiserv.Job TitleSenior Director- Data EngineeringAbout your role:You will be... ...years of experience in data engineering, big data, or software development, including... ....4+ years of hands-on experience with Apache Spark for large-scale distributed data processing...Big dataFull timeTemporary workH1bWork at officeMonday to Friday
Fiserv
Berkeley Heights, NJ2 days ago - Job Title: Big Data Engineer (Java,Spark, Hadoop, Ozone CH) Location: Berkeley Heights, NJ Role Overview: We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop...Big dataWork at office
- Position: Data Architect Onsite: 3 days a week Location: irving, Texas... ..., you will be responsible for big data and analytics projects... ...Expertise in Synapse, Presto, Spark SQL, Druid, and SuperSet. Strong... ...data warehouses. Knowledgeable in Apache Airflow. Previous experience in...Big dataContract workWork experience placement3 days per week
- Job ID: 19657488Reference Number: 23-00858Title: Big data roleLocation: Iselin, NJ, 08830Posted Date: 2023-05-09Contact: Shyam MaramContact... ...large datasets and building data pipelines. Hadoop, Apache Spark using PySpark or Scala, Hive/Presto, Hue etc. Expertise in handling...Big data
HAN Staffing
Iselin, NJ5 days ago - Job ID: 23396055Reference Number: 24-01077Title: AWS Data Engineer with EMRLocation: Iselin, NJ, 08830Posted Date: 20... ...Configure, manage, and optimize Amazon EMR clusters for big data processing using Apache Spark, Hive, or Presto.Container Orchestration: Utilize Kubernetes...Big data
HAN Staffing
Iselin, NJ5 days ago - ...financial crime risks.This role combines big data engineering, entity resolution, graph analytics... ....Data Engineering & Modeling• Build Spark-based ingestion pipelines for customer,... ...proficiency in Scala or Java, with hands-on Apache Spark experience.• Experience with data...Big data
HAN Staffing
Iselin, NJ5 days ago - ...Oracle DB, Angular, NodeJS, CI/CD, Junit Secondary Skill Big Data, Cloud enablement, TOGAF, Database architecture principles... ...is preferred • Experience with Big Data technologies like Apache Hadoop, HBase, Spark, Hive • Agile Development experience, SDLC Tools, Gitlab,...Big dataFull time
Zortech Solutions
Cranford, NJ3 days ago - Job ID: 24358702Reference Number: 24-01224Title: ETL big data TesterLocation: Iselin, NJ, 08830Posted Date: 2024-12-09Contact: Shyam MaramContact... ...ETL Big Data Teater Boston, New Jersey - Hybrid Must scala spark quires sql queries are must etl testing using hadoop, spark ,...Big data
HAN Staffing
Iselin, NJ5 days ago $157.6k - $205.43k
...clinical real-world evidence (RWE) study rescue and clinical data repositories (CDR) beyond a purely technical or SME lens... ...-based architectures. Awareness of cloud and big data ecosystems (AWS/Azure/GCP Spark/Databricks) sufficient to speak credibly with technical...Big dataHourly payFull timeRelocationAxtria
Berkeley Heights, NJ16 days ago- Job ID: 19797312Reference Number: 23-00904Title: AWS Big Data EngineerLocation: Iselin, NJ, 08830Posted Date: 2023-05-18Contact: Shyam... ...services on big data platformsQualificationsFamiliar with AWS, Spark, GLUE, Python and LinuxNice to have skills will be:Knowledge of...Big dataRemote work
HAN Staffing
Iselin, NJ5 days ago $125k - $130k
...Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an... ...batch and real-time data pipelines using Apache Spark, Delta Lake, and Structured Streaming.... ...years of experience in data engineering, big data, or cloud data platforms. • 3+...Big dataFull timeTemporary workFlexible hoursSynechron
Warren, NJ4 days ago- ...Posted Date: 2023-03-15Company: HAN Staffing JOB DESCRIPTION: Apache Airflow Architect to help SFDC migrate existing workflow orchestration... ...and DAG best practices for Airflow Additional Skills/Knowledge: Big data processing pipelines AWS (Azuer or Google Cloud Platform).Big data
HAN Staffing
Iselin, NJ5 days ago $100k - $150k
...Hadoop Big Data Developer - Remote Bright Vision Technologies is a technology consulting and software development company delivering... ...-data pipelines on Hadoop. Strong hands-on expertise with Apache Spark (Scala, Python, or Java) in production environments. Solid...Big dataFull timeH1bLocal areaImmediate startRemote workVisa sponsorshipBright Vision Technologies
South Plainfield, NJ22 days ago- ...telemetry into actionable insights that protect client data and critical systems. You will help us... ....4+ years of experience with SQL and one or more big data or cloud analytics platforms (for example, Hadoop, Spark, Databricks, or cloud-native services) to process...Big dataFull timeTemporary workH1bWork at officeMonday to Friday
Fiserv
Berkeley Heights, NJ5 days ago $157.6k - $205.43k
...real-world evidence (RWE), study rescue, and clinical data repositories (CDR) — beyond a purely technical or SME lens... ...-based architectures. Awareness of cloud and big data ecosystems (AWS/Azure/GCP, Spark/Databricks) sufficient to speak credibly with technical...Big dataHourly payRelocationAxtria, Inc.
Berkeley Heights, NJ21 days ago- ...Title: Big Data Engineer Location: Basking Ridge, NJ (5-day onsite/week) Job Responsibilities: Design and lead end-to-end data... ...on Kafka, and various open-source software platforms (e.g., Apache ecosystem, Linux). ~ Proven ability to optimize data flow and...Big dataFull time
PROCAL TECHNOLOGIES
Basking Ridge, NJ4 days ago - ...Mandatory Skills: SQL/Python/Data Engineer/AWS Data Engineers who can build scalable, reliable data pipelines and infrastructure... ...mastery in SQL, Python/Scala, cloud platforms (AWS), and big data technologies (Spark, Kafka, Hadoop). Key skills include data modeling, ETL...Big data
Omni Inclusive
Rahway, NJ5 days ago - ...Cloud Data Engineer This position is for a Cloud Data engineer with a background in Python... ...integrity. Design, develop, and deploy Spark program in databricks environment to... ...for performance and scalability to handle big data workloads. Monitor and troubleshoot...Big data
United IT
Iselin, NJ3 days ago - Job ID: 19913905Reference Number: 23-00987Title: SR AWS Big Data EngineerLocation: Iselin, NJ, 08830Posted Date: 2023-06-08Contact: Shyam... ...Remote NO CPT AND NO OPT c2cMust Skills : Big Hadoop, Hive, Spark, ETL And Strong AWS Services ResponsibilitiesWrites ETL (Extract...Big dataFor contractorsRemote workRelocation
HAN Staffing
Iselin, NJ5 days ago - ...\n Job Summary: \n We are seeking highly skilled Azure Data Engineer with strong expertise in SQL, Python, Datawarehouse... ...in Azure Cloud Services. \n Solid Understanding of Spark and PySpark for big data processing. \n Experience in relational databases....Big data
Arkhya Tech Inc.
Iselin, NJ17 days ago $125k - $130k
...Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an... ...batch and real-time data pipelines using Apache Spark, Delta Lake, and Structured Streaming.... ...years of experience in data engineering, big data, or cloud data platforms. \n • 3+...Big dataTemporary workFlexible hoursSynechron
Warren, NJ17 days ago- ...to look for patterns/trends and provide administrative direction and coaching to the sales field... ...key sales support activities: Master Data Maintenance, Pricing Entry, Order Tracking... ...including VLOOKUP, pivot tables, formulas, big data & reporting ~ Power BI a plus ~...Big dataWork at officeRemote workMonday to Friday
AA2IT
New Providence, NJ5 days ago - Job ID: 22361668Reference Number: 24-00440Title: Big Data DeveloperLocation: Iselin, NJ, 08830Posted Date: 2024-06-06Company: HAN StaffingdevBig data
HAN Staffing
Iselin, NJ5 days ago - Job ID: 20535849Reference Number: 23-01503Title: BIG DATA DEVELOPERLocation: Iselin, NJ, 08830Posted Date: 2023-09-20Company: HAN StaffingBIG DATA DEVELOPERBig data
HAN Staffing
Iselin, NJ5 days ago - ...detailed technical design, and associated presentation materials.Desired:Hands-on experience with IBM I-series/System I/AS400 and/or Mainframe experienceFinancial Services backgroundData Warehouse - both traditional and Big Data paradigmsMaster Data Management Georgia IT IncBig data
- ...****@*****.*** Skill: Need Strong coder, Fast Api, Pandas Object oriented Python experience there they would have used FastAPi or server side programming using Python in distributed environment via dask or flask Object oriented Python + aws/big data backgroundBig data
HAN Staffing
Iselin, NJ5 days ago - ...Reference Number: 25-00499Title: snowflake develoLocation: Iselin, NJ, 08830Posted Date: 2025-05-21Contact: Deepak KumarContact Email: ****@*****.*** Phone: (***) ***-****Company: HAN Staffing snowflake developer , scala and java -lifting big data load into awsBig data
HAN Staffing
Iselin, NJ5 days ago $151.12k - $226.68k
...real difference, there’s only one choice.Principal Data Scientist PURPOSE:A Principal Data Scientist has complete... ...algorithms and techniques;Familiarity with big data technologies and frameworks (e.g., Hadoop, Spark);Proficiency in SQL, relational database, and NoSQL...Big dataLocal areaRemote workBayer
Whippany, NJ3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Apache Spark / Big Data Administrator. Be the first to apply!
