Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Apache Spark / Big Data Administrator

Mphasis

Job Description

Role : Apache Spark / Big Data Administrator

Location : Berkley Heights, NJ

Work Type : 5 days a week at client office

Job Description

  • 10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.
  • Focuses on cluster health, performance tuning, and production data pipelines.
  • Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).
  • OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).
  • Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).
  • Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loads
  • Excellent coordination with stakeholders and communication skills

Must have experience with below responsibilities

  • Cluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.
  • Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.
  • Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads.
  • Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.
  • Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster.
  • Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.

Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs

Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Apache Spark / Big Data Administrator in Berkeley Heights, NJ vacancy
  •  ...Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJ Work Type : 5 days a week at client office Job Description ~10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated... 
    Big data
    Work at office

    Mphasis

    Berkeley Heights, NJ
    10 days ago
  •  ...JavaScript, Angular 11 or higher, React/JavaScript frameworks, CSS, HTML, Nodejs, JSON etc. Experience with Big Data technologies like Apache Hadoop, HBase, Spark, Hive, Snowflake. Agile Development experience. SDLC Tools like Maven, Jenkins, Git, etc. Testing tools... 
    Big data

    E-Solutions

    Berkeley Heights, NJ
    3 days ago
  • Job Title: Big Data Engineer (Java,Spark, Hadoop, Ozone CH) Location: Berkeley Heights, NJ Role Overview: We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop... 
    Big data
    Work at office

    Prophecy Technologies

    Berkeley Heights, NJ
    3 days ago
  •  ...Job Description Position: Data Architect Onsite: 3 days a week...  ...Architect, you will be responsible for big data and analytics projects...  ...Expertise in Synapse, Presto, Spark SQL, Druid, and SuperSet....  ...warehouses. Knowledgeable in Apache Airflow. Previous experience... 
    Big data
    Contract work
    Work experience placement
    3 days per week

    Expert In Recruitment Solutions

    Summit, NJ
    26 days ago
  •  ...difference at Fiserv.Job TitleSenior Director- Data EngineeringAbout your role:You will be...  ...years of experience in data engineering, big data, or software development, including...  ....4+ years of hands-on experience with Apache Spark for large-scale distributed data processing... 
    Big data
    Full time
    Temporary work
    H1b
    Work at office
    Monday to Friday

    Fiserv

    Berkeley Heights, NJ
    17 hours ago
  • GCP Senior Big Data Engineer Looking for a Senior Big Data Engineer/Architect on Google Cloud Platform to help strategize, architect...  ...be comfortable with Big Data Technologies like Hadoop, Hive, Apache Spark, Oozie and Google Cloud platform. You will work on advanced Apache... 
    Big data

    Samprasoft

    Basking Ridge, NJ
    3 days ago
  •  ...Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We...  ..., software engineering, and modern Big Data analytics infrastructure, building scalable...  ...EngineeringLeverage Databricks, Apache Spark, or comparable technologies to perform distributed... 
    Big data
    Live in
    Work at office
    Local area

    Accenture

    Morristown, NJ
    3 days ago
  •  ...Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve...  ...Factory, Snowflake, or Databricks. You have experience with big data tools such as Apache Spark or distributed databases. You have experience designing and... 
    Big data
    Work experience placement
    Live in
    Work at office
    Local area
    Remote work

    Accenture

    Morristown, NJ
    4 days ago
  •  ...Job Title : Data Engineer Ab initio Location: Berkeley Height, NJ We are seeking a highly skilled resource to design and implement...  ...data validation, data security, integrity, and compliance across big data platforms. • Monitor and troubleshoot performance issues... 
    Big data

    Envision Technology Solutions

    Berkeley Heights, NJ
    25 days ago
  •  ...language. (said that this must have might have to go. Not finding anyone with it. Wants us to let him know, but as long as they have 2 of the 3 he will interview. Nice to Haves: Awareness of Big Data Platforms like Hadoop & Teradata. Know what Yarn/Hive is... 
    Big data
    Contract work
    3 days per week

    Expert In Recruitment Solutions

    Summit, NJ
    26 days ago
  • Sr. Data Engineer Data Engineers who can build scalable, reliable data pipelines and infrastructure, requiring mastery in SQL, Python/Scala, cloud platforms (AWS), and big data technologies (Spark, Kafka, Hadoop). Key skills include data modeling, ETL/ELT pipeline construction... 
    Big data

    E-Solutions

    Rahway, NJ
    3 days ago
  •  ...Job Description Contract Cloud Data Engineer MUST be local to Iselin...  ...loading. Develop and deploy Spark/PySpark programs and Databricks jobs for big data workloads. Write robust Python...  ...-Haves: Cloud certification(s), Apache Airflow experience, and cloud... 
    Big data
    Contract work
    Local area

    Shulman Fleming & Partners

    Iselin, NJ
    23 days ago
  •  ...degree in computer science Business Administration or equivalent educational or...  ...10 years of indepth experience in Data Management Data Warehouse solutions...  ...Preferred: Prior experience working in big data technologies like Hadoop Spark Kafka Snowflake certifications SnowPro... 
    Big data
    Contract work

    Samprasoft

    Warren, NJ
    3 days ago
  •  ...Description Job Description Job Title: Data Engineer - Data Platforms Location:...  ...- 5825048 Required Skills: " Apache Spark & Scala Strong hands-on experience developing...  ...experience working with enterprise Big Data platforms and large datasets. " SQL... 
    Big data

    eCom Solutions Inc

    Piscataway, NJ
    26 days ago
  •  ...Risk management solution. • 3+ years implementing Data ops, ML ops • MS or BS in Computer Science, Information...  ..., JSP, Servlet, Node.js, Angular, • Proficient in Big Data Technologies, Data Transport (Pulsar/Kafka), Spark, Jupyter/ Python. • Experience working with... 
    Big data

    Samprasoft

    Piscataway, NJ
    3 days ago
  • NAVA Software solutions is looking for a Sr. Risk Data Analyst for one of our clients. Details: Sr. Risk Data Analyst Location: Berkeley...  ...data analytics and visualization tools. Predictive modeling and big data experience. Background in a modeling and testing (champion/... 
    Big data
    Work at office

    Nava Software Solutions

    Berkeley Heights, NJ
    3 days ago
  • Data Architect With Dq Location: Iselin, NJ (Hybrid) Job Overview: We are seeking a highly skilled...  ...and their application in data analysis. Big Data Technologies: Experience with big data technologies like Hadoop, Spark, etc. Cloud Computing: Familiarity with cloud... 
    Big data

    United IT

    Iselin, NJ
    3 days ago
  • Data Architect USA - Clark, NJ 07066 Overview GEP is different. It's a place where individuality...  ...Provide thought leadership around big data ecosystem and data warehouse design by...  ...Guides team members on the development of Spark programs to ingest/process data Qualifications... 
    Big data
    Local area

    N B Ventures

    Clark, NJ
    3 days ago
  • $58 per hour

     ...future launches, and business development projects Provide a deep assessment and understanding of where the current landscape of big data stands, from scientific methodologies to capabilities to the various corporations and what is relevant to rare diseases Support... 
    Big data
    Hourly pay
    Apprenticeship
    Summer work
    Internship
    Summer internship
    H1b
    Work at office
    Worldwide
    Relocation
    Monday to Friday

    Sanofi

    Morristown, NJ
    1 day ago
  • We Are:The beginning of a new Data & AI decade that will reshape work and society has begun. Accenture is...  ...Minimum of 3 years of hands-on experience on Cloud and Big Data technologies such as serverless data processing, Spark/SparkSQL, batch and streaming services, event-... 
    Big data
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Morristown, NJ
    1 day ago
  • $120.5k - $231k

     ...you’ll be doing...The Principal Engineer - Data Science joins the high-visibility Impact...  ...structured and unstructured datasets across big data systems using SQL and Python.Advanced...  ....Experience with Big Data frameworks (Spark, Splunk, Hadoop) and cloud analytics platforms... 
    Big data
    Full time
    Temporary work
    Part time
    Work experience placement
    Work at office
    Work from home
    Shift work
    3 days per week

    Verizon

    Basking Ridge, NJ
    1 day ago
  •  ...strength in technology and leadership in cloud, AI and data with unmatched industry experience, functional expertise...  ...AI, Anthropic, AWS, Azure, Google Cloud).Knowledge of big data technologies (e.g., Hadoop, Spark).Knowledge of agent-based modeling, reinforcement learning... 
    Big data
    Live in
    Work at office
    Local area

    Accenture

    Morristown, NJ
    1 day ago
  •  ...s biggest business challenges. As a Data Scientist, you’ll apply innovative statistical...  ...distributed processing to scale to big data + Minimum of 5 years of experience...  ...distributed data processing pipelines (Apache Spark or Apache Beam) + Ability to effectively... 
    Big data
    Hourly pay
    Live in
    Work at office
    Local area
    Flexible hours

    Accenture

    Morristown, NJ
    24 days ago
  •  ...-oriented and organized HR Specialist to assist with various administrative tasks. The ideal candidate will be responsible for handling mail...  ...-oriented and able to manage multiple tasks simultaneously. Required Skills : Oracle Additional Skills : Big Data... 
    Big data
    Contract work
    Work at office
    Remote work
    Monday to Friday

    Kaav Inc.

    Madison, NJ
    3 days ago
  •  ...Hadoop Developer / Data Engineer / Big Data Developer Iselin, NJ 08830 - Onsite 1 Day a Week! 12 + Months At least 1 day onsite a week Requirements: Under the leadership of the Data Management and Analytics organization, Data Engineer is responsible for delivering... 
    Big data
    1 day per week

    Samprasoft

    Iselin, NJ
    17 hours ago
  • We Are: The beginning of a new Data & AI decade that will reshape work and society has begun. Accenture...  ...Minimum of 3 years of hands-on experience on Cloud and Big Data technologies such as serverless data processing, Spark/SparkSQL, batch and streaming services, event-... 
    Big data
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Convent Station, NJ
    7 hours ago
  • $110.7k - $218.3k

     ...commerce, sales, and service. We are a team of strategists, data scientists, operators, creatives, designers, engineers,...  ...(Python, Java, or Bash scripting)Experience with Big Data technologies (i.e., Hadoop, Spark, Redshift, Snowflake, Hive, etc.)Experience with Martech... 
    Big data
    Local area
    Visa sponsorship

    Deloitte

    Morristown, NJ
    17 hours ago
  • $95.01k - $98k

    Hadoop Administrator Softstandard Solutions LLC, a Green Brook Township, NJ-based IT Solutions Firm...  ...Linux RedHat servers. Performing various data operations on Hadoop environment....  ...platforms that support client's vision of Big Data. Working with Cloudera support to maintain... 
    Big data
    Relocation

    SoftStandard Solutions

    Dunellen, NJ
    3 days ago
  • Data Engineer Choosing Capgemini means choosing a company where you will be empowered to...  ...working with cloud-based data platforms and big data technologies. Knowledge of data...  ...lakehouse architectures. Familiarity with Spark, Databricks, Snowflake, or similar data technologies... 
    Big data
    Permanent employment
    Full time
    Contract work
    Local area

    Capgemini

    Whippany, NJ
    3 days ago
  •  ...initiatives across the enterprise with focus on network design, big data platform, infrastructure as a service, etc. Provide high...  ...escalation for L1 and L2 respectively. Manage the day to day administration of VMware and vBlock environment including system capacity and... 
    Big data
    Work experience placement

    ExterNetworks

    Piscataway, NJ
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Apache Spark / Big Data Administrator. Be the first to apply!