Apache Spark / Big Data Administrator
Mphasis
Job Description
Role : Apache Spark / Big Data Administrator
Location : Berkley Heights, NJ
Work Type : 5 days a week at client office
Job Description
- 10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated big data frameworks (such as Hadoop, Kafka, or cloud-based data platforms) to ensure high availability and performance.
- Focuses on cluster health, performance tuning, and production data pipelines.
- Technical Experience: Deep hands-on experience with Apache Spark architecture (Driver, Workers, Executors, RDDs/DataFrames) and cluster managers (YARN, Kubernetes, or Mesos).
- OS & Scripting: Strong proficiency in Linux/Unix administration and scripting languages (Bash, Python, or Scala).
- Ecosystem Knowledge: Familiarity with big data and streaming tools like Hadoop HDFS, Kafka, Hive, or monitoring suites (Splunk, Prometheus, Grafana).
- Debugging Skills: Strong analytical capabilities to inspect logs, thread dumps, and metrics under heavy production loads
- Excellent coordination with stakeholders and communication skills
Must have experience with below responsibilities
- Cluster Management: Install, configure, and maintain Apache Spark clusters and node infrastructure in production and staging environments.
- Performance & Monitoring: Monitor cluster health, resource utilization (CPU, memory), and execution logs using tools like Spark UI, Splunk, or Elastic. Identify bottlenecks and optimize job execution times.
- Troubleshooting: Diagnose and resolve out-of-memory errors, executor failures, shuffle issues, and node crashes in distributed workloads.
- Pipeline & Integration Support: Support data engineers and developers in managing Spark applications, streaming jobs, and integrations with message brokers like Kafka.
- Security & Access Control: Implement security principles, role-based access control, authentication, and data encryption across the cluster.
- Backup & Disaster Recovery: Execute backup, recovery, and failover plans for Spark applications, configurations, and critical data stores.
Patching & Upgrades: Apply software patches, driver updates, and version upgrades with minimal disruption to ongoing production jobs
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Apache Spark / Big Data Administrator in Berkeley Heights, NJ vacancy
- ...Role : Apache Spark / Big Data Administrator Location : Berkley Heights, NJ Work Type : 5 days a week at client office Job Description ~10+ years solid experience with deploying, configuring, monitoring, and troubleshooting Apache Spark clusters and associated...Big dataWork at office
- ...JavaScript, Angular 11 or higher, React/JavaScript frameworks, CSS, HTML, Nodejs, JSON etc. Experience with Big Data technologies like Apache Hadoop, HBase, Spark, Hive, Snowflake. Agile Development experience. SDLC Tools like Maven, Jenkins, Git, etc. Testing tools...Big data
- Job Title: Big Data Engineer (Java,Spark, Hadoop, Ozone CH) Location: Berkeley Heights, NJ Role Overview: We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop...Big dataWork at office
- ...Job Description Position: Data Architect Onsite: 3 days a week... ...Architect, you will be responsible for big data and analytics projects... ...Expertise in Synapse, Presto, Spark SQL, Druid, and SuperSet.... ...warehouses. Knowledgeable in Apache Airflow. Previous experience...Big dataContract workWork experience placement3 days per week
- ...difference at Fiserv.Job TitleSenior Director- Data EngineeringAbout your role:You will be... ...years of experience in data engineering, big data, or software development, including... ....4+ years of hands-on experience with Apache Spark for large-scale distributed data processing...Big dataFull timeTemporary workH1bWork at officeMonday to Friday
- GCP Senior Big Data Engineer Looking for a Senior Big Data Engineer/Architect on Google Cloud Platform to help strategize, architect... ...be comfortable with Big Data Technologies like Hadoop, Hive, Apache Spark, Oozie and Google Cloud platform. You will work on advanced Apache...Big data
- ...Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We... ..., software engineering, and modern Big Data analytics infrastructure, building scalable... ...EngineeringLeverage Databricks, Apache Spark, or comparable technologies to perform distributed...Big dataLive inWork at officeLocal area
- ...Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve... ...Factory, Snowflake, or Databricks. You have experience with big data tools such as Apache Spark or distributed databases. You have experience designing and...Big dataWork experience placementLive inWork at officeLocal areaRemote work
- ...Job Title : Data Engineer Ab initio Location: Berkeley Height, NJ We are seeking a highly skilled resource to design and implement... ...data validation, data security, integrity, and compliance across big data platforms. • Monitor and troubleshoot performance issues...Big data
- ...language. (said that this must have might have to go. Not finding anyone with it. Wants us to let him know, but as long as they have 2 of the 3 he will interview. Nice to Haves: Awareness of Big Data Platforms like Hadoop & Teradata. Know what Yarn/Hive is...Big dataContract work3 days per week
- Sr. Data Engineer Data Engineers who can build scalable, reliable data pipelines and infrastructure, requiring mastery in SQL, Python/Scala, cloud platforms (AWS), and big data technologies (Spark, Kafka, Hadoop). Key skills include data modeling, ETL/ELT pipeline construction...Big data
- ...Job Description Contract Cloud Data Engineer MUST be local to Iselin... ...loading. Develop and deploy Spark/PySpark programs and Databricks jobs for big data workloads. Write robust Python... ...-Haves: Cloud certification(s), Apache Airflow experience, and cloud...Big dataContract workLocal area
- ...degree in computer science Business Administration or equivalent educational or... ...10 years of indepth experience in Data Management Data Warehouse solutions... ...Preferred: Prior experience working in big data technologies like Hadoop Spark Kafka Snowflake certifications SnowPro...Big dataContract work
- ...Description Job Description Job Title: Data Engineer - Data Platforms Location:... ...- 5825048 Required Skills: " Apache Spark & Scala Strong hands-on experience developing... ...experience working with enterprise Big Data platforms and large datasets. " SQL...Big data
- ...Risk management solution. • 3+ years implementing Data ops, ML ops • MS or BS in Computer Science, Information... ..., JSP, Servlet, Node.js, Angular, • Proficient in Big Data Technologies, Data Transport (Pulsar/Kafka), Spark, Jupyter/ Python. • Experience working with...Big data
- NAVA Software solutions is looking for a Sr. Risk Data Analyst for one of our clients. Details: Sr. Risk Data Analyst Location: Berkeley... ...data analytics and visualization tools. Predictive modeling and big data experience. Background in a modeling and testing (champion/...Big dataWork at office
- Data Architect With Dq Location: Iselin, NJ (Hybrid) Job Overview: We are seeking a highly skilled... ...and their application in data analysis. Big Data Technologies: Experience with big data technologies like Hadoop, Spark, etc. Cloud Computing: Familiarity with cloud...Big data
- Data Architect USA - Clark, NJ 07066 Overview GEP is different. It's a place where individuality... ...Provide thought leadership around big data ecosystem and data warehouse design by... ...Guides team members on the development of Spark programs to ingest/process data Qualifications...Big dataLocal area
$58 per hour
...future launches, and business development projects Provide a deep assessment and understanding of where the current landscape of big data stands, from scientific methodologies to capabilities to the various corporations and what is relevant to rare diseases Support...Big dataHourly payApprenticeshipSummer workInternshipSummer internshipH1bWork at officeWorldwideRelocationMonday to Friday- We Are:The beginning of a new Data & AI decade that will reshape work and society has begun. Accenture is... ...Minimum of 3 years of hands-on experience on Cloud and Big Data technologies such as serverless data processing, Spark/SparkSQL, batch and streaming services, event-...Big dataWork experience placementLive inWork at officeLocal area
$120.5k - $231k
...you’ll be doing...The Principal Engineer - Data Science joins the high-visibility Impact... ...structured and unstructured datasets across big data systems using SQL and Python.Advanced... ....Experience with Big Data frameworks (Spark, Splunk, Hadoop) and cloud analytics platforms...Big dataFull timeTemporary workPart timeWork experience placementWork at officeWork from homeShift work3 days per week- ...strength in technology and leadership in cloud, AI and data with unmatched industry experience, functional expertise... ...AI, Anthropic, AWS, Azure, Google Cloud).Knowledge of big data technologies (e.g., Hadoop, Spark).Knowledge of agent-based modeling, reinforcement learning...Big dataLive inWork at officeLocal area
- ...s biggest business challenges. As a Data Scientist, you’ll apply innovative statistical... ...distributed processing to scale to big data + Minimum of 5 years of experience... ...distributed data processing pipelines (Apache Spark or Apache Beam) + Ability to effectively...Big dataHourly payLive inWork at officeLocal areaFlexible hours
- ...-oriented and organized HR Specialist to assist with various administrative tasks. The ideal candidate will be responsible for handling mail... ...-oriented and able to manage multiple tasks simultaneously. Required Skills : Oracle Additional Skills : Big Data...Big dataContract workWork at officeRemote workMonday to Friday
- ...Hadoop Developer / Data Engineer / Big Data Developer Iselin, NJ 08830 - Onsite 1 Day a Week! 12 + Months At least 1 day onsite a week Requirements: Under the leadership of the Data Management and Analytics organization, Data Engineer is responsible for delivering...Big data1 day per week
- We Are: The beginning of a new Data & AI decade that will reshape work and society has begun. Accenture... ...Minimum of 3 years of hands-on experience on Cloud and Big Data technologies such as serverless data processing, Spark/SparkSQL, batch and streaming services, event-...Big dataWork experience placementLive inWork at officeLocal area
$110.7k - $218.3k
...commerce, sales, and service. We are a team of strategists, data scientists, operators, creatives, designers, engineers,... ...(Python, Java, or Bash scripting)Experience with Big Data technologies (i.e., Hadoop, Spark, Redshift, Snowflake, Hive, etc.)Experience with Martech...Big dataLocal areaVisa sponsorship$95.01k - $98k
Hadoop Administrator Softstandard Solutions LLC, a Green Brook Township, NJ-based IT Solutions Firm... ...Linux RedHat servers. Performing various data operations on Hadoop environment.... ...platforms that support client's vision of Big Data. Working with Cloudera support to maintain...Big dataRelocation- Data Engineer Choosing Capgemini means choosing a company where you will be empowered to... ...working with cloud-based data platforms and big data technologies. Knowledge of data... ...lakehouse architectures. Familiarity with Spark, Databricks, Snowflake, or similar data technologies...Big dataPermanent employmentFull timeContract workLocal area
- ...initiatives across the enterprise with focus on network design, big data platform, infrastructure as a service, etc. Provide high... ...escalation for L1 and L2 respectively. Manage the day to day administration of VMware and vBlock environment including system capacity and...Big dataWork experience placement
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Apache Spark / Big Data Administrator. Be the first to apply!





