Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Big data platform engineer

$68 - $70 per hour

Randstad

job summary: Seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage, support, and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark, HDFS, and Kafka is required and will be the primary focus of this role. Experience administering these technologies within a Cloudera environment is preferred but secondary to strong expertise in the underlying technologies. location: Jersey City, New Jersey job type: Contract salary: $68 - 70 per hour work hours: 8am to 5pm education: Bachelors responsibilities:

  • Spark Administration: Deploy, configure, monitor, tune, and troubleshoot Apache Spark environments and workloads. Support Spark job performance, resource management, capacity planning, and production operations.
  • HDFS Administration: Manage and support HDFS storage, including capacity, performance, availability, permissions, replication, and troubleshooting. Strong understanding of HDFS architecture and integration with distributed compute platforms is required.
  • Kafka Administration: Deploy, configure, manage, monitor, and troubleshoot Apache Kafka clusters, topics, producers, and consumers. Support performance, capacity, availability, security, and production operations.
  • Monitoring and Performance Tuning: Monitor platform connectivity, health, capacity, and performance using tools such as Grafana, Splunk, and relevant platform management tools. Proactively identify and resolve performance bottlenecks across Spark, HDFS, and Kafka.
  • Security and Compliance: Implement and manage security configurations, including Kerberos, SSL/TLS, authorization policies, HDFS ACLs, and enterprise IAM/LDAP integrations.
  • Automation and Scripting: Develop and maintain automation using Python, Bash, or Perl for routine administration, configuration management, monitoring, and operational support.
  • Troubleshooting and Support: Provide expert-level troubleshooting and technical support for Spark, HDFS, Kafka, and related big data technologies, including participation in an on-call rotation.
  • Platform Administration: Support installations, patching, upgrades, migrations, and platform lifecycle activities while minimizing downtime. Cloudera Hadoop/CDP administration will be part of the role but is secondary to expertise in Spark, HDFS, and Kafka.
qualifications:
  • Excellent problem-solving, analytical, and communication skills.
  • Minimum 3 years of hands-on production experience administering and supporting Apache Spark, HDFS, and Kafka.
  • Strong, demonstrable expertise in Spark administration, configuration, performance tuning, and troubleshooting.
  • Strong, demonstrable expertise in HDFS administration, storage management, security, performance, and troubleshooting.
  • Strong, demonstrable expertise in Kafka administration, topic management, producer/consumer architecture, performance, security, and troubleshooting.
  • Experience with Linux operating systems and command-line tools.
  • Experience with enterprise monitoring and operational tooling such as Grafana and Splunk.
  • Experience with Cloudera Hadoop/CDP and Cloudera Manager is preferred but is not the primary skill requirement.
skills: Hadoop,Kafka,Apache Kafka,Spark,Apache Spark,Bash,big data platforms,big data technologies,big data,Cloudera,Cloudera Manager,command-line tools,configuration management,storage management,distributed data,Grafana,HDFS,IAM,Kerberos,LDAP,Linux operating systems,enterprise monitoring,Performance Tuning,Perl,Python,Splunk,integrations,SSL/TLS,analytical,communication skills,troubleshoot,Troubleshooting,problem-solving,Proactively,Administration,ACLs,architecture,Automation and Scripting,automation,capacity planning,big data,security concepts,installations,operational support,production operations,resource management,security,security configurations,technical support,tooling Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status. At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact View email address on click.appcast.io. Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility). This posting is open for thirty (30) days. Randstad

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Big data platform engineer in Jersey City, NJ vacancy
  • Spark, HDFS, And Kafka Administrator/Engineer Seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage, support, and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark, HDFS, and Kafka is required and will be the... 
    Big data

    Randstad

    Jersey City, NJ
    2 days ago
  • Big Data Platform Engineer - Jersey City, NJ Jersey City, NJ  Seeking an experienced Spark, HDFS, and Kafka Administrator/Engineer to manage, support, and optimize enterprise-grade big data platforms. Deep hands-on expertise with Apache Spark, HDFS, and Kafka... 
    Big data

    hyrusinc

    Jersey City, NJ
    11 days ago
  •  ...Job Description Job Description Title: Big Data Platform Engineer. Location: Jersey City, NJ (Onsite). Type: Contract, 12 months. Overview: ZipStaff is hiring a Big Data Platform Engineer for our client, a leading global financial services firm. This... 
    Big data
    Full time
    Contract work
    Local area

    ZipStaff Inc.

    Jersey City, NJ
    12 days ago
  •  ...Technology Enabled Services.Job DescriptionJob Description:-Our Big Data capability team needs hands-on developers who can produce...  ...a strong advantage....Operating knowledge of cloud computing platforms (AWS, especially EMR, EC2, S3, SWF services and the AWS CLI)Experience... 
    Big data
    Permanent employment
    Full time
    H1b

    Sonsoft

    Jersey City, NJ
    4 days ago
  • Job Title: Data Engineer with GCP Location: New Jersey (Local Prefers) Employment Type: Contract...  ...Data Engineer with strong Google Cloud Platform (GCP) experience to design, build, and...  ...candidate will have hands-on expertise in Big Data technologies, cloud-native... 
    Big data
    Contract work
    Local area

    AceStack LLC

    Jersey City, NJ
    2 days ago
  • Role description We are seeking a skilled Data Engineer with 5-7 years of experience in...  ...maintaining scalable data pipelines and data platforms The ideal candidate should have strong expertise...  ...warehousing cloud technologies SQL and big data ecosystems The role involves... 
    Big data

    LTM

    Jersey City, NJ
    2 days ago
  •  ...Full TimeCompany: SRI Tech SolutionsRole: Data Engineer - Python/PySparkLocation: Jersey City,...  ...building scalable ETL/ELT pipelines in big data environments- Strong understanding...  ...experience with Spark, Hadoop, Kafka, and cloud platforms- Solid knowledge of SQL and NoSQL... 
    Big data
    Full time
    3 days per week

    SRI Tech

    Jersey City, NJ
    4 days ago
  • Data Engineer Location: Jersey City, NJ (Onsite)/Houston, TX (On-site) Duration: 12 Months (Contract with possible conversion) W2 Candidates...  ...expertise in Spark, SQL, and building data pipelines (preferably in Big Data technologies). Basic knowledge of Java and Spring... 
    Big data
    Contract work

    Kaav Inc.

    Jersey City, NJ
    2 days ago
  • Data Engineer (Insurance) Job Id: 98054 Location: Jersey City, NJ local to the location Must be a U.S. Military Veteran USC Only Key Skills...  ..., Spark, AWS (Glue, EMR, S3, Aurora, RDS, AWS Architecture), Big Data Concepts Insurance Domain Knowledge (Preferred) Must Have... 
    Big data
    Local area

    3B Staffing LLC

    Jersey City, NJ
    2 days ago
  • $82.42k - $126.6k

     ...specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help our clients accelerate growth and unlock...  ...with Hadoop ecosystem (Cloudera). • Experience in working with a Big Data implementation in production environment. • Experience with... 
    Big data
    Full time
    Temporary work
    Relocation

    Infosys Technologies

    Jersey City, NJ
    3 days ago
  •  ...We are looking for an experienced Big Data Developer with strong expertise in big data technologies, distributed data processing, and large-scale data analysis. The ideal candidate should have hands-on experience with Kafka, Spark, Scala, SQL, Hadoop, Hive, and Impala... 
    Big data
    Full time

    Techvilla Solutions

    Jersey City, NJ
    a month ago
  •  ...Artificial Intelligence, Generative AI, Data Science, Data Engineering and Advanced Analytics to solve their...  ...automation, and cloud-native data platforms. A global presence across North America...  .... Work with Hadoop and related big data technologies where applicable.... 
    Big data
    Full time
    Temporary work
    Local area

    Tiger Analytics Inc.

    Jersey City, NJ
    3 days ago
  • Senior Big Data Engineer Location: New Jersey (hybrid) Duration: 6+ Months We are seeking a highly skilled and experienced Senior Data Engineer...  ...such as Flask or Spring. Familiarity with cloud-based data platforms (e.g.Azure) Experience with data warehousing concepts and... 
    Big data

    Zortech Solutions

    Jersey City, NJ
    2 days ago
  • Senior AWS Data Engineer Location: New Jersey - Onsite Duration: Fulltime Job Description: Key...  ...using Docker within container orchestration platforms such as AWS ECS, EKS, Google Anthos, or...  ...NACLs, and SCPs Familiar with ETL and big data tool-chains such as those provided... 
    Big data
    Full time

    Zortech Solutions

    Jersey City, NJ
    2 days ago
  •  ...Description We are seeking a talented AWS Data Engineer to join our dynamic Data Engineering...  ...implement CI/CD strategies for enterprise data platforms. Collaborate with product, operations,...  ...of ETL/ELT, data modeling, and big data technologies. Familiarity with data... 
    Big data

    EXL

    Jersey City, NJ
    1 day ago
  • Job Description The Data/DataOps Engineer II is a junior-to-mid level position that will provide data pipeline, orchestration, and platform operations expertise to guide the development, deployment...  ...like you to help us translate big data into big ideas. Join us and create... 
    Big data

    Verisk Analytics

    Jersey City, NJ
    2 days ago
  • Hi, Position: Lead Data Engineer Experience Required: 12 years+ Location: Jersey City, NJ |...  ...ETL/ELT frameworks for cloud-based data platforms. Drive development, code reviews, unit testing...  ...-scale projects. Strong expertise in Big Data concepts , distributed systems, and... 
    Big data
    Full time

    Centraprise

    Jersey City, NJ
    2 days ago
  • About the job Lead Data Engineer Job Title: Lead Data Engineer No of Positions:2 Location: Jersey City/ Bedford, NJ (Hybrid) (3 days from office...  ...Key Skills: Snowflake, SQL, Python, Spark, AWS-Glue, Big Data Concepts Responsibilities: Lead the design, development, and... 
    Big data
    Work experience placement
    Work at office
    3 days per week

    Inizio Partners

    Jersey City, NJ
    2 days ago
  • Lead Data Engineer (Forward Deployed) As a Lead Data Engineer (Forward Deployed) at JPMorganChase within the Infrastructure Data Platforms team, you will embed with infrastructure product teams to turn...  ...detection, pattern analysis, or big data frameworks such as Hadoop/Spark... 
    Big data
    Contract work
    Work at office

    Chase

    Jersey City, NJ
    2 days ago
  • Lead Data Engineer-Python/SQL/Snowflake Join us as we embark on a journey of collaboration and...  ...operate secure, stable and scalable data platforms. As a hands-on technical contributor on...  ...capabilities, and skills Hands-on experience with big data technologies and large-scale data... 
    Big data

    Hackajob

    Jersey City, NJ
    7 hours ago
  • $102.5k - $210.6k

     ...Role Overview: As an Applied AI Platform Engineer III, you will actively engage in your engineering...  ...systems, and modern AI/ML and Data infrastructure, consistently demonstrating...  ...adoption-validated increments rather than big-bang builds, and keeping them supportable... 
    Big data
    Visa sponsorship
    Flexible hours

    Deloitte

    Jersey City, NJ
    1 day ago
  •  ...Hybrid work environment in Newark, NJ. Data Engineer (AWS) Job Summary: We are seeking a...  ...CI/CD strategies for enterprise data platforms. Collaborate with product, operations...  ...of ETL/ELT, data modeling, and big data technologies. ~ Familiarity with... 
    Big data
    Contract work

    JSR Tech Consulting

    Newark, NJ
    9 days ago
  •  ...DescriptionJob Description Mandatory skills: Eligible to workP, Big Query, Dataflow, Looker, and Python Key Responsibilities: We are seeking a skilled Data Analytics Engineer with strong experience in Google Cloud Platform (Eligible to workP) and expertise across modern data... 
    Big data

    Diverse Lynx

    Secaucus, NJ
    2 days ago
  • Lead Software Engineer Join AIML Data Platforms to help shape the future of AI/ML by building enterprise-scale data platforms that drive innovation across the firm. As a Lead Software Engineer at JPMorganChase within the AI/ML Data Platforms (AMDP) - Data Governance Engineering... 

    Hackajob

    Jersey City, NJ
    2 days ago
  • $130.9k - $268.7k

     ...As an Associate Director, Applied AI Platform Engineering, you will set the engineering vision and...  ...frameworks, tooling, control plane, and data platform that every other engineering team...  ...adoption-validated increments rather than big-bang builds. Apply a leaning-forward... 
    Big data
    Work at office
    Local area
    Visa sponsorship
    3 days per week

    Deloitte

    Jersey City, NJ
    17 hours ago
  •  ...Job Title: Lead Python Engineer – Big Data & Risk Analytics Location: Jersey City, NJ Duration: Contract - 6 months (C2H potential)...  ...design and operate high-performance, auditable aggregation platforms that deliver reportable risk metrics . Responsibilities:... 
    Big data
    Contract work

    TSR Consulting

    Jersey City, NJ
    5 days ago
  • Data Engineer Role: Data Engineer Required legal status: Green Card, Green Card-EAD, US Citizen Location: Charlotte, NC (Hybrid work model)...  ...Scala programming language Working with Spark and PySpark for big data processing Utilizing Hadoop for data storage and processing... 
    Big data

    Zortech Solutions

    Passaic, NJ
    2 days ago
  • $120k - $150k

     ...Role: AWS Analytics Engineer Location: Newark, NJ (Hybrid: 3 days onsite...  ...Cloud Engineer within the Data Processing team and will have...  ...ETL, distributed compute, and big data frameworks such as Spark,...  ...log analytics and observability platforms (Splunk, ELK, Prometheus,... 
    Big data
    Work at office

    EXL

    Newark, NJ
    2 days ago
  • $116.8k - $208.3k

     ...Overview Are you an experienced, passionate pioneer in technology who wants to work in a collaborative environment? As a Data Platform Engineering Manager, you will have the ability to share new ideas and collaborate on high-impact projects supporting mission-critical... 
    Remote work
    Flexible hours

    Deloitte

    Jersey City, NJ
    1 day ago
  • Data Engineer Position Looking for data engineers with 9+ years of experience. Experience developing and deploying application code using SQL...  ...developing Linux, UNIX shell scripts. Developer experience in big data or warehouse (Hadoop, DB2 LUW) environment. Experience with... 
    Big data

    Cedent Life Talent

    Newark, NJ
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big data platform engineer. Be the first to apply!