Hadoop / Big Data Platform Engineer
Shree Narayani Networking Solutions Pvt Ltd
Job Summary (List Format):
- Serve as a subject matter expert (SME) in Hadoop/Big Data platform engineering, focusing on Cloudera Hadoop administration and related technologies (HDFS, YARN, Spark, Impala, Kafka, Linux/Unix).
- Manage and administer Hadoop clusters, including production support, troubleshooting, upgrades, monitoring, and performance tuning.
- Support NextGen Big Data platforms (Hadoop, Spark, Kafka, Impala, HBase, Docker, Ansible, etc.), including cluster management for Cloudera and Databricks.
- Provide end-to-end platform operations across all environments (upgrades, bug fixes, deployments, metrics/monitoring, disaster recovery, incident/problem/capacity management).
- Act as a technical liaison between clients, vendors, and project managers, offering solutions that meet user needs.
- Deliver L1/L2/L3 level support for the Hadoop platform and resolve issues related to job execution, NameNode, HDFS space, permissions, and backups.
- Analyze, debug, and troubleshoot Yarn, Spark, and Impala jobs; perform query and resource utilization tuning.
- Maintain strong technical knowledge of Hadoop architecture, relational databases, and data warehousing (SQL skills).
- Ensure integration with other applications and platforms, including authentication (AD/Kerberos), job scheduling (Autosys), and monitoring systems.
- Hands-on expertise with Cloudera Hadoop components (HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, Postgres).
- Utilize DevOps and automation tools (Ansible, Jenkins, SVN, Bitbucket) and scripting languages (Java, Python, Perl, Shell).
- Support ETL tools from the Hadoop stack and BI tools such as Tableau, including dashboard creation connecting Hive tables.
- Participate in agile development processes, collaborating with cross-functional teams at high technical levels.
- Preferred certifications: Cloudera Admin/Dev, Cloud, Docker-Container, OpenShift.
- Hybrid role based in Chicago, IL or Charlotte, NC (3 days onsite, 2 days remote); 4+ months contract.
- Shift timings: 8AM 5PM or 12:30PM ET 9:30PM ET.
Note: Strong experience with Kerberos/AD, Ansible, Jenkins, Docker/OpenShift, and Big Data technologies is highly preferred.
- Serve as a subject matter expert (SME) in Hadoop/Big Data platform engineering, focusing on Cloudera Hadoop administration and related technologies (HDFS, YARN, Spark, Impala, Kafka, Linux/Unix).
- Manage and administer Hadoop clusters, including production support, troubleshooting, upgrades, monitoring, and performance tuning.
- Support NextGen Big Data platforms (Hadoop, Spark, Kafka, Impala, HBase, Docker, Ansible, etc.), including cluster management for Cloudera and Databricks.
- Provide end-to-end platform operations across all environments (upgrades, bug fixes, deployments, metrics/monitoring, disaster recovery, incident/problem/capacity management).
- Act as a technical liaison between clients, vendors, and project managers, offering solutions that meet user needs.
- Deliver L1/L2/L3 level support for the Hadoop platform and resolve issues related to job execution, NameNode, HDFS space, permissions, and backups.
- Analyze, debug, and troubleshoot Yarn, Spark, and Impala jobs; perform query and resource utilization tuning.
- Maintain strong technical knowledge of Hadoop architecture, relational databases, and data warehousing (SQL skills).
- Ensure integration with other applications and platforms, including authentication (AD/Kerberos), job scheduling (Autosys), and monitoring systems.
- Hands-on expertise with Cloudera Hadoop components (HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, Postgres).
- Utilize DevOps and automation tools (Ansible, Jenkins, SVN, Bitbucket) and scripting languages (Java, Python, Perl, Shell).
- Support ETL tools from the Hadoop stack and BI tools such as Tableau, including dashboard creation connecting Hive tables.
- Participate in agile development processes, collaborating with cross-functional teams at high technical levels.
- Preferred certifications: Cloudera Admin/Dev, Cloud, Docker-Container, OpenShift.
- Hybrid role based in Chicago, IL or Charlotte, NC (3 days onsite, 2 days remote); 4+ months contract.
- Shift timings: 8AM 5PM or 12:30PM ET 9:30PM ET.
Note: Strong experience with Kerberos/AD, Ansible, Jenkins, Docker/OpenShift, and Big Data technologies is highly preferred.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Hadoop / Big Data Platform Engineer in Chicago, IL vacancy
- ManpowerGroup Global, Inc. is seeking a Hadoop Engineer for Chicago or Charlotte to join the Data Platforms Support Team. You will manage and optimize Big Data platforms built around Hadoop, Spark, Kafka, and Impala, supporting data infrastructure and performance. The...Big data
- ...solutions company is looking for a skilled Hadoop Admin in Chicago, IL. The role involves... ...Hadoop infrastructure, optimizing big data solutions, and ensuring comprehensive security... ...flavors like Cloudera and manage cloud platforms such as Azure or AWS. Additional...Big data
$110k - $125k
...Big Data EngineerWe are seeking an experienced Big Data Engineer to design, develop, and optimize enterprise-scale big data solutions... ...hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive,... ...and governance across Big Data platforms.Participate in architecture discussions...Big dataSelect Minds LLC
Chicago, IL3 days ago$175k - $225k
...We are seeking a dedicated and experienced Data Engineer to join our Chicago team. The ideal candidate is energized by working... ...Responsibilities Architect, develop and deploy our Big Data environment (Kafka, Hadoop, Dremio, etc.) Build, deploy, and monitor our data processing...Big dataPermanent employmentFull timeittihad medical centre
Chicago, IL4 days ago- ...As a Senior Data Engineer, you will have the opportunity to build solutions that ingest, transform, store, and distribute our big data to be consumed by data scientists and our products. Our... ...engineers use PySpark/Python, Databricks, Hadoop, Hive, and other data engineering...Big data
84.51
Chicago, IL3 days ago $179.4k - $225.1k
Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy... ...environment? At Capital One, you'll be part of a big group of makers, breakers, doers and... ...data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL...Big dataFull timePart timeInternshipH1bLocal area$99k - $232k
...& SummaryAt PwC, our people in data and analytics engineering focus on leveraging advanced technologies... ...Apart- Certification in Cloud Platforms [e.g., AWS Solutions Architect,... ..., or Scala- Familiarity with big data technologies like Hadoop, Spark, or Kafka is a plus-...Big dataFull timeH1b- 84.51 in Chicago is seeking a Senior Data Engineer to build solutions for big data that will be utilized by data scientists and various products. You will work with tools like PySpark, Databricks, and Hadoop to deliver vital data capabilities. Key responsibilities include...Big data
- ...Applied Statistics, Electrical Engineering, Mechanical Engineering or... ...very good experience of data tuning on Green Plum or other... ...technologies. Expertise on big data technologies such as HDFS... ...datasets using Big Data tools and platforms (Hadoop, PIG/HIVE/Mahout) is highly...Big dataFull time
- ittihad medical centre is looking for a dedicated Data Engineer to join our Chicago team. The ideal candidate will manage the deployment of our Big Data environment and the data processing pipelines, ensuring data integrity and availability. The role requires more than...Big data
$179.4k - $204.7k
...Lead Data EngineerDo you love building and pioneering in the technology... ...One, you'll be part of a big group of makers, breakers,... ...customer needs. We are seeking Data Engineers who are passionate about... ...data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi...Big dataFull timePart timeInternshipLocal areaCapital One
Chicago, IL18 hours ago- ...leading technology consultancy is looking for a Senior Platform Engineer–Administrator to oversee data and analytics platforms. This role involves... ...have extensive experience in platform administration and big data technologies like Spark and Python. Attractive benefits...Big dataFlexible hours
Sogeti
Chicago, IL17 hours ago - ...position requires extensive data and integration experience.... ...software solutions in Hadoop cluster with extensive experience... ...Data Integration, Talend Big Data migration platform Edition 6.2.1 or comparable... ...unexpected data, etc., as well as engineering the code base for...Big dataWork experience placement
Samprasoft
Chicago, IL3 days ago - ...Adyen provides payments, data, and financial products... ...the financial technology platform of choice. At Adyen, everything we do is engineered for ambition. For our... ...data pipelines on our Big Data Platform. As a Data... ...Python, PySpark, Airflow, Hadoop, Spark, Kafka, Druid,...Big dataBank staffWork at office
TechBrains
Chicago, IL16 hours ago $154k - $170k
...company in the middle of a platform transformation, moving... ...time. Groupon's data infrastructure... ...applications, databases, and big data platforms (e.g., BigQuery, Bigtable, Hadoop, Google Dataproc, Redis... ...; Partner with data engineering and analytics teams to...Big dataWork at officeLocal areaGroupon
Chicago, ILmore than 2 months ago- ...solutions company. Our capabilities range from Data Engineering, Visualization, Data Management to Advanced analytics, Big Data, Cloud and Machine Learning. Our... ...Python, C++, Java, R, and SQL. Familiarity with Hadoop or suitable equivalent. Excellent analytical...Big data
$175.8k - $200.7k
Lead Data Engineer at Capital One - Chicago, IL. Do you love building and pioneering in the technology... ...? At Capital One, you’ll be part of a big group of makers, breakers, doers and... ...data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL...Big dataInternshipH1bLocal area- ...Job TitleOnsite/hybrid***4-6 years of experience in Big Data development using Hadoop with a good understanding in all phases of software development life cycle.***Experience using Hadoop Technologies like Hive, Spark, Pig, or Kafka***Experience using Talend***Participate...Big data
Samprasoft
Chicago, IL3 days ago $73.5k - $136.5k
...to ingest new complex data quality use cases. In both... ...with a passion for big data technologies. Responsibilities... .... Work closely with Engineering and Business resources... ...—including Kafka, Hadoop, HDFS, Spark, and... ...distributed systems. Our platform includes Kafka, Spark,...Big dataLocal areaFlexible hours- ...motivated candidate to join our talented Team. Job Title: Sr. Data Engineer Location(s): Chicago, IL (Hybrid) Key Skills: AWS (EMR,... ...extensive experience in data engineering, cloud services, and big data technologies, with a strong focus on AWS services, Python...Big dataLocal areaRemote work
- ...and development experience At least 4+ years of working on data management domain with big data, Oracle, SQL or other database solutionsAt least 2+... ...of experience with Big Data & Analytics solutions - Hadoop, MapReduce, Pig, Hive, Spark, Storm, Amazon Kinesis, AWS...Big dataPermanent employmentFull timeH1b
- A consulting firm is seeking a Big Data consultant with expertise in Hadoop, Cloudera, Spark, and Kafka. This is a 3+ month assignment located in Chicago, IL, requiring local candidates who can attend in-person interviews. Ideal candidates should have a proven background...Big dataLocal area
$98k - $182k
...marketing firm in Chicago seeks an experienced Data Engineer to build and maintain robust data... ...will have over 5 years of experience in big data environments, utilizing technologies... ...to the world's largest advertising platforms. This role offers a competitive salary range...Big dataPublicis Groupe ANZ
Chicago, IL3 days ago- ...Sr. Data EngineerThis role is responsible for providing technical... ...design and deliver end-to-end data engineering solutions to support advanced... ...across the advanced analytics platform.Design, develop and... ...perform Spark optimizations for Big Data use cases.Design, develop...Big dataFor contractorsRemote work
1872 Consulting
Oak Brook, IL7 hours ago $150k - $200k
...software systems are continually re-engineered, optimized, and maintained to... ...looking for a proven Senior Data Engineer to work directly with... ...designing and developing big data warehouses and ETL pipelines... ...data across multiple data platforms, including RDBMS, NoSQL, and Time...Big dataFull time- ...Senior Data EngineerChicago, ILONSITE FROM DAY 1We need a STRONG... ...(CI/CD)Senior-level software engineer or data engineer with experience developing and deploying big data analytics applications using... ...using Databricks on the Azure Platform.Contribute to the development...Big data
Zortech Solutions
Chicago, IL3 days ago - ...investing heavily in the next generation of its data platform. As data volumes continue to grow across... ...and trading, they're expanding the engineering team responsible for building the... ...distributed data pipelines using modern Big Data technologies Improve performance, reliability...Big data
Fintal Partners
Chicago, IL4 days ago - ...Big Data Optimization ProjectBig Data optimization project to optimize the data flow.Job ResponsibilitiesTest programs or databases, correct... ...’s Contributions to Work GroupThe main function of a data engineer is to ensure that the data assets of an organization are supported...Big dataWork experience placement
Software Technology Inc
Chicago, IL3 days ago - ...are looking for a candidate working on Big Data Technologies, who has Extensive experience... ...big data technologies like Map reduce, Hadoop, Hive, Spark, Scala, Flume, Sqoop, HBase... ...depth experience in implementing Big data platforms on one of the leading distributions (...Big dataPermanent employmentFull timeWork experience placementH1b
- ...Hadoop Engineer (Chicago or Charlotte)Our client, a leading organization in the technology and data solutions sector, is seeking a Hadoop Engineer (Chicago... ...will be part of the Data Platforms Support Team, contributing... ...Job?Support and manage Big Data platforms built around...Big dataLong term contractFlexible hours
Experis
Chicago, IL18 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Hadoop / Big Data Platform Engineer. Be the first to apply!
Related searches
- junior data engineer remote Chicago, IL
- director data engineering Chicago, IL
- hadoop big data developer Chicago, IL
- data engineer intern Chicago, IL
- senior data quality engineer Chicago, IL
- senior cloud data engineer Chicago, IL
- data engineer contract Chicago, IL
- data infrastructure engineer Chicago, IL
- entry level big data engineer Chicago, IL
- data science developer Chicago, IL