Hadoop / Big Data Platform Engineer
Shree Narayani Networking Solutions Pvt Ltd
Job Summary (List Format):
- Serve as a subject matter expert (SME) in Hadoop/Big Data platform engineering, focusing on Cloudera Hadoop administration and related technologies (HDFS, YARN, Spark, Impala, Kafka, Linux/Unix).
- Manage and administer Hadoop clusters, including production support, troubleshooting, upgrades, monitoring, and performance tuning.
- Support NextGen Big Data platforms (Hadoop, Spark, Kafka, Impala, HBase, Docker, Ansible, etc.), including cluster management for Cloudera and Databricks.
- Provide end-to-end platform operations across all environments (upgrades, bug fixes, deployments, metrics/monitoring, disaster recovery, incident/problem/capacity management).
- Act as a technical liaison between clients, vendors, and project managers, offering solutions that meet user needs.
- Deliver L1/L2/L3 level support for the Hadoop platform and resolve issues related to job execution, NameNode, HDFS space, permissions, and backups.
- Analyze, debug, and troubleshoot Yarn, Spark, and Impala jobs; perform query and resource utilization tuning.
- Maintain strong technical knowledge of Hadoop architecture, relational databases, and data warehousing (SQL skills).
- Ensure integration with other applications and platforms, including authentication (AD/Kerberos), job scheduling (Autosys), and monitoring systems.
- Hands-on expertise with Cloudera Hadoop components (HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, Postgres).
- Utilize DevOps and automation tools (Ansible, Jenkins, SVN, Bitbucket) and scripting languages (Java, Python, Perl, Shell).
- Support ETL tools from the Hadoop stack and BI tools such as Tableau, including dashboard creation connecting Hive tables.
- Participate in agile development processes, collaborating with cross-functional teams at high technical levels.
- Preferred certifications: Cloudera Admin/Dev, Cloud, Docker-Container, OpenShift.
- Hybrid role based in Chicago, IL or Charlotte, NC (3 days onsite, 2 days remote); 4+ months contract.
- Shift timings: 8AM 5PM or 12:30PM ET 9:30PM ET.
Note: Strong experience with Kerberos/AD, Ansible, Jenkins, Docker/OpenShift, and Big Data technologies is highly preferred.
- Serve as a subject matter expert (SME) in Hadoop/Big Data platform engineering, focusing on Cloudera Hadoop administration and related technologies (HDFS, YARN, Spark, Impala, Kafka, Linux/Unix).
- Manage and administer Hadoop clusters, including production support, troubleshooting, upgrades, monitoring, and performance tuning.
- Support NextGen Big Data platforms (Hadoop, Spark, Kafka, Impala, HBase, Docker, Ansible, etc.), including cluster management for Cloudera and Databricks.
- Provide end-to-end platform operations across all environments (upgrades, bug fixes, deployments, metrics/monitoring, disaster recovery, incident/problem/capacity management).
- Act as a technical liaison between clients, vendors, and project managers, offering solutions that meet user needs.
- Deliver L1/L2/L3 level support for the Hadoop platform and resolve issues related to job execution, NameNode, HDFS space, permissions, and backups.
- Analyze, debug, and troubleshoot Yarn, Spark, and Impala jobs; perform query and resource utilization tuning.
- Maintain strong technical knowledge of Hadoop architecture, relational databases, and data warehousing (SQL skills).
- Ensure integration with other applications and platforms, including authentication (AD/Kerberos), job scheduling (Autosys), and monitoring systems.
- Hands-on expertise with Cloudera Hadoop components (HDFS, Sentry, HBase, Kafka, Impala, SOLR, Hue, Spark, Hive, YARN, Zookeeper, Postgres).
- Utilize DevOps and automation tools (Ansible, Jenkins, SVN, Bitbucket) and scripting languages (Java, Python, Perl, Shell).
- Support ETL tools from the Hadoop stack and BI tools such as Tableau, including dashboard creation connecting Hive tables.
- Participate in agile development processes, collaborating with cross-functional teams at high technical levels.
- Preferred certifications: Cloudera Admin/Dev, Cloud, Docker-Container, OpenShift.
- Hybrid role based in Chicago, IL or Charlotte, NC (3 days onsite, 2 days remote); 4+ months contract.
- Shift timings: 8AM 5PM or 12:30PM ET 9:30PM ET.
Note: Strong experience with Kerberos/AD, Ansible, Jenkins, Docker/OpenShift, and Big Data technologies is highly preferred.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Hadoop / Big Data Platform Engineer in Chicago, IL vacancy
$110k - $125k
...Big Data Engineer We are seeking an experienced Big Data Engineer to design, develop, and optimize... ...have strong hands-on expertise in the Hadoop ecosystem, Spark, Scala, Hive, Apache... ..., and governance across Big Data platforms. Participate in architecture discussions...Big dataSelect Minds LLC
Chicago, IL1 day ago$175k - $225k
...Data Engineer Chicago, United States We are seeking a dedicated and experienced Data Engineer to join our Chicago team... ...Responsibilities Architect, develop and deploy our Big Data environment (Kafka, Hadoop, Dremio, etc.) Build, deploy, and monitor our data...Big dataPermanent employmentFull timeIMC Inc
Chicago, IL2 days ago- ...Role : Sr Data Engineer Location : Chicago IL (look for Nearby candidates) Hybrid... ...have: Azure, Databricks, ETL, Python or Hadoop, INFORMATICA or Abinitio, Data lake, SQL... ...• Strong understanding of RDBMS, NoSQL, big data, SQL, and ETL tools. •...Big dataWork at officeWork from home
Concord IT Systems
Chicago, IL2 days ago - ...Data Engineer (NO OPT) Visa Status: U.S. Citizens and those authorized to work in the U.S. are encouraged to apply. Tax Terms:... ...object function scripting languages: Python. Experience with big data tools: Hadoop, Apache Spark, Kafka, etc. Experience with AWS cloud...Big dataWork experience placement
Keylent Inc
Chicago, IL1 day ago $46.07 - $68.64 per hour
...and industry specific market data. The pay range for each role... .... Summary: The Data Engineer is responsible for designing... ...implementing analysis solutions on Hadoop-based platforms such as Cloudera Hadoop, or... ..., and manages multiple big data sources. • Builds data...Big dataHourly payFull timeRemote workShift workRush University
Chicago, IL2 days ago- ...Data Engineer - I The Data Engineer - I will be responsible for building and maintaining... ...data sources using SQL and AWS 'big data' technologies. Build... ...: Apache Spark and PySpark, Hadoop SQL and any NoSQL and RDBMS platforms Python, R, Java, JavaScript PL/...Big data
1872 Consulting
Chicago, IL2 days ago $154k - $170k
...company in the middle of a platform transformation, moving... ...time. Groupon's data infrastructure... ...applications, databases, and big data platforms (e.g., BigQuery, Bigtable, Hadoop, Google Dataproc, Redis... ...; Partner with data engineering and analytics teams to...Big dataWork at officeLocal areaGroupon
Chicago, ILa month ago$99k - $232k
...& SummaryAt PwC, our people in data and analytics engineering focus on leveraging advanced technologies... ...Apart- Certification in Cloud Platforms [e.g., AWS Solutions Architect,... ..., or Scala- Familiarity with big data technologies like Hadoop, Spark, or Kafka is a plus-...Big dataFull timeH1b- ...Applied Statistics, Electrical Engineering, Mechanical Engineering or... ...very good experience of data tuning on Green Plum or other... ...technologies. Expertise on big data technologies such as HDFS... ...datasets using Big Data tools and platforms (Hadoop, PIG/HIVE/Mahout) is highly...Big dataFull time
$209k - $238.5k
...Senior Lead Data Engineer Do you love building and pioneering in the technology space? Do... ...environment? At Capital One, you'll be part of a big group of makers, breakers, doers and... ...data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL...Big dataFull timePart timeInternshipLocal areaCapital One
Chicago, IL5 days ago- ...Google Cloud Data Engineer Are you ready to step up to the New and take... ...technology at Google Cloud Platform (GCP) Data Engineers will be... ...data team to efficiently use Hadoop/Cloud infrastructure to analyze... ...consumption within a hybrid big data architecture, using Java...Big dataWork experience placement
ClifyX
Chicago, IL4 days ago - ...solutions company. Our capabilities range from Data Engineering, Visualization, Data Management to Advanced analytics, Big Data, Cloud and Machine Learning. Our... ...Python, C++, Java, R, and SQL. Familiarity with Hadoop or suitable equivalent. Excellent analytical...Big data
$179.4k - $204.7k
...Lead Data Engineer (Python, AWS, SQL, GenAI) (Enterprise Platforms Technology) Do you love building and pioneering in the technology... ...One, you'll be part of a big group of makers, breakers, doers... .../computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, or MySQL...Big dataFull timePart timeInternshipH1bLocal areaCapital One
Chicago, IL1 day ago- Lead Data Engineer Do you love building and pioneering in the technology... ...One, you'll be part of a big group of makers, breakers, doers... ...this mature, rapidly growing platform, your work will directly influence... .../computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark,...Big dataInternshipH1bLocal area
$73.5k - $136.5k
...to ingest new complex data quality uses cases. In... ...features with a passion for big data technologies.... ...Working closely with Engineering and Business resources... ...ecosystem—including Kafka, Hadoop, HDFS, Spark, and multiple... ...systems. Our platform is ever evolving, but currently...Big dataLocal areaFlexible hours- ...and development experience At least 4+ years of working on data management domain with big data, Oracle, SQL or other database solutionsAt least 2+... ...of experience with Big Data & Analytics solutions - Hadoop, MapReduce, Pig, Hive, Spark, Storm, Amazon Kinesis, AWS...Big dataPermanent employmentFull timeH1b
$150k - $200k
...software systems are continually re-engineered, optimized, and maintained to... ...looking for a proven Senior Data Engineer to work directly with... ...designing and developing big data warehouses and ETL pipelines... ...data across multiple data platforms, including RDBMS, NoSQL, and Time...Big dataFull time- ...Big Data Engineer We are currently seeking a Big Data Engineer for an onsite position located in Charlotte, NC, Denver, CO, Jacksonville, FL, and Chicago, IL. This is a 12+ month W2 contract opportunity. Join a passionate, tight-knit, and supportive team as a Big...Big dataContract work
Redolent
Chicago, IL4 days ago - ...Senior Data Engineer Chicago, IL ONSITE FROM DAY 1 We need a STRONG database candidate who has good... ...with experience developing and deploying big data analytics applications using Databricks on the Azure platform Duties: Contributing member of a team...Big data
Zortech Solutions
Chicago, IL4 days ago - ...Our client, a premier investment bank, is looking for a Big Data Engineer to join their growing team in Chicago, IL. Currently our client spends a lot of time trying to answer client's questions by logging in to several disparate data sources. It is estimated each support...Big dataFor contractors
Liniumit
Chicago, IL1 day ago - ...are looking for a candidate working on Big Data Technologies, who has Extensive experience... ...big data technologies like Map reduce, Hadoop, Hive, Spark, Scala, Flume, Sqoop, HBase... ...depth experience in implementing Big data platforms on one of the leading distributions (...Big dataPermanent employmentFull timeWork experience placementH1b
- ...Senior Data Engineer Location: Chicago, IL or Minneapolis, MN (Hybrid 3 to 4 Days Onsite/Week) Duration... ...| NoSQL | Cloud Architecture | Enterprise Data Platforms | Data Integration | ETL/ELT | AI Enablement | Big Data Note: Must be local with DL...Big dataContract workLocal area3 days per week
Akaasa Technologies
Chicago, IL2 days ago - ...Big Data Optimization Project Big Data optimization project to optimize the data flow. Job Responsibilities Test programs... ...Contributions to Work Group The main function of a data engineer is to ensure that the data assets of an organization are supported...Big dataWork experience placement
Software Technology Inc
Chicago, IL4 days ago $215.2k - $244.7k
...opportunity to work with clients developing Big Data strategy and roadmaps, while gaining... ...Utilities.Responsibilities:Provide guidance and platform selection advice for Databricks,... ...with Large Language Models (LLMs), prompt engineering, embeddings, vector databases, and...Big dataLocal areaImmediate startFlexible hours$130k
...collaboration, cutting-edge technology, data driven solutions, and... ...What you’ll do as a Software Engineer on the Data Engineering team at... ...and expansion of our data platform across a wide variety of data... ...transform our rapidly growing Big Data set within our hybrid cloud...Big dataFull timeInternshipWork at officeAkuna Capital
Chicago, IL10 hours ago$180k - $250k
...this role, you will: Work within a growing Data Engineering division supporting the strategic role... ...design and expansion of our data platform across a wide variety of data sources, supporting... ...and transform our rapidly growing Big Data set within our hybrid cloud architecture...Big dataLeoforce
Chicago, IL1 day ago- ...maintenance and development of an eCommerce platform for a big US-based retail company, serving... ...Customer, Loyalty, Search and Browse, Data Integration, Cart. Current overriding... ...Demonstrated ability to lead cross-functional engineering teams, define technical strategy and...Big data
Luxoft
Chicago, IL1 day ago $98k - $169.05k
...Senior Data Engineer Cincinnati, OH; Chicago, IL 84.51° is a retail data science, insights and media company. We help The Kroger Co.,... ...at Kroger. We use agile development methodology, starting with Big Room Planning bringing everyone into the planning process to build...Big dataWork at office84.51°
Chicago, IL2 days ago- ...ROLE We are looking for a Senior Data Engineer to design and build scalable data lakes... ...architectures supporting a thematic research platform that processes large volumes of... ...DuckDB ; - 2+ years of experience with Big Data technologies ( Spark , Snowflake...Big dataWork at officeRemote workVisa sponsorshipWork visaFlexible hours
AgileEngine
Chicago, IL1 day ago - ...paced market environment where data is central to our competitive... ...organization values high-performance engineering, hybrid-cloud scalability, and... ..., pushing our data platform beyond current tech limits to... ...transform our rapidly growing Big Data set within our hybrid cloud...Big dataLocal area
Jobot
Chicago, IL3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Hadoop / Big Data Platform Engineer. Be the first to apply!
Related searches
- data platform engineer Chicago, IL
- aws data engineer Chicago, IL
- big data devops engineer Chicago, IL
- big data developer Chicago, IL
- software data engineer Chicago, IL
- data center engineer Chicago, IL
- big data cloud engineer Chicago, IL
- director data engineering Chicago, IL
- hadoop big data developer Chicago, IL
- big data engineer Chicago, IL