Big Data DevOps Engineer (Hadoop & AWS EMR) Client: Lending Club

ShiftCode Analytics

Senior Big Data DevOps Engineer

Must have experience designing, building, scaling and maintaining the infrastructure in datacenter and AWS Big Data analysis tools and machine learning applications.

Accountability for the end-to-end availability, efficiency, reliability and security of our Big Data infrastructure in AWS.
Ensure that our developers & analysts are central to the experience of our platform.
Build custom tools that automate day-to-day operational tasks on the new AWS platform.
Provide input and accountability for setting the standards of our production environment.
Agile and flexible when managing multiple and changing priorities.
Participate in 24X7 on-call rotation investigating and responding to issues

Requirements:

Must have deep understanding of Hadoop ecosystem like HDFS, Yarn, Hive, Spark, Oozie, Presto and Ranger
Excellent knowledge in AWS including EMR, RDS and good understanding on IaaS, PaaS
Experience in infrastructure automation using Terraform
Demonstrated proficiency in Bash & Python, Java
Good understanding of all aspects of JRE/JVMand GC tuning
Experience providing and implementing observability solutions based on "Logs, Metrics & Graphs AWS Cloud watch, Splunk, Elastic stack, Prometheus, Grafana etc.
Hands-on experience with RDBMS (Oracle, Mysql) and NoSql (Cassandra, MongoDB)
Good grasp of the basics of SQL (joins, etc.)
Conduct post-incident reviews and follow-up RCA resulting from platform issues.
Experience with configuration management and remote execution tools such as Chef, Puppet, SaltStack, etc.
Experience with Airflow, Qubole and Hortonworks is a plus
Experience with cloud native Bigdata platforms like Databricks, Snowflake etc., is a plus.

Apply

Vacancy posted 5 hours ago

Similar jobs that could be interesting for youBased on the Big Data DevOps Engineer (Hadoop & AWS EMR) Client: Lending Club in United States vacancy

Data Engineer/Admin
...Data Engineer/Admin Location: Franklin, TN, 37068 Duration: 12 Months Skills: AWS EMR Job Description Hands on experience on Hadoop /Big Data related technology experience in Storage, Querying... ...to Communicate with clients and gather requirements to...
Amazon Web Service
Big data
Work experience placement
Keylent Inc
Franklin, TN
1 day ago
Data Engineer - AWS EMR at Chicago,IL
...Hi Professionals, Title: Data Engineer - AWS EMR Location: Chicago,IL(Onsite) Duration: Contract JD: AWS Bedrock... ..., Lambda, Glue, Athena, Kinesis, IAM, EMR/PySpark. Big Data Technologies: EMR, Spark, Kafka/Kinesis, Airflow....
Amazon Web Service
Big data
Contract work
Local area
E-Solutions
Chicago, IL
2 days ago
Data Devops platform Engineer
...Data Platform Engineer With Devops Location: Hartford CT, Stamford CT Onsite position... ...Apache Airflow, Prefect). Big Data Technologies: Familiarity with Hadoop, Spark, Kafka, and other big data... ...Proficiency in cloud services (AWS, Azure, GCP) and tools like AWS...
Amazon Web Service
Big data
Kaav Inc.
Hartford, CT
1 day ago
Data Engineer @ Plano TX - Long term contract
...REQUIREMENT Job Description: Data Engineer Location: Plano... ...ID: 96989-1 Prior Client : Capital One... ...years of experience in big data technologies At... ...with cloud computing (AWS, Client Azure, Google... ...computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi,...
Amazon Web Service
Big data
Long term contract
Maintec Technologies
Murphy, TX
4 days ago
Big Data Engineer
...Big Data Engineer Responsible for designing and building big data platforms... ...using Bigdata tools and let client's Analysts team to confirm... ...technologies like SQOOP, HADOOP, HIVE, Unix Scripting, SPARK... ...SCALA, PYTHON, PYSPARK, SCALA, AWS and EMR to achieve the requirements....
Amazon Web Service
Big data
Relocation
HCL Global Systems
Farmington Hills, MI
6 hours ago
Data Engineer / Big Data Engineer / Hadoop Developer
...Data Engineer / Big Data Engineer / Hadoop Developer Iselin, NJ 08830 - Locals Only! 12+ Months At least 1 day onsite a week Under the leadership... ...develop, and test enhancements to Data Lake environment in AWS Design, develop, and test enhancements to In-House C#...
Amazon Web Service
Big data
Local area
1 day per week
Samprasoft
Iselin, NJ
2 days ago
Data Engineer
...Genpact locations are fine with client Atlanta, GA Jacksonville, FL... ...- Senior - Intermediate AWS/Cloud - EMR[Hive, Pyspark],S3, Athena or... ...small amount of hands-on Big Data/Hadoop - Working experience with... ...as GitHub or BitBucket. DevOps/CICD - Jenkins, Docker, and...
Amazon Web Service
Big data
Work experience placement
Work at office
Remote work
Staffing the Universe
United States
1 day ago
Big Data Engineer
...Job Opportunity Our client is a custom software development... ...+ years of experience in big data engineering, building real time data... ...tools including: Hadoop, Hive Spark, Kafka... ...including MySQL, Postgres AWS cloud services: EC2, EMR, RDS, Redshift Stream-...
Amazon Web Service
Big data
Hire Talent
Fairport, NY
4 hours ago
AWS Data Engineer
...About the client: Our client is a global technology consulting... ...more than 30 countries. Role--AWS Data Engineer Location -Seattle WA... ...Functions Experience with big data technologies such as Hadoop Hive Spark EMR Understanding of data...
Amazon Web Service
Big data
Local area
E-Solutions
Seattle, WA
5 days ago
Data Engineer (Contract) (Remote)
...seeking a skilled Data Engineer to join our dynamic... ...pipelines, implementing big data solutions, and... ...communication with clients. Availability for... ...solutions using AWS, Azure, or GCP.... ...AWS: Glue, Redshift, EMR, Lambda, S3, Kinesis... ...big data tools like Hadoop, Spark, or Airflow....
Amazon Web Service
Big data
Contract work
Remote work
Flexible hours
Cloudacio
New York, NY
4 days ago
Data Engineer
...ROLE: - Data Engineer LOCATION: - Mclean - VA... ...term contract Client: - Freddie Mac Experience... ...Skills: Snowflake, AWS Big Data, Kafka Job Summary... ...and techniques such as Hadoop Spark Map Reduce Sqoop... ...AWS Cloud services EC EMR ECS Docker OpenShift Kubernetes...
Amazon Web Service
Big data
Long term contract
Work experience placement
Shift work
Maintec Technologies
McLean, VA
5 days ago
Java/Big Data Engineer
...Experian Java/Big Data Engineer Costa Mesa, CA... ...Strong API services on AWS stack using Cloud... ...AWS PySpark, EMR/EC2/ECS/ Airflow/Step... ...performance tuning (Spark/Hadoop/EMR/Python)... ...) Must have DevOps experience Must... ...:MSA: Restricted Client Letter: Will Not Provide...
Amazon Web Service
Big data
Contract work
Local area
Remote work
My3Tech Inc
Costa Mesa, CA
4 days ago
Data Engineer
...for a highly skilled Data Engineer to help design and build... ...for enterprise clients. In this role, you will... ...strong technical depth in big data technologies, AWS services, and... ...AWS services including EMR, Redshift, S3, Glue,... ...technologies such as Hadoop, Hive, Spark, HBase,...
Amazon Web Service
Big data
Solvd
New York, NY
2 days ago
Data Engineer
$75 per hour
Position: Data Engineer Location: Greenwood Village... ...opportunity. Client: Charter Communications... ...and have work with Big Data Tools very... ...and familiarity with Hadoop ecosystems -... ...cloud platforms like AWS, including services... ...Lambda, Redshift, and EMR - Strong...
Amazon Web Service
Big data
Long term contract
Full time
H1b
Remote work
Relocation
Intellisoft Technologies
Glen Allen, VA
2 days ago
Senior Data Engineer (Databricks)
...Infinitive is a data and AI consultancy... ...enables its clients to modernize, monetize... ...Databricks on AWS or Azure to join... ...to data engineering teams. Stay updated... ...as S3, Redshift, EMR, Lambda, Glue,... ...experience with big data technologies (e.g., Hadoop, Spark). Familiarity...
Amazon Web Service
Big data
Local area
Infinitive
Ashburn, VA
2 days ago
Senior Data Engineer
...Design and develop data and integration solutions... ...provide solutions to our clients most challenging enterprise... ...using SQL, Azure, and AWS technologies Build... ...computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, or Spark)... ...processing, and highly scalable big data data stores...
Amazon Web Service
Big data
Remote work
Flexible hours
VS Tech Solutions
Boise, ID
5 days ago
Big Data Engineer (No C2C)
...Our client, a leading organization in the technology and data solutions sector, is seeking a Big Data Engineer to join their team. As a Big Data... ...such as Hadoop, Spark, Python, and... ...workloads on Amazon EMR on EKS (Elastic Kubernetes... ...Proficiency with AWS services like S3,...
Amazon Web Service
Big data
Weekly pay
Temporary work
Flexible hours
Experis/Manpower Group
Rockville, MD
3 days ago
Senior Data Engineer
...high-volume weather data, insights,... ...hundreds of enterprise clients and more than 360... ...a Senior Data Engineer to design, build,... ...work with modern big data technologies and AWS services, contribute... ...including S3, EMR, SQS, SNS, Lambdas... ...Cassandra, Spark, Hadoop, Parquet, Redash,...
Amazon Web Service
Big data
Flexible hours
The Weather Channel
Atlanta, GA
1 day ago
Data Engineer
...Data Engineer On-site role in Phoenix, AZ... ...sponsorship. Our client is a fast-... ...of business. An AWS Data Engineer... ...Glue, Lambda, EMR, S3, Redshift,... ...Experience with big data technologies... ...Apache Spark, Hadoop, Kafka, and potentially... ...with DevOps practices for managing...
Amazon Web Service
Big data
Local area
Night shift
ATS Staffing
Phoenix, AZ
5 days ago
Data Engineer
$75 - $93 per hour
...Our client, a leading organization in the technology and data industry, is seeking a Data Engineer to join their team. As a Data Engineer... ...Experience with Big Data platforms such as Databricks, EMR, Hadoop. Hands-on... ...cloud environments like AWS, GCP, or Azure. What...
Amazon Web Service
Big data
Weekly pay
Temporary work
Flexible hours
Experis/Manpower Group
Dallas, TX
1 day ago
Big Data Engineer (No C2C)
...Our client, a leading organization in the technology and data solutions sector, is seeking a Big Data Engineer to join their team. As a Big Data... ...such as Hadoop, Spark, Python, and... ...workloads on Amazon EMR on EKS (Elastic Kubernetes... ...Proficiency with AWS services like S3,...
Amazon Web Service
Big data
Weekly pay
Temporary work
Flexible hours
Manpower Group Inc.
Rockville, MD
5 days ago
Data Engineer
$110k - $160k
...Data Engineer In today's rapidly evolving technology... ...experts work with our clients to support their mission... ...in cloud (Preferably AWS. Alternatively Azure,... ...tools such as: Big data tools: Hadoop, Spark, Kafka, etc.... ...cloud services: EC2, EMR, RDS, Redshift (or...
Amazon Web Service
Big data
Navstar
McLean, VA
1 day ago
Senior Data Engineer
$125k - $165k
...an organization's data has never been a more... ...work with our clients to support their mission... ...Senior Data Engineer to work with our... ...cloud (Preferably AWS. Alternatively Azure... ...tools such as: ~ Big data tools: Hadoop, Spark, Kafka, etc... ...services: EC2, EMR, RDS, Redshift (or...
Amazon Web Service
Big data
Steampunk.com
McLean, VA
19 hours ago
Senior Data Engineer
...Senior Data Engineer As an employee, you... ...Value® - for our clients, for our... ...solutions using AWS services such as... ...Lambda, RedShift, EMR, Spark, and DynamoDB... ...from Cloudera Hadoop platforms to... ...consumption within a big data... ...through CI/CD in a DevOps model ~ General...
Amazon Web Service
Big data
Remote work
Flexible hours
Shift work
xScion Solutions
United States
4 days ago
Big Data Engineer
..., is searching for a Big Data Engineer for a CONTRACT assignment... ...premier Regulatory clients in Rockville, MD.... ...technologies such as Hadoop, Spark, and Kubernetes... ...data workloads on Amazon EMR on EKS (Elastic... ...Experience with AWS services like S3, EMR,...
Amazon Web Service
Big data
Contract work
Work experience placement
Remote work
SGA
United States
3 days ago
Senior Cloud Data Operations Engineer (Contract in San Francisco)
...Senior Cloud Data Operations Engineer Responsibilities Support... ...Platform (RedShift/EMR/OpenSearch Service)... ...Identify gaps and configure AWS services in use for... ...teams Strong client-facing experience... ...processing engine & Big Data/Hadoop framework Experience...
Amazon Web Service
Big data
Long term contract
Contract work
Remote work
LABINE AND ASSOCIATES, INC.
San Francisco, CA
1 day ago
Data Engineer III
...and seniors. Our engineers, designers, and... ...optimizing our data and data... ...Spectrum, Databricks, AWS EMR, SageMaker, etc... .... Work with DevOps engineers on CI... ...data sets, 'big data' data pipelines... ...like Spark and Hadoop; relational... ...our government clients. Expected hours...
Amazon Web Service
Big data
Work experience placement
Remote work
Flexible hours
eSimplicity Inc
United States
3 days ago
Data Engineer Technology Lead
...Data Engineer Technology Lead Working at Citi is far more... ...Citigroup, its clients and assets, by driving... ...multiple sources. Big Data Infrastructure: Develop... ...frameworks like Apache Spark, Hadoop, and Kafka.... ...cloud data platforms like AWS (Glue, EMR, Redshift), Azure (...
Amazon Web Service
Big data
Citi
New York, NY
1 day ago
Senior Data Engineer
$120k - $140k
...leading brands. We give clients access to near real-... ...seeking a senior data engineer to join our technology... ...using Spark and other Hadoop ecosystem software. ~4+ years using AWS cloud services, esp. EMR, Glue, Athena, and Redshift... .... Experience with big data and/or...
Amazon Web Service
Big data
Work at office
Local area
Immediate start
Flywheel Digital
New York, NY
3 days ago
Data Systems Engineer (AWS, Snowflake, RedShift, Python, Scala, Hadoop, Spark, Kafka, Hive, API, Handling, API Development, Data Migration, Batch Data Pipelines) in Charlotte, NC
Data Systems Engineer (AWS, Snowflake, RedShift, Python, Scala, Hadoop, Spark, Kafka, Hive, API, Handling... ..., Glue, Lambda, EMR, Snowflake, RedShift... ...of our reputed client in Financial... ...Hadoop ecosystem and Big Data technologies... ...in alignment with DevOps practices with...
Amazon Web Service
Big data
Full time
Local area
Immediate start
Remote work
DBA Web Technologies
Charlotte, NC
more than 2 months ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data DevOps Engineer (Hadoop & AWS EMR) Client: Lending Club. Be the first to apply!