Data Engineer (Scala)

Data EngineerBP Inc is building new data pipelines from scratch to inherit data in an existing third-party platform over to a new home-grown product they have created, and will be working heavily in Scala Coding, Hadoop/Hive, Spark, and Azure Data Factory (AWS experience OK).The interview process would be 1- 30 Minutes Coding Interview (hackerrank.com style of problem), 2- 30 Minute Engineering Manager interview.This role would be remote working in CST time zone, and I can begin arranging interviews immediately.Innovation & Engineering – Digital Customers, Market, Intelligent OpsRole:In short – We are mainly seeking a skilled Senior Data Engineer, good hands-on skill with Scala, Spark, Hadoop/Hive, and Databricks. We work mainly in Azure cloud (ADF, AzureDevops etc.), but if the person has worked in AWS or others, that is suitable as well.Skills/Responsibilities :Extensive hand on experience working on Scala Coding, Spark and PySpark programmingExperience in Hadoop/Hive big data systemsAnalyze data from multiple sources and create integrated views that can be used in reporting/analytics.Worked on Data Parsing, Data wrangling, Cleansing & Transformation, and sanitizing.Worked on Regular expressions.Troubleshoot and resolve complex customer issues by handling/curating them at DataBricks (pyspark or sql) and other product areas.Good to have SQL coding (T-SQL and other ANSI-compliant SQL) experience with performance optimization experience for data queries.