Data Engineer With Scala Exp.

Overview:

**Mandatory skills / Required Experience**- Azure (Bigdata Azure), DataBricks , Scala, Java, Spark, Pyspark, Python,**Job Description: -**- 11+ years of experience with Azure Data Factory and Databricks with Big Data ecosystem tools (e.g. Hadoop, Hive) and data processing within the EDL for big data pipelines, architectures & data sets- Knowledge of Python/C# is a MUST- Assure that data is cleansed, mapped, transformed, and otherwise optimized for storage and use according to business and technical requirements- Develop and maintain innovative Azure solutions- The ability to automate tasks and deploy production standard code (with unit testing, continuous integration, versioning etc.)- Load transformed data into storage and reporting structures in destinations including data warehouse, high speed indexes, real-time reporting systems and analytics applications- Build data pipelines to collectively bring together data- Other responsibilities include extracting data, troubleshooting and maintaining the data warehouse