Azure Data Engineer

Role Description: Collect, store, process and analyze large datasets to build and implement extract, transfer, load (ETL) processes.Develop reusable frameworks to reduce the development effort involved thereby ensuring cost savings for the projects.Develop quality code with thought through performance optimizations in place right at the development stage.Appetite to learn new technologies and be ready to work on new cutting-edge cloud technologies.Work with team spread across the globe in driving the delivery of projects and recommend development and performance improvementsBuilding and Implementing data ingestion and curation process developed using Big data tools such as Spark (ScalapythonJava), Hive, HDFS, Sqoop ,Kafka, Kerberos, Impala etc. and CDP 7.xIngesting huge volumes data from various platforms for Analytics needs and writing high-performance, reliable, and maintainable ETL code StrongStrong analytic skills related to working with unstructured datasetsStrong experience in building designing Data warehouses, data stores for analytics consumption On prem and Cloud (real time as well as batch use cases). Proficiency and extensive Experience with Spark Scala, Python and performance tuning is a MUSTHive database management and Performance tuning is a MUST. (Partitioning Bucketing )Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.