Senior Data Engineer

Senior Data EngineerSkill - Last Used - Experience in Years/month - Hands on Exp. Yes/NoBig DataHadoop/ SparkDatabricks/ ClouderaETLJob Description: Senior Data Engineer who is passionate about data and analytics, highly motivated and can take technical ownership of data solutions with an emphasis on engineering efficiency and on-time delivery. The ideal candidate is exposed to the fast-paced world of Big Data technology and has experience in building data solutions using new and emerging technologies while maintaining stability of the platform. The person will get a chance to work with diverse datasets and be on the cutting edge of transforming the way Client captures, processes, stores and visualizes data. Role- In this role, you will:Design & Develop batch and streaming solutions using various Big Data technologiesWork closely with architect to design secure and scalableBe responsible for assessing technologies and approaches for ingestion, transformation and storageDevelop software utilizing open source technologies to interface with distributed and relational data sourcesWork closely with team members from across Client to identify functional and system requirementsAll About You- The ideal candidate for this position should:Hands-on experience with Big Data and analytics technologies including Hadoop, Spark, and related frameworks.Programming expertise in languages such as Java, Scala, or Python, with an emphasis on developing data processing and transformation logic.Real-time and batch data processing skills with experience in building robust data pipelines for real-time or near-real-time ingestion.Proficiency in data modeling and schema design for relational and non-relational databases, supporting high data volume, and optimized query performance.Exposure to cloud technologies with hands-on experience in Databricks and Cloudera, including knowledge of cloud architecture, data security, and scalability best practices.Experience in optimizing and managing ETL workflows using scheduling, monitoring, and debugging tools.Strong problem-solving abilities with a mindset for automation and efficiency improvements.Excellent communication and collaboration skills to work effectively across technical and business teams, facilitating knowledge sharing and alignment on data engineering best practices.Proven track record of delivering software projects and willingness to roll up sleeves to get the job done,Have excellent oral and written communicationBe familiar with Agile/Scrum methodologiesHave the ability to manage multiple priorities simultaneouslyHave a BS/BA degree in Computer Science, Information Systems or related field