Senior Data Engineer

Senior Data Engineer

Skill - Last Used - Experience in Years/month - Hands on Exp. Yes/No

Big Data

Hadoop/ Spark

Databricks/ Cloudera

ETL

Job Description: Senior Data Engineer who is passionate about data and analytics, highly motivated and can take technical ownership of data solutions with an emphasis on engineering efficiency and on-time delivery. The ideal candidate is exposed to the fast-paced world of Big Data technology and has experience in building data solutions using new and emerging technologies while maintaining stability of the platform. The person will get a chance to work with diverse datasets and be on the cutting edge of transforming the way Client captures, processes, stores and visualizes data. Role- In this role, you will:

1. Design & Develop batch and streaming solutions using various Big Data technologies
2. Work closely with architect to design secure and scalable
3. Be responsible for assessing technologies and approaches for ingestion, transformation and storage
4. Develop software utilizing open source technologies to interface with distributed and relational data sources
5. Work closely with team members from across Client to identify functional and system requirements

All About You- The ideal candidate for this position should:

• Hands-on experience with Big Data and analytics technologies including Hadoop, Spark, and related frameworks.
• Programming expertise in languages such as Java, Scala, or Python, with an emphasis on developing data processing and transformation logic.
• Real-time and batch data processing skills with experience in building robust data pipelines for real-time or near-real-time ingestion.
• Proficiency in data modeling and schema design for relational and non-relational databases, supporting high data volume, and optimized query performance.
• Exposure to cloud technologies with hands-on experience in Databricks and Cloudera, including knowledge of cloud architecture, data security, and scalability best practices.
• Experience in optimizing and managing ETL workflows using scheduling, monitoring, and debugging tools.
• Strong problem-solving abilities with a mindset for automation and efficiency improvements.
• Excellent communication and collaboration skills to work effectively across technical and business teams, facilitating knowledge sharing and alignment on data engineering best practices.
• Proven track record of delivering software projects and willingness to roll up sleeves to get the job done,
• Have excellent oral and written communication
• Be familiar with Agile/Scrum methodologies
• Have the ability to manage multiple priorities simultaneously
• Have a BS/BA degree in Computer Science, Information Systems or related field