Lead Data Engineer

6-12 Months

Responsibilities

• Design and Implement scalable solutions for ever-increasing data volumes, using big data/cloud technologies like Pyspark, Kafka, etc.
• Collaborate with cross-functional teams to understand data requirements and provide effective solutions.
• Implement real-time data ingestion and processing solutions
• Develop and maintain ETL/ELT processes to support data analytics and reporting.
• Implement best practices for data security, integrity, and quality.
• Developing data integration and migration strategies to move data from legacy systems to Cloud
• Monitoring and troubleshooting the performance and availability of the data systems and implementing strategies to improve their reliability and scalability
• Managing security and access controls for the data systems, including managing roles and permissions, implementing encryption, and complying with regulatory requirements.

Requirements

• Bachelor's in Engineering / Master's degree in Computer Science, Information Systems or related field
• Minimum of 5 - 9 years of experience in data engineering.
• Experience with databases and data warehouse implementations (Preferred Snowflake, PostGreSQL)
• Strong SQL skills with experience in writing and optimising complex queries
• Working knowledge of Data warehousing, Data Governance, Data Quality and Data Architecture
• Experience in Data Modelling
• Ability to handle large scale structured and unstructured data from internal and third-party sources
• Hands On Experience in Python, Pyspark, Kafka
• Experience with data engineering tools/technologies in Google Cloud Platform Cloud environment
• Proficiency in designing and maintaining scalable data architectures
• Experience with CICD Tools like GitHub

For applications and inquiries, contact:View email address on us.fitly.work