Lead Data Engineer

Description: Senior Data Engineer - Data Platforms Role Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely with product managers, data analysts, BI developers, and other engineers to deliver reliable, high quality data pipelines across both batch and real time streaming architectures. The ideal candidate has deep experience building cloud native data solutions on AWS, strong fundamentals in distributed systems, and hands on experience with stream processing frameworks such as Apache Flink, using Python and Java. Key Responsibilities:

• Design, develop, and maintain scalable data pipelines supporting batch and real time streaming workloads
• Build and optimize data processing jobs using AWS Glue, PySpark, Python, and Java
• Develop and maintain stream processing applications using Apache Flink
• Implement reliable data ingestion solutions using AWS DMS, Kafka, and AWS Lambda
• Design and manage data persistence layers using Amazon Aurora (PostgreSQL) and related AWS services
• Design and support real time and near real time streaming pipelines using Kafka and Apache Flink
• Build and maintain stateful stream processing jobs, including windowing, aggregation, and event time processing
• Develop efficient batch processing workflows for large scale data transformation and enrichment
• Ensure data consistency, latency, fault tolerance, and reliability across streaming and batch systems
• Implement monitoring, logging, and alerting for batch and streaming data pipelines
• Diagnose and resolve data quality, performance, and scalability issues
• Apply best practices for schema evolution, checkpointing, fault tolerance, and back pressure handling
• Optimize pipelines for performance and cost efficiency
• Enable downstream analytics and reporting through well modeled, well documented datasets
• Partner with analytics and BI teams supporting tools such as Qlik
• Support data consumers by improving data discoverability, usability, and trust
• Collaborate with product, analytics, and platform teams to translate requirements into technical solutions
• Participate in architecture discussions, design reviews, and code reviews
• Contribute to data engineering standards, reusable frameworks, and documentation
• Mentor junior engineers and promote best practices across the team

Core Technologies: AWS Glue, PySpark, Python, Java, Apache Flink (Stream Processing), Kafka, AWS DMS, AWS Lambda, Amazon Aurora (PostgreSQL), Streaming and Batch Data Processing, Qlik (analytics / BI consumption)

For applications and inquiries, contact:View email address on us.fitly.work