Senior Kafka Developer

Required Technical/Functional Skills

• Minimum 10 years of experience in software development and data engineering.
• Strong hands-on experience with Apache Kafka development and implementation.
• Expertise in designing and developing real-time data streaming applications and event-driven architectures.
• Experience building scalable, reliable, and fault-tolerant data pipelines.
• Strong understanding of Kafka components including:
  • Kafka Brokers
  • Topics and Partitions
  • Producers and Consumers
  • Kafka Streams
  • Kafka Connect
  • Schema Registry
• Experience with real-time stream processing using Kafka Streams and/or KSQL.
• Knowledge of Hadoop ecosystem technologies preferred.
• Strong troubleshooting and performance optimization skills.

Primary Responsibilities

• Design, develop, and maintain real-time streaming applications using Apache Kafka.
• Build and optimize high-throughput, fault-tolerant data pipelines.
• Develop and maintain Kafka producer and consumer applications.
• Configure, manage, and monitor Kafka clusters, brokers, topics, and partitions.
• Implement integrations using Kafka Connect with external systems and data sources.
• Develop stream processing solutions using Kafka Streams and KSQL.
• Manage data serialization formats and resolve serialization-related issues.
• Analyze performance bottlenecks and optimize Kafka cluster performance.
• Troubleshoot message delivery, connectivity, and data processing issues.
• Collaborate with engineering teams to design scalable event-driven solutions.

Preferred Qualifications

• Experience with large-scale distributed systems and cloud-based data platforms.
• Strong understanding of data engineering and streaming architecture patterns.
• Experience working with Hadoop and big data processing environments.
• Excellent analytical, troubleshooting, and communication skills.