Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Kafka Engineer

Macpower Digital Assets Edge

Job Overview: We are seeking an experienced Kafka Engineer with expertise in Confluent Kafka, Java/Scala, and distributed systems . The ideal candidate should be skilled in designing scalable, fault-tolerant Kafka-based data pipelines , troubleshooting messaging issues, and optimizing performance. A strong background in cloud deployments, microservices, and Agile development with an utomate-first approach is essential.

Responsibilities:

  • Identify and resolve Kafka messaging issues within a justified timeframe.
  • Collaborate with business and IT teams to understand business problems and design, implement, and deliver appropriate solutions using gile methodology within a larger program.
  • Work independently to implement solutions across multiple environments (DEV, QA, UAT, PROD).
  • Provide technical direction, guidance, and code reviews for other engineers working on the same project.
  • dminister distributed Kafka clusters in DEV, QA, UAT, and PROD environments and troubleshoot performance issues .
  • Implement and debug subsystems, microservices, and components .
  • Follow an utomate-first/automate-everything philosophy.
  • Demonstrate hands-on experience with programming languages relevant to the role.
Key Skills & Expertise:
  • Deep understanding of Confluent Kafka - Proficient in Kafka concepts, including producers, consumers, topics, partitions, brokers, and replication mechanisms .
  • Programming proficiency - Expertise in Java or Scala , with potential Python usage depending on the project.
  • System design and architecture - Ability to design robust, scalable Kafka-based data pipelines considering data throughput, fault tolerance, and latency .
  • Data management skills - Knowledge of data serialization formats such as JSON, Avro, and Protobuf , and schema evolution management .
  • Kafka Streams API (optional) - Familiarity with Kafka Streams for real-time data processing within the Kafka ecosystem.
  • Monitoring & troubleshooting - Experience with Kafka cluster health monitoring , identifying performance bottlenecks , and troubleshooting issues.
  • Cloud integration - Experience deploying and managing Kafka on WS, Azure, or GCP .
  • Understanding of distributed systems concepts .
Must-Have Qualifications:
  • 8-12 years of experience in software engineering.
  • Kafka expertise - Deep knowledge of Kafka producers, consumers, topics, partitions, brokers, and replication .
  • Programming proficiency - Strong in Java or Scala , with potential Python usage.
  • System design & architecture - Experience in designing high-throughput, scalable Kafka pipelines .
  • Cloud & DevOps - Experience deploying Kafka on AWS, Azure, or GCP .
  • Monitoring & troubleshooting - Familiarity with Kafka cluster health monitoring and performance tuning .
Vacancy posted more than 2 months ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Kafka Engineer. Be the first to apply!