Kafka Administrator

We are seeking an experienced Kafka Administrator with strong expertise in Apache Kafka, Confluent technologies, AWS MSK, and large-scale production environments. The ideal candidate will have a background in Site Reliability Engineering (SRE) or DevOps, with hands-on experience managing highly available Kafka clusters, performing major cluster rebuilds and hardware refreshes, and executing migrations with minimal downtime. Key Responsibilities Design, deploy, configure, and maintain highly available Apache Kafka clusters across on-premises, cloud, and managed environments, including AWS MSK and Confluent Cloud. Administer Kafka topics, partitions, quotas, retention policies, replication settings, and consumer groups to ensure optimal performance and cost efficiency. Manage Kafka ecosystem components, including Kafka Connect, Schema Registry, MirrorMaker, Confluent Replicator, and REST Proxy. Execute Kafka upgrades, patching, hardware refreshes, cluster rebuilds, and migrations while minimizing downtime and operational risks. Perform Kafka performance tuning, throughput testing, and benchmarking across producers, consumers, brokers, and controllers. Optimize batching, compression, acknowledgments ( acks ), in-sync replicas (ISR), and cluster health. Develop capacity plans, partitioning strategies, infrastructure sizing models, and cluster right-sizing recommendations. Integrate Apache Kafka with AWS MSK and other enterprise systems. Implement monitoring, alerting, troubleshooting, backup and recovery, and disaster recovery procedures. Apply strong Linux and networking fundamentals, including TCP/IP, DNS, and load balancing, to troubleshoot infrastructure and connectivity issues. Develop architecture diagrams, operational standards, runbooks, and migration documentation. Conduct knowledge transfer sessions and ensure smooth documentation handoff at key milestones and project completion. Required Skills and Qualifications 8–10 years of overall relevant IT experience. 4+ years of hands-on experience operating Kafka in production at scale within an SRE or DevOps environment. Strong experience with Apache Kafka administration and production operations. Hands-on experience with AWS MSK operations, configuration, and integration. Practical experience with Kafka Connect, Schema Registry, MirrorMaker, and Confluent Replicator. Strong Linux administration and networking fundamentals, including TCP/IP, DNS, and load balancing. Proven experience executing hardware refreshes, major cluster rebuilds, and migrations with minimal downtime. Experience with Kafka performance tuning, capacity planning, cluster sizing, and troubleshooting. Knowledge of high availability, replication, disaster recovery, monitoring, and operational best practices. Ability to create technical documentation, migration plans, architecture diagrams, and operational runbooks. Strong communication and collaboration skills. 2T Consulting