Arama Yap Mesaj Submit
Request a Callback
+90
X
X

Select Your Currency

Turkish Lira $ US Dollar Euro
X
X

Select Your Currency

Turkish Lira $ US Dollar Euro

Contact Us

Location Halkali merkez neighborhood fatih st ozgur apt no 46 , Kucukcekmece , Istanbul , 34303 , TR
KAFKA · KRAFT · BROKER · PARTITION · RETENTION

Apache Kafka Server: Design KRaft, Partitions and Retention Before Message Count

Modern Apache Kafka clusters use KRaft for metadata management; controller and broker roles can be separate or combined. Capacity depends on retention, partitions, replication factor, message size and disk/network throughput, not event rate alone.

protocol / 2026
01KRaft
02Controller quorum
03Partitions
04Retention
Updated · 18.08.2026
01
On this page

Does Kafka still require ZooKeeper?

Current KRaft architecture manages cluster metadata through the controller quorum and does not require ZooKeeper for new KRaft clusters. In production, controller quorum and broker capacity should be considered separately; combined mode is better suited to smaller/test deployments.

On this pageApache Kafka Server: Design KRaft, Partitions and Retention Before Message Count
01
Event flow

Kafka flow from producer to consumer group

Producers write to topic partitions, brokers replicate according to the replication factor and consumer groups divide partitions among members.

01Producer
02Broker
03Topic / Partition
04Replica Brokers
05Consumer Group
02
KRaft roles

Why separate controller and broker roles?

Controllers manage metadata quorum while brokers handle data traffic and partition IO. Separating roles in larger production clusters improves resource isolation.

ControllerMetadata quorumLow data IOCritical quorum
BrokerPartition dataDisk/network heavyWorkload node
CombinedController+brokerSimplerSmall/test
03
Capacity plan

Do not forget retention and replication factor in Kafka storage sizing

Raw ingest multiplied by retention is not enough; include replicas, segment overhead and growth headroom.

Daily ingest GB
Retention days/hours
Replication factor
Partition count
Average message size
Replication network traffic
04
Kafka checks

Basic Kafka cluster and topic commands

Binary paths depend on installation; adapt the bootstrap server to your cluster.

Command 1
kafka-metadata-quorum.sh --bootstrap-server localhost:9092 describe --status
Command 2
kafka-topics.sh --bootstrap-server localhost:9092 --list
Command 3
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic events
Command 4
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --all-groups --describe
05
Production scaling

Sequence when scaling a Kafka cluster

Adding brokers may not automatically rebalance partitions evenly; plan partition reassignment and client capacity separately.

Measure ingest/retention
Inspect partition distribution
Measure broker disk/network
Add broker
Reassign partitions
Validate consumer lag
Official documentation

Official sources

Apache KafkaDocumentationkafka.apache.orgApache KafkaKRaftkafka.apache.orgEKA SunucuVPSwww.ekasunucu.comEKA SunucuDedicatedwww.ekasunucu.com
FAQ

Frequently asked questions

What is Kafka KRaft?

It manages Kafka cluster metadata using a Raft-based controller quorum, removing ZooKeeper dependency.

Is NVMe required for Kafka?

No, but fast storage throughput and latency benefit heavy log writes, retention and replication workloads.

How many partitions should a topic have?

It depends on throughput, consumer parallelism, ordering and growth; there is no universal number.

EKA YAZILIM VE BİLİŞİM SİSTEMLERİ

Plan Kafka from retention, partitions and replication

Share daily events, message size, retention, replication factor and consumer count; we can design broker/controller topology.

Ask on WhatsApp0850 307 34 58
WhatsAppCall NowExplore
Top