There is a change for users of Apache Kafka, the leading distributed platform for streaming events. Apache ZooKeeperKafka’s metadata management tool will soon be phased out in favor of internal technology.

Colin McCabe, a member of the Apache Kafka project management committee and a Confluent engineer who uses Kafka, explained the reason for the change. ZooKeeper stores persistent clustered metadata and handles the maintenance of dynamic configurations and themes, as well as partitions within themes, he said. But ZooKeeper adds an extra management layer. Storing metadata internally in Kafka will make it easier to manage and provide better guarantees about issues like the version, according to McCabe.

Sal Kafka, or KRaft, an internally managed metadata protocol, will replace ZooKeeper. In KRaft mode, Kafka metadata will be stored in a distributed log. Scalability will be the main advantage, McCabe said, but governance will also improve. Kafka users will no longer need to have a separate management system for their Kafka clusters.

It is unclear exactly when ZooKeeper will be decommissioned, but a vote on the schedule is expected next week or two. The current proposal is to make KRaft publicly available in the upcoming version of Kafka 3.3. ZooKeeper will be obsolete in the release after that and will be removed in Kafka 4.0.

Targeted for August, Kafka 3.3 will include options for both ZooKeeper and KRaft. The end date of ZooKeeper’s life has not been determined. “The KRaft regime will start production soon, very soon this year,” McCabe said. “This will be a big step forward for the project.

Kafka’s developers are currently working on a full-featured parity between KRaft and ZooKeeper, which is said to be closing. The KRaft mode is actually available from Kafka 2.8, launched in April 2021, but is not ready for production; Kafka 3.3 will be the first version ready for production. For developers accustomed to ZooKeeper, McCabe doesn’t expect a steep learning curve when switching to KRaft. The same APIs are supported for developers, but operators may have a few things to learn. “We hope that the new administrators will really find this easier and that the existing administrators will find the transition manageable.”

Copyright © 2022 IDG Communications, Inc.

Previous articleThe best science fiction TV shows on Prime Video
Next article10 best integrations with Slack: its types and advantages