Artwork

Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.
Player FM - Podcast-App
Gehen Sie mit der App Player FM offline!

Apache Kafka 3.4 - New Features & Improvements

5:13
 
Teilen
 

Manage episode 424666712 series 2510642
Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

Apache Kafka® 3.4 is released! In this special episode, Danica Fine (Senior Developer Advocate, Confluent), shares highlights of the Apache Kafka 3.4 release. This release introduces new KIPs in Kafka Core, Kafka Streams, and Kafka Connect.
In Kafka Core:

  • KIP-792 expands the metadata each group member passes to the group leader in its JoinGroup subscription to include the highest stable generation that consumer was a part of.
  • KIP-830 includes a new configuration setting that allows you to disable the JMX reporter for environments where it’s not being used.
  • KIP-854 introduces changes to clean up producer IDs more efficiently, to avoid excess memory usage. It introduces a new timeout parameter that affects the expiry of producer IDs and updates the old parameter to only affect the expiry of transaction IDs.
  • KIP-866 (early access) provides a bridge to migrate between existing Zookeeper clusters to new KRaft mode clusters, enabling the migration of existing metadata from Zookeeper to KRaft.
  • KIP-876 adds a new property that defines the maximum amount of time that the server will wait to generate a snapshot; the default is 1 hour.
  • KIP-881, an extension of KIP-392, makes it so that consumers can now be rack-aware when it comes to partition assignments and consumer rebalancing.

In Kafka Streams:

  • KIP-770 updates some Kafka Streams configs and metrics related to the record cache size.
  • KIP-837 allows users to multicast result records to every partition of downstream sink topics and adds functionality for users to choose to drop result records without sending.

And finally, for Kafka Connect:

  • KIP-787 allows users to run MirrorMaker2 with custom implementations for the Kafka resource manager and makes it easier to integrate with your ecosystem.

Tune in to learn more about the Apache Kafka 3.4 release!
EPISODE LINKS

  continue reading

Kapitel

1. Intro (00:00:00)

2. KIP-866: ZooKeeper to KRaft cluster upgrade (00:00:23)

3. KIP-830: Allow disabling JMX Reporter (00:00:57)

4. KIP-881: Rack-aware Partition Assignment for Kafka Consumers (00:01:31)

5. KIP-876: Time based Cluster Metadata Snapshots (00:02:08)

6. KIP-854: Separate configuration for producer ID expiry (00:02:38)

7. KIP-837: Allow MultiCasting a Result Record (00:03:31)

8. KIP-787: MM2 manage Kafka resources with custom Admin implementations (00:04:12)

9. It's a wrap! (00:04:47)

265 Episoden

Artwork
iconTeilen
 
Manage episode 424666712 series 2510642
Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

Apache Kafka® 3.4 is released! In this special episode, Danica Fine (Senior Developer Advocate, Confluent), shares highlights of the Apache Kafka 3.4 release. This release introduces new KIPs in Kafka Core, Kafka Streams, and Kafka Connect.
In Kafka Core:

  • KIP-792 expands the metadata each group member passes to the group leader in its JoinGroup subscription to include the highest stable generation that consumer was a part of.
  • KIP-830 includes a new configuration setting that allows you to disable the JMX reporter for environments where it’s not being used.
  • KIP-854 introduces changes to clean up producer IDs more efficiently, to avoid excess memory usage. It introduces a new timeout parameter that affects the expiry of producer IDs and updates the old parameter to only affect the expiry of transaction IDs.
  • KIP-866 (early access) provides a bridge to migrate between existing Zookeeper clusters to new KRaft mode clusters, enabling the migration of existing metadata from Zookeeper to KRaft.
  • KIP-876 adds a new property that defines the maximum amount of time that the server will wait to generate a snapshot; the default is 1 hour.
  • KIP-881, an extension of KIP-392, makes it so that consumers can now be rack-aware when it comes to partition assignments and consumer rebalancing.

In Kafka Streams:

  • KIP-770 updates some Kafka Streams configs and metrics related to the record cache size.
  • KIP-837 allows users to multicast result records to every partition of downstream sink topics and adds functionality for users to choose to drop result records without sending.

And finally, for Kafka Connect:

  • KIP-787 allows users to run MirrorMaker2 with custom implementations for the Kafka resource manager and makes it easier to integrate with your ecosystem.

Tune in to learn more about the Apache Kafka 3.4 release!
EPISODE LINKS

  continue reading

Kapitel

1. Intro (00:00:00)

2. KIP-866: ZooKeeper to KRaft cluster upgrade (00:00:23)

3. KIP-830: Allow disabling JMX Reporter (00:00:57)

4. KIP-881: Rack-aware Partition Assignment for Kafka Consumers (00:01:31)

5. KIP-876: Time based Cluster Metadata Snapshots (00:02:08)

6. KIP-854: Separate configuration for producer ID expiry (00:02:38)

7. KIP-837: Allow MultiCasting a Result Record (00:03:31)

8. KIP-787: MM2 manage Kafka resources with custom Admin implementations (00:04:12)

9. It's a wrap! (00:04:47)

265 Episoden

Tất cả các tập

×
 
Loading …

Willkommen auf Player FM!

Player FM scannt gerade das Web nach Podcasts mit hoher Qualität, die du genießen kannst. Es ist die beste Podcast-App und funktioniert auf Android, iPhone und im Web. Melde dich an, um Abos geräteübergreifend zu synchronisieren.

 

Kurzanleitung