Artwork

Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.
Player FM - Podcast-App
Gehen Sie mit der App Player FM offline!

What can Apache Kafka Developers learn from Online Gaming?

55:32
 
Teilen
 

Manage episode 354794863 series 2355972
Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern online games are complex, distributed real-time data systems with decades of innovative techniques to teach us.
In this episode, Ben talks with Kris about integrating gaming concepts with Apache Kafka®. Using Kafka’s state management stream processing, Ben has built systems that can handle real-time event processing at a massive scale, including interesting approaches to conflict resolution and collaboration.
Building latency into a system is one way to mask data processing time. Ben says that you can efficiently hide latency issues and prioritize performance improvements by setting an initial target and then optimizing from there. If you measure before optimizing, you can add an extra layer to manage user expectations better. Tricks like adding a visual progress bar give the appearance of progress but actually hide latency and improve the overall user experience.
To effectively handle challenging activities, like resolving conflicts and atomic edits, Ben suggests “slicing” (or nano batching) to break down tasks into small, related chunks. Slicing allows each task to be evaluated separately, thus producing timely outcomes that resolve potential background conflicts without the user knowing.
Ben also explains how he uses pooling to make collaboration seamless. Pooling is a process that links open requests with potential matches. Similar to booking seats on an airplane, seats are assigned when requests are made. As these types of connections are handled through a Kafka event stream, the initial open requests are eventually fulfilled when seats become available.
According to Ben, real-world tools that facilitate collaboration (such as Google Docs and Slack) work similarly. Just like multi-player gaming systems, multiple users can comment or chat in real-time and users perceive instant responses because of the techniques ported over from the gaming world.
As Ben sees it, the proliferation of these types of concepts across disciplines will also benefit a more significant number of collaborative systems. Despite being long established for gamers, these patterns can be implemented in more business applications to improve the user experience significantly.
EPISODE LINKS

  continue reading

Kapitel

1. Intro (00:00:00)

2. Building real-time inspection machines (00:05:29)

3. Going Multiplayer with Kafka (00:07:14)

4. What does a real-time experience look like in gaming? (00:14:35)

5. Broken message offsets vs. dead letter queues (00:17:05)

6. The distributed transaction problem (00:19:15)

7. Solving for conflicting events with nano-batching (00:25:02)

8. Pooling in real-time systems (00:31:56)

9. Optimizing for latency (00:38:29)

10. Using column modeling to improve performance (00:44:08)

11. Principles for stream-processing (00:50:49)

12. It's a wrap! (00:52:46)

265 Episoden

Artwork
iconTeilen
 
Manage episode 354794863 series 2355972
Inhalt bereitgestellt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka®. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Confluent, founded by the original creators of Apache Kafka® and Founded by the original creators of Apache Kafka® oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

What can online gaming teach us about making large-scale event management more collaborative in real-time? Ben Gamble (Developer Relations Manager, Aiven) has come to the world of real-time event streaming from an usual source: the video games industry. And if you stop to think about it, modern online games are complex, distributed real-time data systems with decades of innovative techniques to teach us.
In this episode, Ben talks with Kris about integrating gaming concepts with Apache Kafka®. Using Kafka’s state management stream processing, Ben has built systems that can handle real-time event processing at a massive scale, including interesting approaches to conflict resolution and collaboration.
Building latency into a system is one way to mask data processing time. Ben says that you can efficiently hide latency issues and prioritize performance improvements by setting an initial target and then optimizing from there. If you measure before optimizing, you can add an extra layer to manage user expectations better. Tricks like adding a visual progress bar give the appearance of progress but actually hide latency and improve the overall user experience.
To effectively handle challenging activities, like resolving conflicts and atomic edits, Ben suggests “slicing” (or nano batching) to break down tasks into small, related chunks. Slicing allows each task to be evaluated separately, thus producing timely outcomes that resolve potential background conflicts without the user knowing.
Ben also explains how he uses pooling to make collaboration seamless. Pooling is a process that links open requests with potential matches. Similar to booking seats on an airplane, seats are assigned when requests are made. As these types of connections are handled through a Kafka event stream, the initial open requests are eventually fulfilled when seats become available.
According to Ben, real-world tools that facilitate collaboration (such as Google Docs and Slack) work similarly. Just like multi-player gaming systems, multiple users can comment or chat in real-time and users perceive instant responses because of the techniques ported over from the gaming world.
As Ben sees it, the proliferation of these types of concepts across disciplines will also benefit a more significant number of collaborative systems. Despite being long established for gamers, these patterns can be implemented in more business applications to improve the user experience significantly.
EPISODE LINKS

  continue reading

Kapitel

1. Intro (00:00:00)

2. Building real-time inspection machines (00:05:29)

3. Going Multiplayer with Kafka (00:07:14)

4. What does a real-time experience look like in gaming? (00:14:35)

5. Broken message offsets vs. dead letter queues (00:17:05)

6. The distributed transaction problem (00:19:15)

7. Solving for conflicting events with nano-batching (00:25:02)

8. Pooling in real-time systems (00:31:56)

9. Optimizing for latency (00:38:29)

10. Using column modeling to improve performance (00:44:08)

11. Principles for stream-processing (00:50:49)

12. It's a wrap! (00:52:46)

265 Episoden

Alle Folgen

×
 
Loading …

Willkommen auf Player FM!

Player FM scannt gerade das Web nach Podcasts mit hoher Qualität, die du genießen kannst. Es ist die beste Podcast-App und funktioniert auf Android, iPhone und im Web. Melde dich an, um Abos geräteübergreifend zu synchronisieren.

 

Kurzanleitung