Artwork

Inhalt bereitgestellt von Kostas Pardalis, Nitay Joffe. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Kostas Pardalis, Nitay Joffe oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.
Player FM - Podcast-App
Gehen Sie mit der App Player FM offline!

Reinventing Stream Processing: From LinkedIn to Responsive with Apurva Mehta

58:13
 
Teilen
 

Manage episode 469992700 series 3594857
Inhalt bereitgestellt von Kostas Pardalis, Nitay Joffe. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Kostas Pardalis, Nitay Joffe oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

Summary

In this episode, Apurva Mehta, co-founder and CEO of Responsive, recounts his extensive journey in stream processing—from his early work at LinkedIn and Confluent to his current venture at Responsive.

He explains how stream processing evolved from simple event ingestion and graph indexing to powering complex, stateful applications such as search indexing, inventory management, and trade settlement.

Apurva clarifies the often-misunderstood concept of “real time,” arguing that low latency (often in the one- to two-second range) is more accurate for many applications than the instantaneous response many assume. He delves into the challenges of state management, discussing the limitations of embedded state stores like RocksDB and traditional databases (e.g., Postgres) when faced with high update rates and complex transactional requirements.

The conversation also covers the trade-offs between SQL-based streaming interfaces and more flexible APIs, and how Responsive is innovating by decoupling state from compute—leveraging remote state solutions built on object stores (like S3) with specialized systems such as SlateDB—to improve elasticity, cost efficiency, and operational simplicity in mission-critical applications.

Chapters

00:00 Introduction to Apurva Mehta and Streaming Background
08:50 Defining Real-Time in Streaming Contexts
14:18 Challenges of Stateful Stream Processing
19:50 Comparing Streaming Processing with Traditional Databases
26:38 Product Perspectives on Streaming vs Analytical Systems
31:10 Operational Rigor and Business Opportunities
38:31 Developers' Needs: Beyond SQL
45:53 Simplifying Infrastructure: The Cost of Complexity
51:03 The Future of Streaming Applications

Click here to view the episode transcript.

  continue reading

22 Episoden

Artwork
iconTeilen
 
Manage episode 469992700 series 3594857
Inhalt bereitgestellt von Kostas Pardalis, Nitay Joffe. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Kostas Pardalis, Nitay Joffe oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.

Summary

In this episode, Apurva Mehta, co-founder and CEO of Responsive, recounts his extensive journey in stream processing—from his early work at LinkedIn and Confluent to his current venture at Responsive.

He explains how stream processing evolved from simple event ingestion and graph indexing to powering complex, stateful applications such as search indexing, inventory management, and trade settlement.

Apurva clarifies the often-misunderstood concept of “real time,” arguing that low latency (often in the one- to two-second range) is more accurate for many applications than the instantaneous response many assume. He delves into the challenges of state management, discussing the limitations of embedded state stores like RocksDB and traditional databases (e.g., Postgres) when faced with high update rates and complex transactional requirements.

The conversation also covers the trade-offs between SQL-based streaming interfaces and more flexible APIs, and how Responsive is innovating by decoupling state from compute—leveraging remote state solutions built on object stores (like S3) with specialized systems such as SlateDB—to improve elasticity, cost efficiency, and operational simplicity in mission-critical applications.

Chapters

00:00 Introduction to Apurva Mehta and Streaming Background
08:50 Defining Real-Time in Streaming Contexts
14:18 Challenges of Stateful Stream Processing
19:50 Comparing Streaming Processing with Traditional Databases
26:38 Product Perspectives on Streaming vs Analytical Systems
31:10 Operational Rigor and Business Opportunities
38:31 Developers' Needs: Beyond SQL
45:53 Simplifying Infrastructure: The Cost of Complexity
51:03 The Future of Streaming Applications

Click here to view the episode transcript.

  continue reading

22 Episoden

Alle Folgen

×
 
Loading …

Willkommen auf Player FM!

Player FM scannt gerade das Web nach Podcasts mit hoher Qualität, die du genießen kannst. Es ist die beste Podcast-App und funktioniert auf Android, iPhone und im Web. Melde dich an, um Abos geräteübergreifend zu synchronisieren.

 

Kurzanleitung

Hören Sie sich diese Show an, während Sie die Gegend erkunden
Abspielen