Digital Replicas That Can Have Real Conversations

Infinite ML with Prateek Joshi

Hassaan Raza is the cofounder and CEO of Tavus, a video API platform for digital twins. They've raised more than $28M in funding from investors such as Sequoia and Scale VP.
Hassaan's favorite book: Go Like Hell (Author: A. J. Baime)
(00:01) Introduction
(00:38) Overview of AI in video generation
(01:44) AI models used in video generation
(03:35) Capturing intricate facial movements in real-time
(06:46) Data capture and 3D modeling from basic video input
(09:01) Explanation of neural radiance fields and Gaussian splatting
(10:14) Capturing facial expressions for video generation
(15:22) Temporal coherence in video generation
(18:05) Challenges in conversational video, including lip-syncing and emotion alignment
(20:38) Inference challenges in conversational video
(22:47) Bottlenecks in the pipeline: LLMs and time-to-first-token
(26:58) Multimodal models and trade-offs
(27:36) Advice for founders running API businesses
(30:04) Pitfalls to avoid in API businesses
(32:15) Technological breakthroughs in AI
(34:10) Rapid-fire round
--------
Where to find Prateek Joshi:
Newsletter: https://prateekjoshi.substack.com
Website: https://prateekj.com
LinkedIn: https://www.linkedin.com/in/prateek-joshi-91047b19
Twitter: https://twitter.com/prateekvjoshi