))
CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers
Manage episode 432986994 series 2954468
Inhalt bereitgestellt von Rob. Alle Podcast-Inhalte, einschließlich Episoden, Grafiken und Podcast-Beschreibungen, werden direkt von Rob oder seinem Podcast-Plattformpartner hochgeladen und bereitgestellt. Wenn Sie glauben, dass jemand Ihr urheberrechtlich geschütztes Werk ohne Ihre Erlaubnis nutzt, können Sie dem hier beschriebenen Verfahren folgen https://de.player.fm/legal.
Large-scale pretrained transformers have created milestones in text (GPT-3) and text-to-image (DALL-E and CogView) generation. Its application to video generation is still facing many challenges: The potential huge computation cost makes the training from scratch unaffordable; The scarcity and weak relevance of text-video datasets hinder the model understanding complex movement semantics. In this work, we present 9B-parameter transformer CogVideo, trained by inheriting a pretrained text-to-image model, CogView2. We also propose multi-frame-rate hierarchical training strategy to better align text and video clips. As (probably) the first open-source large-scale pretrained text-to-video model, CogVideo outperforms all publicly available models at a large margin in machine and human evaluations.
2022: Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, Jie Tang
https://arxiv.org/pdf/2205.15868
…
continue reading
2022: Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, Jie Tang
https://arxiv.org/pdf/2205.15868
298 Episoden
Teilen
