bytedance/Bernini

@htahboub We're currently organizing the training code, and we'll decide whether to release it depending on how things go.

Thank you @chenchenliu21! Are you considering releasing the training code too by any chance?

The full Bernini (planner + renderer) is now released. @htahboub @RealJosephus thanks for following up! Closing this, feel free to reopen if anything's missing.

> > 我来分享一下我的经历，我是在Windows上安装并运行的。我用的是RTX 3060，使用Flash Attention 2（源代码为Flash-attn 2.8.4）。我没试过Gradio，用的是B选项（WAN+Bernini）。我已经开始逐个运行这些示例。我成功运行了t2i和i2i的示例，模型加载大约需要310go（我用swap），大约需要2小时，GPU进程才开始。对于T2i，GPU的处理步骤大约需要一分钟，也就是总共大约40分钟，其中从第12步切换到加载参数时会多花10分钟（50分钟）。T2I大约需要3小时。 i2I加载时间差不多，GPU上分步骤大约3分钟（第12步同样慢），大约需要5小时。我试过T2V的例子，过程大约25分钟，但我遇到过手电筒。在第12步（我预期会更早）出现OutOfMemoryError。我可能会稍后再试一次。如果你想在Windows上运行，我建议手动配置最大交换量以保证内存充足。如果有人能提供某个特定GPU（计算能力）的大部分功能持续时间，分享一下会很方便，这样可以估算时间，我特别感兴趣的是时间的复杂性，以及视频越长会不会大幅增加进程时间。
>
> 为啥我在RTX 5090上运行不了，显示RTX 5090，PyTorch 2.5.1 不支持 sm_120，可以告知我。你使用的PyTorch版本吗


i check 5090 have sm120. I think you haven't correctly install pytorch. I recommend you to check your version of cuda (nvcc --version) , pytorch (pip show torch) and flash attention (pip show flash-attn) . you should have a cuda version and pytorch version like X.X.X+cuYYY where YYY is your cuda version. for example i have cuda 13.0 and pytorch 2.12.0+cu130 . you can find pytorch installation instruction on https://pyto...

> 我来分享一下我的经历，我是在Windows上安装并运行的。我用的是RTX 3060，使用Flash Attention 2（源代码为Flash-attn 2.8.4）。我没试过Gradio，用的是B选项（WAN+Bernini）。我已经开始逐个运行这些示例。我成功运行了t2i和i2i的示例，模型加载大约需要310go（我用swap），大约需要2小时，GPU进程才开始。对于T2i，GPU的处理步骤大约需要一分钟，也就是总共大约40分钟，其中从第12步切换到加载参数时会多花10分钟（50分钟）。T2I大约需要3小时。 i2I加载时间差不多，GPU上分步骤大约3分钟（第12步同样慢），大约需要5小时。我试过T2V的例子，过程大约25分钟，但我遇到过手电筒。在第12步（我预期会更早）出现OutOfMemoryError。我可能会稍后再试一次。如果你想在Windows上运行，我建议手动配置最大交换量以保证内存充足。如果有人能提供某个特定GPU（计算能力）的大部分功能持续时间，分享一下会很方便，这样可以估算时间，我特别感兴趣的是时间的复杂性，以及视频越长会不会大幅增加进程时间。

为啥我在RTX 5090上运行不了，显示RTX 5090，PyTorch 2.5.1 不支持 sm_120，可以告知我。你使用的PyTorch版本吗

I will share my experience, i install it on window and run it. i have rtx 3060 and use flash attention 2 (flash-attn 2.8.4 from source ). i haven't try the gradio and use the b option (wan + Bernini).
I have start running one by one the examples. I successfully run the t2i and i2i example, loading of the model need me around 310go (i use swap) and take me around 2hours, before the GPU process start.
For the t2i the gpu process step take around a minute aka around 40 min in total with 10 extra min (50min) of loading parameter from swap at step 12. t2i take my around 3hours.
i2i take my around the same time to load and around 3 min by step on GPU (with the same slow down at step 12) and take me around 5 hours.
I have try the t2v example , it result of around 25 min by step but i have encounter a torch.OutOfMemoryError at the 12 step (which i expected sooner). I will probably retry it later.
If you want to run it on windows, I recommend to manually configure the maximum amount of swap to ...

Hi @1820366459, thanks for your great work here! To clarify, are you planning on releasing both code and checkpoints for the planner?

Thanks for the interest in our work! 🙏 We haven't tested on Windows or the RTX 6000, so this isn't a configuration we can vouch for. It may need some config adjustment to run there.

The MLLM planner code is fairly complex and we're currently cleaning it up. We expect to release it next week. Thanks for your patience!

I install all in wsl linux windows, but when i lunch the app it crash loading the gradio