ROIpad ← Back to Search
github.com › repository issue

Release Kimi Linear AttnRes model on Hugging Face

MoonshotAI/Attention-Residuals
Status: Open
Opened: Mar 17, 2026
Hi @yzhangcs 🤗 I'm Niels and work as part of the open-source team at Hugging Face. I discovered your work through Hugging Face's daily papers as yours got featured: https://huggingface.co/papers/2603.15031. The paper page lets people discuss about your paper and lets them find artifacts about it (your models for instance), you can also claim the paper as yours which will show up on your public profile at HF, add Github and project page URLs. Your paper introduces "Attention Residuals" and mentions integrating it into the Kimi Linear architecture (48B total / 3B activated parameters), pre-training it on 1.4T tokens, and evaluating its improved downstream performance. The paper also states that the weights for this model are available at your GitHub repository. It would be fantastic if you would consider hosting this pre-trained Kimi Linear AttnRes model checkpoint on https://huggingface.co/models! Hosting on Hugging Face will give you more visibility/enable better discoverability. We can add tags in the model cards so that people find the models easier, link it to the paper page, etc. If you're interested, here's a guide: https://huggingface.co/docs/hub/models-uploading. If it's a custom PyTorch model, you can use the [PyTorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) class which adds `from_pretrained` and `push_to_hub` to the model, allowing users to download and use models right away. Alternat...
Unknown
View on GitHub ↗
Related Content