SaaS Metrics
Comment on: Talk is cheap. Show me the code.
Repo: MoonshotAI/Attention-Residuals by Smu-Tan
I've implemented one based on Megatron-LM, with llama3 dense arch pretrained 1k steps, see https://github.com/Smu-Tan/Megatron-LM
GitHub Issue