Pool spare GPU capacity to run LLMs at larger scale

Pool spare GPU capacity to run LLMs at larger scale. Models that don't fit on one machine are automatically distributed dense models via pipeline parallelism, MoE models via expert sharding with zero… [+12994 chars]