DivMoE: Fine-Grained MoE Upcycling via Cross-Domain Expert Composition
arXiv:2610.11317v1 Announce Type: new Abstract: Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, with recent work demonstrating the benefits of fine-grained expert designs. Training such...
Read more: https://arxiv.org/abs/2610.11317
Scan with your phone
Scan with your phone