뉴스 & 기능
로딩중
Microsoft Research 블로그
Dion: the distributed orthonormal update revolution is here
| Kwangjun Ahn 그리고 John Langford
Dion is a new AI model optimization method that boosts scalability and performance over existing leading methods by orthonormalizing only a top rank subset of singular vectors, enabling more efficient training of large models such as LLaMA-3 with reduced overhead.