DeBERTa is a large scale pre-trained language models which surpass T5 11B models with 1.5 parameters and achieve human performance on SuperGLUE.
microsoft/DeBERTa: The implementation of DeBERTa (github.com) (opens in new tab)
Research