Human language technologies
Researching software and systems that bridge the linguistic divide between people and machines to make communicating with computers as natural as speaking with family and friends.
Video
Understanding Knowledge Distillation in Neural Sequence Generation
Sequence-level knowledge distillation (KD) — learning a student model with targets decoded from a pre-trained teacher model — has been widely used in sequence generation applications (e.g. model compression, non-autoregressive translation (NAT), low-resource translation, etc).…
Publication