"Using PET, the researchers fine-tuned an ALBERT Transformer model and achieved an average score of 76.8 on the SuperGLUE benchmark, compared to GPT-3’s 71.8."
https://www.infoq.com/news/2020/10/training-exceeds-gpt3/
 |
AI Training Method Exceeds GPT-3 Performance with 99.9% Fewer Parameters A team of scientists at LMU Munich have developed Pattern-Exploiting Training (PET), a deep-learning training technique for natural language processing (NLP) models. Using PET, the team trained a Tran www.infoq.com |