We have studied recipes for large-scale open domain chatbots, and are releasing 90M, 2.7B and 9.4B parameter models with SOTA results.
Blending conversational skills (personality, empathy, knowledge) and detailed decoding are key, in addition to scaling.
Paper: https://arxiv.org/abs/2004.13637
Project: https://parl.ai/projects/recipes
Blog post: https://ai.facebook.com/blog/state-of-the-art-open-source-chatbot
with Stephen Roller Emily Dinan Naman Goyal, Dexter Wang, Mary Williamson, Yinhan Liu, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston