Our project task was to create a model that, given a speaker ID, chat history, and an utterance query, can predict the response utterance in a conversation. The model is personalized for each speaker. This task can be a useful tool for building speech bots that talk in a human-like manner in a live conversation. Further, we succeeded at using dense-vector encoding clustering to be able to retrieve relevant historical dialogue context, a useful strategy for overcoming the input limitations of neural-based models when predictions require longer-term references from the dialogue history. In this paper, we have implemented a state-of-the-art model using pre-training and fine-tuning techniques built on transformer architecture and multi-headed attention blocks for the Switchboard corpus. We also show how efficient vector clustering algorithms can be used for real-time utterance predictions that require no training and therefore work on offline and encrypted message histories.
Citation: Weitzman, Tyler, and Hoon Pyo. “CloneBot: Personalized Dialogue-Response Predictions,” March 22, 2021, 13. arXiv:2103.16750 [Cs], March 30, 2021. https://cs.stanford.edu/~tylerw/CloneBot.pdf.
Coauthor: Hoon Pyo (Tim) Jeon, PhD Candidate, Graduate School of Business
Advisors: Professor Chris Manning (CS224N), Professor Andrew Maas (CS224S), Dilara Soylu (M.S., 224N TA 2021)
Review: Quite interesting project with an impressive number of custom datasets! Not only I quite enjoyed reading your paper, but I have also learned a lot. Congratulations on making it to the end of the quarter and all the best! - Dilara Soylu