Chatbot Start Prompt for GPT-J

A Hugging Face forum user identified as Eichhof reported that using EleutherAI's GPT-J 6B model as a chatbot produces inconsistent conversations because the model continues the sample conversation prompt instead of responding to new user input. Forum user PanEa advised adding example conversations as few-shot prompts and inserting a special line or token to separate each episode, and Eichhof asked how many conversations and turns are reasonable given GPT-J's 2,024-token context limit.

Eichhof https://discuss.huggingface.co/u/Eichhof 1 Hello I’m using GPT-J EleutherAI/gpt-j-6B as a chatbot. As a prompt, I provide a sample conversation as shown below. When now a new conversation starts, I append the input of the user to this sample conversation “Hello, how are you doing?” in the example below . Now, the problem is that the conversation is sometimes inconsistent because GPT-J might want to continue the sample conversation but the new user input could break that. How can this be solved? This is a discussion between a Human and a Chatbot. Human: Can you do push-ups? Chatbot: Of course I can. It’s a piece of cake Believe it or not, I can do 30 push-ups a minute. Human: Really? I think that’s impossible Chatbot: You mean 30 push-ups? Human: Yeah Chatbot: It’s easy. If you do exercise everyday, you can make it, too. Human: Hello, how are you doing? Chatbot: Eichhof https://discuss.huggingface.co/u/Eichhof 2 Does somebody have an idea? PanEa https://discuss.huggingface.co/u/PanEa 4 you may add some example conversations/episodes as few-shots, which can lead the model to generate much better. also you need a special line or token to seperate each episode incase the information in previous episodes been used in the last one Eichhof https://discuss.huggingface.co/u/Eichhof 5 Thank you very much for your answer. How many conversations and how many turns per conversation do you think are reasonable given the restricted history of 2024 tokens and that I also want to store the history of the conversation? What special line / token would you use? I agree with your point. Asking ChatGPT to explain things step by step is very useful because it makes difficult topics easier to understand. I also find that asking for examples helps connect the explanation to real tasks. I agree with your point. Asking ChatGPT to explain things step by step is very useful because it makes difficult topics easier to understand. I also find that asking for examples helps connect the explanation to real tasks.