“Did She Get Off the Plane?!”
GPT-3 is the third version of the generative pre-trained transformer which can be used for text translation, summarization, generation, and answering questions. It is very easy to use by simply providing the model with a prompt, and the fact that it is open source allows anyone – even beginners – to get their hands dirty. While GPT-3 can generate believable text, its creativity generates outputs far from the facts and can sometimes lack common sense, so one has to decide when to use GPT-3 for text generation.
I wanted to play around with GPT-3 for text generation just for fun and to get an idea of how it works. Specifically, the prompt I am going to give the model is a summary of the last episode of Friends, up until the point where Ross asks the question, “did she get off the plane?”. So let’s see…how would GPT-3 re-write the end of Friends?
Let’s first understand GPT-3 a bit better.
As I mentioned, GPT stands for generative pre-trained transformer. At a high level, a transformer is a type of deep learning model that can track the relationship of sequential data – like words in a sentence. It uses a technique called self-attention to detect how different elements of the sequence depend upon each other. Understanding the relationship between words is what allows the transformer to predict the probability of a word being next in a sequence.
GPT-3 specifically is pre-trained on over 45 TB of data from multiple sources including web crawls, Wikipedia, book corpi, and more.1 GPT-3 can also be fine-tuned by training it on a relevant dataset, but for this post, I will be keeping things simple and will be generating text with the model as is.
How to use GPT-3?
I used EleutherAI‘s GPT-3 model with Python in a Jupyter notebook on Google Colaboratory.
There are 5 simple steps which include:
- Installing the transformer library from HuggingFace (a platform for building/sharing applications, models, and datasets)
- Importing the transformer and pipeline (a simplified API that extracts code from the transformer library)
- Generating the GPT-3 text generation model using the pipeline() function
- Entering the text sequence you want to use as the prompt for the model
- Running the model on the prompt to generate new text after where the prompt text ends
Take a look at the code below:
#install transformers library from HuggingFace
pip install transformers
#import transformers and pipeline
import transformers
from transformers import pipeline
#create gpt model with pipeline function
generator = pipeline('text-generation', model = 'eleutherai/gpt-neo-1.3b')
#save your prompt to a variable
sequence = ("After ending up at the wrong airport and calling Monica to verify their info, Ross and Phoebe........")
#generate text output using prompt
result = generator(sequence, max_length = 400)As you will notice in my code above when running the model I did have to indicate a desired max length for my text output. Note, the max_length also includes the length of the input prompt. Since my input was 236 words, I set the max_length to 400 tokens so that I could have a couple new sentences generated.
The GPT-3 model also contains some parameters for easy tuning which I will talk about a bit more as I go through the results that I got below.
Results and Comparison of Parameters.
Do Sample:
For the first model, I added the optional parameter “do_sample = True”. This parameter prevents the model from returning the most likely (optimal) word at each step, which could lead to a very repetitive output, and to instead do sampling which would provide a more natural language output.
After ending up at the wrong airport and calling Monica to verify their info, Ross and Phoebe race to the right airport. Afraid they are not going to make it in time to catch Rachel (who is actually already on the plane), Phoebe calls Rachel, telling her there's something wrong with the plane's 'left phalange'. The guy sitting next to Rachel freaks out and alarms the rest of the passengers, making everyone get off. This diversion is perfect for Ross who catches Rachel just before she takes off again (also thanks to Pheobe's screaming, which makes Rachel turn around at the gates). Ross confesses his love for Rachel and says he wants to be with her, but it's too much for Rachel to take at the moment and all she can say to Ross is 'I'm sorry, I'm sorry. I'm so sorry' and passes through the gates again. Ross doesn't meet the others as Monica's apartment because he is too sad about how he left things with Rachel. He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. Rachel's response to him is 'No, Ross, I told you, I love you and I want to get off the plane. I didn't mean to leave you. I didn't mean to leave you, I love you.' Rachel leaves the next hour before she can explain her exit. Ross apologizes to Monica, admitting his guilt and that he loves Rachel too, but Monica tells him it's too late to change the past and makes him promise never to act on them ever again. He then goes to Rachel's apartment and apologises for not being the person to....
Above you can see my prompt in bold, and the text generated by GPT-3 right after.
I was pleasantly surprised by the output, especially the first 3 sentences that GPT-3 generated, where it seems that Rachel is actually responding to Ross’s question about whether or not she got off the plane. While we all know that Ross isn’t actually talking to Rachel at this point in time, if he was, Rachel’s response saying that she wants to get off the plane, that she didn’t mean to leave him, and that she loves him all makes sense. However, after that point, it doesn’t seem to make much sense.
I also just want to show below what the output would look like if I had set do_sample = False and as you can see, it ends up being repetitive as the model looks to only return the optimal output at each step. (Note: for the rest of the post I’m shortening the prompt in what I’m displaying because it takes up too much space.)
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. Rachel says she did, but Ross is still sad. Rachel says she is going to go to the bathroom and Ross says he will wait for her. Rachel goes to the bathroom and Ross follows her. Rachel comes out of the bathroom and Ross says he is sorry, but Rachel says she is not going to leave him. Ross says he is sorry too, but Rachel says she is not going to leave him. Ross says he is sorry too, but Rachel says she is not going to leave him. Ross says he is sorry too, but Rachel says she is not going to...
Top K:
The top_k parameter is used to limit the number of probable tokens (words) by filtering for the next k likely words and redistributing probabilities among these next k words. Limiting this number helps to eliminate words that may not make as much sense and in turn, generates better storytelling. I tested out the top_k parameters with k = 20, 50, and 70 below.
Top_k = 20
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. He hears a loud 'clunk', and turns back to see Rachel standing next to her suitcase, a suitcase that looks exactly like Monica's, and says 'Oh my God, she's gone'. Rachel is devastated, and Ross says, 'I'm sorry, I'm sorry. I can't do this. I'm sorry.' Rachel is crying, and Ross says, 'I'm sorry too, I was a jerk', and goes to Rachel. Rachel is crying as well, and says, 'I'm so stupid.' Ross tries to comfort her and tells her to...
Top_k = 50
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. The voice inside Ross's head says 'Yes' and Ross thinks about how it will feel to have Rachel come home, but he gets off the plane. Ross feels horrible when he leaves the airport to Rachel's apartment and he doesn't get to say good-bye to Phoebe, who is there because she needs to get him some drinks. Ross finds a phone number on his flight manifest and calls the number on the passenger list. Rachel says she has no idea how he can see her, but he does, and he gives her his phone number. That...
Top_k = 70
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. However, Monica has changed her mind and has left the plane to the rest of them. Monica says 'I know what I want', but Rachel does not accept it and screams that she hates him. Ross says he is not ready for all of the ways he thinks Rachel feels and that he is scared he will lose her forever. Monica leaves the plane and Ross, Phoebe and Monica exit the airport before Rachel does. She screams at them, telling them they are all in danger of losing each other in love. Ross says the only way to stop her from losing Rachel...
Comparing different top_k values is very interesting. Based on the outputs, I would say a smaller k leads to better results initially. Where I used k = 20, it says Ross turns around to see Rachel with a suitcase, which makes sense, and their conversation about being sorry, and crying also makes sense. Where I used k=50, the initial text saying “The voice inside Ross’s head says ‘Yes’ and Ross thinks about how it will feel to have Rachel come home” is something that could follow the prompt, but after that, while still airport-related, the storyline stops making sense. Lastly, when k = 70, while also airport/plane related, the storyline does not make sense but there is this one line that I think could follow the prompt: “Ross says he is not ready for all of the ways he thinks Rachel feels and that he is scared he will lose her forever.”
Temperature:
Another parameter is temperature which controls the randomness of the output, or how creative we’ll allow GPT-3 to be with its output. Typically, the smaller the temperature the less random the output is, so tested out temperatures of 0.5, 2.5, and 5.0 below.
Temperature = 0.5
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. Rachel tells him she did, but then says she is scared she won't be able to go through with it. Ross tells her that he loves her and that he will help her get off the plane. Rachel is so happy she can barely speak, and Ross tells her he will be back. Ross and Monica arrive at the airport, and Ross tells Monica he is going to get Rachel to the airport. Monica tells him he is a jerk and that he needs to stop being so selfish. Ross tells her that he will help her get off the plane, and Monica...
Temperature = 2.5
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. He decides she said 'Sorry!' just before getting off of again - they just can use another little distraction right now... Ross tells Amy who was holding their drinks with both of then in mid conversation that Rachel, their 'curee', did finally get off but only went so far because Monica didn't accept everything they (in fact, her sister Brooke in part, was completely beside himself saying - the sister) were telling for love as it seemed he couldn` have had such as love for it wasn?. 'Yeah you? What... Ross!' But Amy...
Temperature = 5.0
He goes home and checks his messages, and finds a message from Rachel. She admits repeatedly she loves him too, and by saying it over and over she realizes she wants to get off the plane. The message is cut in the middle of an argument between Rachel and a flight attendant. Ross tries to hear the rest of the message, asking himself 'Did she get off the plane?'. He walks into someone's yard after hanging his suitcase in place after realizing why Phnebe was called a maniac in her call earlier and thinking her mother could've gone home right as Chandler put into call the night of leaving and his voice as smooth sounding in its ability be more melant or as having better volume, though is that something we expect at our lives?! (Note – Ross really shouldn'd'r keep such things) In the parking lot you're hearing footsteps behind it was an accident..
Comparing the outputs of different temperatures, you can see how increasing the temperature leads to a more random storyline. When temperature = 0.5, the text initially says that Rachel won’t be able to go through with getting off the plane and that Ross will come and help her. While we know they aren’t actually speaking to each other at this point in time, if they did, this would make sense. The only part that doesn’t really make sense in this output is the conversation between Ross and Monica. As soon as I increased the temperature to 2.5 and above, the randomness increase leads to storylines that don’t really make sense.
Conclusion
After playing around with GPT-3, I’ve definitely come across some interesting results including outputs that make sense as a continuation of the Friends episode, and some results that were very creative and may need to be a part of a new storyline altogether. While GPT-3 itself is very good at generating text similar to natural language, I think that its creativity lacks the common sense and logic needed to generate text without requiring some further human input.
For future work, one thing I would like to explore is actually fine-tuning the model by training it on some new dataset to potentially write a brand-new plot.
If you want to try out GPT-3 for yourself, depending on what kind of output you want make sure to tweak the parameters and compare outputs. Overall, it seems that a shorter max_length will generate better text since the initial text generated makes more sense than the text generated further from the prompt. For top_k and temperature, a smaller k and temperature seem to generate better results to match the prompt. If you want something more creative – increase those numbers!
Sources:
Journey to Data Scientist 