gpt 💤

This is an alpha demo of the 175B parameter model.

WARNING: This model will generate MANY offensive things. Due to this being an alpha demo, NO safety measures are in place.

Tips for better generation:

• Use the examples to get an idea of how to control it.
• Avoid spaces at the end of your query. New lines are great though.
• Simple questions trigger chatbot mode. Try "Question: ... Answer:" to get more factual responses.
• Using proper capitalization and punctuation can help avoid chatbot mode. Sometimes.

Examples: News ML paper Chatbot QA Poetry

FAQ and Nerd stuff

How does it work?

OPT-175B is a large Language model. It predicts the next word based on all the previous words. It's the same idea as the predictive auto-complete in your phone or email, but supercharged.

What's the model trained on?

A combination of CCNews, CommonCrawl, DM Mathematics, Enron corpus, Project Gutenberg, HackerNews, OpenSubtitles, OpenWebText2, USPTO, Wikipedia, BookCorpus, Stories, and Pushshift.io Reddit.

How up to date is the model?

The newest data in the model goes to roughly through September 2021, but most of the data is older. The model knows about COVID-19, knows Joe Biden is president, but isn't aware of the rebrand to Meta.

You can encourage the model to focus on newer information by putting "2021" somewhere in your prompt.

It's not following my instructions or answering my question.

The model doesn't work well with declarative instructions, or point blank questions. See the Examples to understand how to best use the model.

It said something deeply offensive, or factually incorrect!

This is a well-known problem with language models, and there is great research being done at Meta AI to address these problems. Due to the alpha nature of this demo, we have not been able to incorporate that work yet.

It's really repetitive!

Yes, it definitely can be. We're sampling from the model, so you can try generating again and see if it helps. We will address repetition in future iterations.

What are you doing with my data in this demo?

During this alpha stage, we are not collecting or storing any data from this demo. We encourage you to create your own document with feedback or analysis of the model, and provide a link in the comments of the workplace post.

It can only generate 512 tokens?

The model has a context length of 2048, but limited 512 in this demo for compute reasons.

What generation algorithm is running?

For this demo, we use Nucleus sampling (p=0.9) with a softmax temperature of T=0.7.