By default, when you send a prompt to the OpenAI Completions endpoint, it computes the entire completion and sends it back in a single response. If you’re generating very long completions from a davinci-level model, waiting for the response can take many seconds. As of Aug 2022, responses from text-davinci-002 typically take something like ~1 second plus ~2 seconds per 100 completion tokens.

To stream completions, set stream=True when calling the Completions endpoint. This will return an object that streams back text as data-only server-sent events.

In the test codes, if don’t apply stream, we got “Full response received 5.26 seconds after request”. it’s quite slow.

If you want to get the response faster, you can ‘stream’ the completion as it’s being generated. This allows you to start printing or otherwise processing the beginning of the completion before the entire completion is finished.

tried the sample codes with a for loop and set the stream=True in openai.Completion.create(). the result is much faster.

The openai package provides several classes that can be used to interact with the OpenAI API and work with the pre-trained models. Here are some of the main classes provided by the openai package:

1. openai.api_key : This class is used to set the API key for the OpenAI API.
2. openai.Completion : This class is used to generate completions for a given prompt using a pre-trained model. It provides several methods for controlling the generation, such as create(), list(), and details().
3. openai.Davinci : This class provides an interface to the “Davinci” model, which is a high-capacity variant of GPT-3.
4. openai.DavinciCodex : This class provides an interface to the “Davinci Codex” model, which is a variant of GPT-3 that is fine-tuned for code generation.
5. openai.Engines : This class provides an interface for listing and inspecting the available models and engines that can be used with the OpenAI API.
6. openai.GPT : This class provides an interface for the original GPT models.
7. openai.ImageCompletion : This class provides an interface for image completion using the DALL-E model.
8. openai.LanguageModeling : This class provides an interface for language modeling tasks.
9. openai.Completion : This class provides an interface for the completion tasks, such as text completion, translation, summarization, and question answering.

This is generated by chatGPT about chatGPT’s classes, obviously there are two duplicated/different explanation about openai.Completion.