Use OpenAI Tools: Web Search, Code Interpreter, File Search and Computer Use

Web search is available in the Responses API as the generally available version of the tool, web_search, as well as the earlier tool version, web_search_preview. To use web search in the Chat Completions API, use the specialized web search models gpt-4o-search-preview and gpt-4o-mini-search-preview. Web search is limited to a context window size of 128000 (even with gpt-4.1 and gpt-4.1-mini models).

The Code interpreter is here to assist you by writing and running Python code in a safe, sandboxed environment. Any files and images generated by the model are thoughtfully returned as annotations on the assistant’s message, helping you keep everything organized. The container_file_citation annotations provide clear pointers to the files created in the container, including the container_id, file_id, and filename. You can easily parse these annotations to access download links or process the files in a way that best suits your needs.

File search is a tool available in the Responses API. It enables models to retrieve information in a knowledge base of previously uploaded files through semantic and keyword search. By creating vector stores and uploading files to them, you can augment the models’ inherent knowledge by giving them access to these knowledge bases or vector_stores. To use it, need to have set up a knowledge base in a vector store and uploaded files to it.

The Retrieval API allows you to perform semantic search over your data, which is a technique that surfaces semantically similar results — even when they match few or no keywords. Retrieval is useful on its own, but is especially powerful when combined with our models to synthesize responses. Certain query styles yield better results, so we’ve provided a setting to automatically rewrite your queries for optimal performance. Enable this feature by setting rewrite_query=true when performing a search. Attribute filtering helps narrow down results by applying criteria, such as restricting searches to a specific date range. You can define and combine criteria in attribute_filter to target files based on their attributes before performing semantic search. Use comparison filters to compare a specific key in a file’s attributes with a given value, and compound filters to combine multiple filters using and and or. If you find that your file search results are not sufficiently relevant, you can adjust the ranking_options to improve the quality of responses. This includes specifying a ranker, such as auto or default-2024-08-21, and setting a score_threshold between 0.0 and 1.0. A higher score_threshold will limit the results to more relevant chunks, though it may exclude some potentially useful ones.

The image generation tool allows you to generate images using a text prompt, and optionally image inputs. It leverages the GPT Image model, and automatically optimizes text inputs for improved performance.

Computer use is a practical application of our Computer-Using Agent (CUA) model, computer-use-preview, which combines the vision capabilities of GPT-4o with advanced reasoning to simulate controlling computer interfaces and performing tasks. Computer use is available through the Responses API. It is not available on Chat Completions. Playwright is a framework for automated browser testing — kind of like a robot that can open a browser, click buttons, type text, and check whether a website behaves the way it should.

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.