OpenAI Launches Plugin Feature, Opens Up Plugin Platform 161

Following the release of the new GPT-4 engine and Whisper API in March, Openai announced on Thursday that it has begun introducing plugins. These will enable chatbots to interact with third-party APIs, tailor their responses to specific situations defined by developers, and expand the operational scope of the bots' capabilities.

OpenAI has already implemented initial support for plugins in its large model. Plugins are tools specifically designed for language models, with safety as a core principle, helping large models access up-to-date information, run computations, or use third-party services.

Suppose you want to develop a chatbot with which users can talk about sports. Before the latest GPT-4 upgrade, the chatbot could only discuss games and scores that happened in the past, particularly up to 2021, when the training data for GPT-3 was collected. It would not pull real-time data, nor would it even be aware of 2022's existence. With plugins, you would be able to attach large model capabilities to your existing code stack, where it would be able to perform anything from retrieving real-time information calls (sports scores, stock prices, breaking news) to extracting specific knowledge base information (such as internal company documents or a personal cloud). It could even take actions on behalf of users, such as booking flights or ordering takeout—think of an installable Google Assistant built by OpenAI personnel.

The plugins themselves are still in an early alpha stage, with limited availability. OpenAI has allowed a small number of partner companies early access, including Expedia, Instacart, KAYAK, OpenTable, Shopify, Slack, Wolfram, and Zapier, to use them within their existing applications. You will need to add your name to a waitlist to try it out in person.

"AI models act as smart API callers. Given an API specification and a natural-language description of when to use the API, the model proactively calls the API to perform actions," the OpenAI team wrote. "For example, if a user asks, 'Where should I stay in Paris for a few nights?', the model can choose to call the hotel booking plugin API, receive the API response, and generate a user-facing answer that combines the API data and its natural language capabilities."

OpenAI also noted that using plugins to bridge the knowledge gap between what the model was trained on and what has happened since should help reduce the tendency of artificial intelligence to hallucinate when answering complex questions. "These references not only enhance the model's usefulness, but also enable users to assess the trustworthiness of the model's output and carefully check its accuracy, thereby potentially mitigating risks associated with over-reliance," the team wrote.

The additional capabilities and information provided to the model through its plugins also substantially increase the chances of the model returning problematic responses. To avoid the $10 billion Google acquisition of Bard, OpenAI reportedly put these plugins through extensive stress testing. "We have conducted red team exercises internally and with external collaborators, revealing many potentially concerning scenarios," the team wrote. They plan to use these findings to "inform safety-by-design mitigations" to increase transparency and deter plugins from engaging in dangerous behavior.