AI Advances in Chatbots

AI technology is advancing to the point that it has general uses for the mainstream public, rather than specialized applications. For example, chatbots use AI to perform many useful functions, such as automating tasks, suggesting fixes for programs and helping develop creative ideas. Chat Generative Pre-trained Transformer (ChatGPT) is a chatbot that enterprises are quickly adopting, primarily for functions like customer support and sales. Other chatbots like Google’s Bard has also begun competing for dominance in this sector to determine the course AI will take in the coming years. However, companies must also take precautions when implementing ChatGPT, due to the rapid evolution of this technology.

Overview

OpenAI developed ChatGPT and launched it as a prototype on November 30, 2022. It’s based on the GPT-3 family of language models that OpenAI also created. Human trainers fine-tuned ChatGPT with transfer learning, a technique that uses supervised and reinforcement learning techniques. ChatGPT has quickly gained the attention of its users, primarily due to its articulate, detailed responses across many bodies of knowledge.

The following timeline from International Data Corporation (IDC) provides more detail on the rapid series of events that have occurred since ChatGPT’s release:

Market Outlook

A 2021 report by research firm Gartner predicts that the market value of AI software will reach nearly $134.8 billion by 2025. The growth rate of this market was 14.4 percent in 2021, which analysts expect will increase to 31.1 percent in 2025, which would far outpace the overall growth of the software market. Chatbots will account for much of the AI market, which is making increasing use of AI and natural language processing (NLP) when responding to users. The latest chatbots provide more human-like answers and are able to engage in multiple exchanges. While the unmodified versions typically serve generalized purposes, organizations can often adapt them to perform more specialized tasks.

Generative AI is playing a particularly important role in the ability of chatbots to create new content. The following diagram from IDC shows the relationship and role of generative AI within the larger AI space:

What's the difference between ChatGPT and GPT-3?

ChatGPT and GPT-3 are both machine learning (ML) language models that generate human-like text responses to prompts. However, their sophistication is quite different, primarily due to the differences in their size and capacity. Muhammed A., a senior solutions architect at TripStax, discusses this issue in this blog post. He observes that ChatGPT is specifically designed as a chatbot, while GPT-3 is a more general-purpose application that can perform a wider range of tasks. As a result, ChatGPT should be more effective at generating conversational responses, while GPT-3 should be better at content creation and translation.

The following diagram from IDC illustrates the generative AI models that ChatGPT and GPT-3 use:

Note in the above diagram that ChatGPT is currently based on GPT-3.5.

ChatGPT itself can’t be customized because its language model isn’t accessible. OpenAI’s name would seem to imply that its software is open-source, but this isn’t the case. However, GPT-3 and Open AI's other large language models (LLMs) are available. Their underlying data is more specific to their objectives than ChatGPT, so these LLMs have greater control over their processes. As a result, LLMs may be able to provide better results, but they also need more assistance in fully realizing this advantage. These factors include more highly skilled trainers and better data curation, which will require more funding.

In addition, there would need to be a market for specialized LLMs large enough to justify this expense. For example, Microsoft's OpenAI Service uses ChatGPT, providing organizations and developers with a means of leveraging this chatbot. However, the latest version of Bing uses GPT-4, which is OpenAI’s latest version of GPT.

In addition, ChatGPT uses a much smaller text model of about 117 million parameters. In comparison, GPT-3 has 175 billion parameters with a size of about 45 terabytes (TB). ChatGPT isn’t connected to the internet, so it’s more likely to provide incorrect answers. Its accuracy is also limited by the fact that it lacks specific knowledge about world events that occurred after 2021, since that’s when its training period ended. Furthermore, the OpenAI FAQ states that ChatGPT can provide biased or even dangerous content.

ChatGPT uses a customized version of GPT-3.5 and also includes pre- and post-preparation steps as well as a screening process. Users submit prompts to ChatGPT that consist of the question and additional information. While they can’t directly access GPT-3.5, the specific wording of the prompt can have a significant effect on the quality of the response GPT-3.5 provides. Other chatbots do allow the user to directly access the underlying LLM.

Use Cases

ChatGPT quickly went viral after its launch in November, 2022, gaining one million users in just five days. It reached this user base much faster than any other software in history, largely due to ChatGPT’s ability to generate detailed, human-like responses. Analysts are already predicting use cases for customized versions of ChatGPT and its language models.

General Uses

The most common out-of-the-box use for this chatbot at the present time is a text-based web-chat interface, since it doesn’t allow access by an application programming interface (API) yet. GPT-3 does offer API access by itself, and Microsoft also plans to develop APIs for Azure OpenAI ChatGPT, which should be available in the near future. Organizations can also use the unmodified version of ChatGPT to create content and manipulate email text to alter text, whether it’s to change the tone, simplify the content or summarize it. These uses would only require a small investment from an enterprise.

ChatGPT can generally improve content creation and automate other tasks, while also providing users with a fast, engaging experience. This chatbot does best with simple question-and-answer type prompts, such as, “What’s the distance between New York and Los Angeles when driving by car?” ChatGPT also offers many possibilities for producing draft text of a specified length and style, which the user can then review and revise. Common uses of this capability include generating drafts of essays, instruction manuals, marketing descriptions, letters of recommendation, social media and training guides.

Customer Service

In the near term, specific uses of chatbots like ChatGPT are likely to include the creation of material for email sales campaigns and suggesting answers to customer service agents. The ability of chatbots to produce detailed, human-like responses also means that they could increase automation in these areas. They can also generate summaries of articles, conversations, emails and web pages, which could be particularly helpful in customer service. Future improvements to GPT that could improve ChatGPT’s performance in this area include greater knowledge of business logic, enterprise context, permissions, service descriptions and tone.

Software Engineering

While users can’t directly modify the GPT-3 LLM that ChatGPT uses, they can modify it separately for use in another chatbot engine. In this scenario, GPT-3 could then be used like any other LLM. For example, users can add data to GTP-3 or tune its parameters. In addition, users can influence a chatbot’s response by altering the wording of a prompt, a technique known as prompt engineering.

Effective prompt engineering will be essential for using ChatGPT in software development, an application that’s just emerging. This capability could allow chatbots to generate code, translate code between languages, verify code and create comments. The first uses of ChatGPT for improving the development process is most likely to occur in integrated developer environment (IDEs), since these environments already provide extensive resources for developers.

Assume, for example, that a programmer was having difficulty with a section of code. They could enter a phrase to the chatbot that said something like, “How do I get this code to work?” The chatbot’s first response would be unlikely to fix the problem, but it could generate follow-up questions from the user that would allow it to devise a solution. ChatGPT can also generate code from a programmer’s description of a problem to solve, convert code from one language to another, suggest fixes and document the code. OpenAI provides examples of coding queries and ChatGPT’s responses that illustrate this scenario.

Additional Uses

Additional uses of ChatGPT and GPT-3 include sales and marketing, especially for potential customers on a website. A chatbot could provide product descriptions and make recommendations based on user input, although this application would require considerable customization to provide the chatbot with the necessary context for each organization. In addition, chatbots are already being used as personal assistants to compose emails, make replies, manage schedules and draft basic documents.

Chatbots have the ability to serve as tutors by creating personalized learning experiences for students. They can also be used in healthcare to simplify medical information, including treatment recommendations.

Risks

The newness of AI-based chatbots creates a number of risks for users, who may not understand the limitations of its underlying data, analytics and security. For businesses, one of the biggest concerns is that ChatGPT’s large database can cause it to provide responses that are too long to be useful. In particular, this chatbot can generate lengthy prose in natural language that contains little information of value. Even worse is the possibility that it can make statements that are factually incorrect. As a result, users should always review a chatbot’s responses for accuracy, usefulness and appropriateness.

Additional risks of using chatbots include the chance that it could expose classified and personal identifiable information (PII), making it crucial to avoid feeding sensitive data to a chatbot. Companies that use chatbots should also ensure they only work with vendors that have strong policies on data governance and ownership policies. This practice will help minimize the possibility of another party introducing sensitive data to a chatbot.

OpenAI has been careful to keep ChatGPT users’ expectations realistic, especially with respect to its risks. CEO Sam Altman advised users in December 2022 that ChatGPT’s capabilities were “incredibly limited” at that time. He added that no one should use it for anything important at this time. Altman also emphasized that his company’s chatbot is a work in progress that still requires a lot of work to improve its accuracy and robustness. He summarized his remarks by saying that creative inspiration is the best use of ChatGPT at this time.

Gartner closely agrees with this assessment in its 2021 market report, saying that AI-based chatbots are still in a very early stage of development. It adds that the technology is being strongly hyped as a result of the developing competition in this space. Gartner also advises against over pivoting when using chatbot responses.

Solutions

Mitigating the risks of chatbots should include innovative thinking about work processes before integrating chatbots. In addition, organizations need to develop data usage and governance guidelines that specifically address the use of AI. Those policies should educate employees on the inherent risks of ChatGPT. Specifically, they should prohibit employees from asking ChatGPT questions that reveal sensitive information. Furthermore, organizations should develop processes that allow humans to manually report issues directly to top executives like CEOs and CIOs.

Gartner analyst Bern Elliot also recommends that enterprises use the Azure Open Service ChatGPT, rather than OpenAI’s ChatGPT. The main reason behind this preference is that Microsoft will provide Open Service with the same enterprise-level compliance and security controls that it does for all its other products. In particular, companies using sensitive information should only use Azure to access ChatGPT. Microsoft has also said that it will enable confidentiality and security services for Azure OpenAI like it already does for other Azure services.

Summary

The developing AI chatbot market is characterized by the competition between OpenAI’s ChatGPT and Google’s Bard. OpenAI was founded in 2015, when Microsoft and Google had just informally agreed to stop developing search engine technology until 2021. Early investors included Amazon Web Services (AWS), Infosys, YC Research, Elon Musk and Sam Altman.

The company went public in 2019, when Altman became OpenAI’s CEO. That same year, Microsoft invested $1 billion in OpenAI, which wasn’t working on search engines at that time. However, in early 2023, Microsoft announced its plans to put an additional $10 billion into OpenAI, specifically for the purpose of developing ChatGPT.

At the same time, Microsoft announced that it was upgrading its Bing search engine to use GPT-4, which is currently the latest version of GPT. Microsoft hopes GPT-4 will provide Bing with the boost it needs to overtake Google Search, which has long dominated the search engine space. In return, Google has also announced the release of Bard, a chatbot that will use the entire internet as its database. This AI-based service uses Language Model for Dialogue Applications (LaMDA) as its language model.

A recent competition between ChatGPT and Bard gave ChatGPT a victory of 23 to 16, based on a series of challenges evaluated by a panel of communications experts. However, ChatGPT’s static database will make this lead difficult to maintain as information changes over time.