ChatGPT AI: Enhance ChatGPT with Web Search (Difficulty: 4)

[info-creaticode]

Introduction

 

Large language models like ChatGPT learn all their knowledge from their training data, so they would not know about any new information after the cut-off date of the training data.

For example, if we ask ChatGPT this question: “Which team won the Copa America 2024?” it will not be able to answer since, at the time of writing, the cutoff date of ChatGPT is October 2023.

 
 

Enhancement to ChatGPT

 

To fix this issue, we can “teach” ChatGPT to ask for help. To illustrate the idea, we can think of ChatGPT, the user and our program as 3 characters, and their conversation may look like this:

• User to Program: Who won the Copa América 2024?
• Program to ChatGPT: Hey ChatGPT, can you try to answer this question: Who won the Copa América 2024? BTW, if you ever find you need to search the web to get some new information, I can help.
• ChatGPT to Program: That question is beyond my cutoff date. Please do a Google search on “Who won the Copa América 2024?”
• Program to ChatGPT: Sure. Let me Google it… OK, I got it, here is the research result: Argentina
• ChatGPT to Program: Thanks. Based on the search result, here is the answer for the user: Argentina
• Program to User: Argentina

 
Now, let’s build this workflow step by step.

 
 

Step 1 - Remix the Helpful Assistant Project

 

play.creaticode.com/projects/6531b7e60fdce080a4481c1d

 
 

Step 2 - Update the Initial Prompt

 

Since we will write a fairly long prompt, let’s still use the “comment” area to compose the prompt.

First, add the comment area by right-clicking the green flag block:

 
Next, copy the following prompt into the comment area:

You are a friendly assistant, and I am an agent who can help you. Make sure your response is concise and simple to understand. Do not say anything inappropriate.

When you need a web search before answering a user question, instead of responding to the user, you should respond to me using a special tag "WEB:", followed by the query to search for. I will provide the answer to you, and then you can generate the response to the user.

Now say "hi"

 
This prompt contains 3 parts:

• It first introduces the context, where ChatGPT is playing the role of a friendly assistant, and we (our program) are an agent that’s helping it.
• Then, a special instruction is given to ChatGPT so that it asks us to do a web search when it needs to get some new information online. The special syntax it should use is is also specified, which is the “WEB:” tag.
• Lastly, we ask ChatGPT to say “hi” and wait for user input.

 
Next, you can copy this entire prompt from the comment area into the request block, then press Enter to confirm:

 

 
Now let’s test with the same question again, and we will get a “WEB:” response from ChatGPT:

 
This is a great improvement: ChatGPT has learned that it can use a web search tool to help it answer the question. Of course, we should not show this to the user since the final answer has not yet been given. Let’s change that below.

 
 

Step 5 - Enforce the Output Format

 

ChatGPT may not always use the same format every time. For example, it might respond like this:

 
The problem with this response is that our program will have a hard time extracting the exact query to be used for web search. To force ChatGPT to use the correct format, we can make 2 changes to our prompt:

• Add a note, which emphasizes the format we expect.
• Give an example, which demonstrates the right way to respond.

Here is the updated prompt:

You are a friendly assistant, and I am an agent who can help you. Make sure your response is concise and simple to understand. Do not say anything inappropriate.

When you need a web search before answering a user question, instead of responding to the user, you should respond to me using a special tag "WEB:", followed by the query to search for. I will provide the answer to you, and then you can generate the response to the user. Note that when you need a web search, you have to start with "WEB:", and say nothing else except for the query. Avoid asking user if you should do a web search since the user does not know about the web search tool.

For example:
user: Who won the 2024 Olympics 100m sprint for men?
you: WEB: winner of 2024 Olympics 100m sprint for men
system: At the 2024 Olympics in Paris, American sprinter Noah Lyles triumphed in the men’s 100m final
you: American sprinter Noah Lyles

Now say "hi"

Now please copy this new prompt to your comment area, then copy that into the system request block. This will help ensure ChatGPT sticks to the format we specify.

 
 

Step 6 - Check if the response starts with “WEB:”

 

Next, we will handle ChatGPT’s response differently depending on whether it starts with the special tag of “WEB:”. We will only add some additional steps when ChatGPT’s response starts with “WEB:”.

 
 

Step 7 - Run the web search

 

When we find ChatGPT is asking us to do a web search, we can extract the extract query sentence from ChatGPT’s response, then run a web search using that query sentence.

 
The logic is the following:

1. The response from ChatGPT starts with “WEB:”, and we only need the expression that comes after that. So, we can use the “substring” block to get the substring of the response starting from the 5th letter. For example, if the response variable is “WEB: winner of 2024 Olympics 100m sprint for men”, then the substring will be " winner of 2024 Olympics 100m sprint for men".
2. We can use the “web search” block from the AI category to search the web and store the top 3 results in the “result_table”.

 
Now, let’s try to get some search results by running the program again. To view the “result_table”, we have to remove the chat window from the stage. We can run the “remove all widgets” block by itself.

 
After that, show the “result_table” on the stage by checking its checkbox:

 

You will see the result_table contains 3 columns: title, link and snippet. The snippet is simply a short summary of that web page.

 
 

Step 8 - Aggregate the Search Result

 

To return the search result to ChatGPT, we first have to aggregate the content in the result_table into a single variable named “answer”. For example, we can compose the answer using a few join blocks like this to include the top search result:

 
Note that “\n” represents line return, which helps format the answer text better. As a result, the “answer” variable will become this:

 
We can use the same code to join the second and third search results. And at the end, we will tell ChatGPT to generate the response for the user:

 
 

Step 9 - Give ChatGPT the Search Result

 

Now we will pass the result result back to ChatGPT. Note that we should send the answer using the “system request” block, which makes it clear to ChatGPT that this answer is from the system, not the user. We will still ask ChatGPT to store the response in the “response” variable, which will overwrite the previous response.

 
After this step, the “response” variable contains the new response based on the web search result, and we can continue to append that to the chat history.

If you test with the same request again, ChatGPT will produce the correct answer!

 
 

Further Improvements

 

In this tutorial, you have learned how to enhance ChatGPT with a web search tool. What can you do to make it better? Here are some ideas worth trying:

• Improve the Prompt: If you test the program more, you may still find that sometimes it would fail to run the web search. It would help if you could add another example conversation to the prompt. You can also try to refine the instructions in the prompt to tell ChatGPT what the expected behavior is.

• Fetch the Page Content: The current program only feeds ChatGPT with the title and snippet of each web page in the search result. For more complex questions, this may not contain the information the user is asking for. A further enhancement is to give ChatGPT another tool: fetch web page. To do that, you need to modify the prompt with a few changes:

  • Give ChatGPT the link URL for each search result
  • Tell ChatGPT if it needs the full content of any page, it can issue another special command “FETCH: URL of the page”.
  • When we receive the FETCH request, fetch the content of that page using the “fetch web page as markdown” block from the Cloud extension, then send the content back to ChatGPT.