ChatGPT AI: Enhance ChatGPT to Do Math Calculations (Difficulty: 3)

CreatiCode

Introduction

Large language models like ChatGPT are really good at processing text and figuring things out, making them seem almost scarily smart. However, as you use them more often, you will find they are terrible at many other tasks.

One such task is math calculation. For example, suppose we test ChatGPT with a multiplication question:

With any calculator, we can easily find out 3516 times 357 equals 1255212. ChatGPT makes this mistake because it doesn’t actually calculate the result. Instead, it has seen many similar multiplication in its training data, so it tries to guess what the output number should look like.

Since math calculations are critical in many tasks, we must find a way to help ChatGPT calculate correctly. In this tutorial, you will learn a simple solution. The basic idea is that we will teach ChatGPT to ask for help with calculations, and we will help it calculate the correct answer in our code.

Step 1 - The Starting Project

We will use the following project as the starting point. Please remix it into your own project, and name the project “ChatGPT with Calculation Help”:

play.creaticode.com/projects/6531b7e60fdce080a4481c1d

Step 2 - Verify the Issue

Before we make any improvements, please try the math question in this project to verify that ChatGPT will not get it correctly without our help:

Step 3 - Sequence Diagram

Before we make any changes, let’s discuss the design of this project using sequence diagrams.

There are 3 entities in our project:

User: the user who inputs questions
ChatGPT: the AI model that answers user questions
System: this is our code

Currently, the system simply relays the messages between the user and ChatGPT like this:

To help ChatGPT do math calculations, we can change the diagram so that sometimes ChatGPT can talk to our system and get answers, as shown by the 2 additional blue arrows below:

The beauty of this design is that the new messages between the system and ChatGPT can be hidden from the user, so it appears to the user that ChatGPT is answering correctly.

Step 4 - Update the Initial Prompt

Since we will write a fairly long prompt, let’s still use the “comment” area to compose the prompt. First, add the comment area by right-clicking the green flag block:

Next, copy the following prompt into the comment area:

You are a friendly assistant. 

When you need to do a math calculation, instead of responding to the user, you can respond to the system using a special tag "CALC:", followed by the expression to calculate. The system will provide the answer to you, and then you can generate the response to the user.

Now say "hi"

This prompt contains 3 parts:

It first introduces the context, where ChatGPT is playing the role of a friendly assistant.
Then, a special instruction is given to ChatGPT so that it responds to the system when it needs to do math calculations. The exact syntax is also specified.
Lastly, we ask ChatGPT to say “hi” and wait for user input.

Next, you can copy this entire prompt from the comment area into the request block, then press Enter to confirm:

Now let’s test with the same question again, and we will get a “CALC:” response from ChatGPT:

This is a great improvement: ChatGPT admits that it needs help! Instead of guessing the answer, it humbly asks the system how to do this calculation.

Step 5 - Control the Output Format

Since ChatGPT generates each output word based on probability, it may decide to change the response format sometimes. This is not good, since our code assumes a fixed format.

For example, if you try to run the program a few times, you might get a response like this:

Similarly, ChatGPT may decide to say something after the expression.

Now, can you think about how to fix this issue? Ideally, we want ChatGPT to ALWAYS generate the response in a fixed format of “CALC: 3516 * 357”. Please give it a try before looking at the solution below.

There are many ways to control the response format of ChatGPT, but most of the time, the best solution is to give an example. ChatGPT is extremely well-trained in learning from examples.

For this specific issue, we can give an example conversation between the user, ChatGPT and our system like this:

You are a friendly assistant helping a k-12 student with some calculations. Your response is concise and simple to understand. Do not say anything inappropriate.

When you need to do a calculation, instead of responding to the user, you can respond with a special tag “CALC:”, followed by the expression to calculate. The system admin will provide the answer to that expression, then you can generate the response to the user.

For example:
user: what is 2345 times 3456?
assistant: CALC: 2345*3456
system: 8104320
assistant: the answer of 2345 times 3456 is 8104320

Now say “hi”

Now please copy this new prompt to your comment area, then copy that into the system request block.

Now if you try to ask the question again, it almost always use the correct format:

Step 6 - Check if the response starts with “CALC:”

Next, we will handle ChatGPT’s response differently depending on whether it contains the special tag of “CALC:”. We will only add some additional steps when ChatGPT’s response starts with “CALC:”.

Step 7 - Do the Calculation

When we find ChatGPT is asking us to do the calculation, we can calculate the answer and store it in a variable.

The logic is the following:

The response from ChatGPT starts with “CALC:”, and we only need the expression that comes after that. So, we can use the “substring” block to get the substring of the response starting from the 6th letter. For example, if the response variable is “CALC: 3516 * 357”, then the substring will be “3516 * 357”.
We can use the “calculate expression” block to calculate this expression, which will produce a number like 1255212
We can store that number in a new variable named “answer”.

Step 8 - Tell ChatGPT the Answer

Now we have done the calculation, we need to give it back to ChatGPT. Note that we should send the answer using the “system request” block, which makes it clear to ChatGPT that this answer is from the system, not the user. We will still ask ChatGPT to store the response in the “response” variable, which will overwrite the previous response.

After this step, the “response” variable contains the new response based on our calculation, and we can continue to append that to the chat history.

If you test with the same question again, you will get the correct answer!

Creative Ideas

In this tutorial, you have learned a very useful technique to enhance ChatGPT with new abilities. Can you try some other ways to help ChatGPT? Below are some ideas for your inspiration:

Today’s Date:

Believe it or not, if the user asks ChatGPT what’s today’s date, it would give a random and wrong date. You can help get the answer using the “current [year/month/date]” block.

Longest Common Substring:

ChatGPT is also bad at carrying out algorithms. For example, it gets this question wrong:

You can help out using the “longest common substring” block, and the correct answer should be “1357911”.