ChatGPT AI: Enhance ChatGPT to Do Math Calculations (Difficulty: 3)

CreatiCode

Introduction

Large language models like ChatGPT are really good at understanding and generating text, making them seem almost like they’re super smart. However, if you use them more, you’ll notice they’re not so great at certain tasks.

One big task they struggle with is math calculations. For example, if we test ChatGPT with a multiplication question:

With any calculator, we can easily find out 3516 times 357 equals 1255212. But ChatGPT gets this wrong because it doesn’t actually know how to do math calculations. Instead, it guesses based on similar problems it saw during its training.

Math is super important, so we need to help ChatGPT get the answers right. In this tutorial, you’ll learn a simple trick: we’ll teach ChatGPT to ask for help with math, and our code will give it the correct answers.

Once you understand how this works, you can apply the same idea to teach ChatGPT how to use other tools to solve problems that it can’t solve by itself.

Step 1 - The Starting Project

We’ll start from this project. Remix it into your own project and name it “ChatGPT with Calculation Help”:

play.creaticode.com/projects/6531b7e60fdce080a4481c1d

Step 2 - Verify the Issue

Before we fix anything, test the math question in the project to see if ChatGPT gets the answer wrong:

Step 3 - Sequence Diagram

Before we make any changes, let’s discuss the design of this project using sequence diagrams.

There are 3 entities in our project:

User: the user who asks questions
ChatGPT: the AI that answers questions
System: our program code

Right now, the system only passes messages between the user and ChatGPT:

To help ChatGPT do math calculations, we can change the diagram so that ChatGPT can choose to talk to our system to get math help, as shown by the 2 additional blue arrows below:

The beauty of this design is that the new messages between the system and ChatGPT can be hidden from the user, so it appears to the user that ChatGPT is answering correctly.

Step 4 - Update the Initial Prompt

Copy the following prompt into the system request block:

You are a friendly assistant.

When you need to do a math calculation, instead of responding to the user, you can respond to the system using a special tag “CALC:”, followed by the expression to calculate. The system will provide the answer to you, and then you can generate the response to the user.

Now say “hi”

This prompt contains 3 parts:

It first introduces the context, where ChatGPT plays the role of a friendly assistant.
Then, instructions are given to ChatGPT: It should use a special syntax when it needs math help.
Lastly, we ask ChatGPT to say “hi” and wait for user input. This makes sure the block finishes quickly.

Now let’s test with the same question again, and we will get a “CALC:” response from ChatGPT:

This is a big improvement: ChatGPT admits that it needs help! Instead of guessing the answer, it humbly asks for help with this calculation.

Of course, we won’t show this part to the end user. We will hide it in the next few steps, and only show a final answer in the chat.

Step 5 - Control the Output Format

Since ChatGPT generates each output word based on probability, it may decide to change the response format sometimes. This is not good, since our code assumes a fixed format that starts with ‘CALC:’.

For example, if you try to run the program a few times, you might get a response like this:

Similarly, ChatGPT may decide to say something after the expression.

Can you think about how to fix this issue? Ideally, we want ChatGPT to ALWAYS generate the response in a fixed format of “CALC: 3516 * 357”. With no extra sentences before or after. Please give it a try before looking at the solution below.

There are many ways to control the response format of ChatGPT, but most of the time, the best solution is to give an example, as explained in the TIRE prompting method. ChatGPT is extremely well-trained in learning from examples.

For this specific issue, we can give an example conversation between the user, ChatGPT and our system like this:

You are a friendly assistant helping a k-12 student with some calculations. Your response is concise and simple to understand. Do not say anything inappropriate.

When you need to do a calculation, instead of responding to the user, you can respond with a special tag “CALC:”, followed by the expression to calculate. The system admin will provide the answer to that expression, then you can generate the response to the user.

For example:
user: what is 2345 times 3456?
assistant: CALC: 2345*3456
system: 8104320
assistant: the answer of 2345 times 3456 is 8104320

Now say “hi”

Now please update your system request block with this new example dialogue.

If you try to ask the question again, it almost always uses the correct format:

Step 6 - Check if the response starts with “CALC:”

Next, we will handle ChatGPT’s response differently depending on whether it contains the special tag “CALC:”. We will only add some additional steps when ChatGPT’s response starts with “CALC:”.

Step 7 - Perform the Calculation

When ChatGPT asks us to help with a calculation, we can calculate the answer and store it in a variable.

The logic is the following:

The response from ChatGPT starts with “CALC:”, and we only need the expression that comes after that. So, we can use the “substring” block to get the substring of the response starting from the 6th letter. For example, if the response variable is “CALC: 3516 * 357”, then the substring will be “3516 * 357”.
We use the “calculate expression” block to calculate this expression, which will produce a number like 1255212
We can store that number in a new variable named “answer”.

Step 8 - Tell ChatGPT the Answer

After our code completes the calculation, we must return it to ChatGPT. Note that we should send the answer using the “system request” block, which makes it clear to ChatGPT that this answer is from the system, not the user.

With this new answer, ChatGPT can generate a new answer for the user’s original question.

After this step, the “response” variable contains the new response based on our calculation, and we can continue to append that to the chat history.

If you test with the same question again, you will get the correct answer!

Additional Challenges

In this tutorial, you have learned a very useful technique to enhance ChatGPT with new abilities. Can you try some other ways to help ChatGPT? Below are some ideas for your inspiration:

Today’s Date:

Believe it or not, if the user asks ChatGPT what’s today’s date, it would give a random date. You can help get the answer using the “current [year/month/date]” block.

Solving Equation:

ChatGPT can not solve math equations by itself, since it can only generate one word at a time, and it does not really understand the logic for solving an equation. For example, if we ask “x * x * x * x = 12345”, it will not be able to give a precise answer.

Therefore, we can teach it to solve it with the “solve equation” block in the “Operators” category: when the user asks a question that would require solving an equation, ChatGPT should write a math equation, and ask our system to help solve it. Based on the answer our system provides, ChatGPT can then answer the user’s question.

Longest Common Substring:

ChatGPT is also bad at carrying out algorithms. For example, it gets this question wrong:

You can help out using the “longest common substring” block, and the correct answer should be “1357911”.