AI - Chat About a PDF File (Difficulty: 3)

[info-creaticode]

Introduction

 

One of the most useful applications of AI (large language models like ChatGPT) is to analyze and process PDF files. For example, we might need help summarizing a file or understanding its content.

In this tutorial, you will build a relatively simple tool, which allows the user to load any PDF file from the computer, and chat with the AI (LLM model) about the content of that file.

 
 

Step 1 - Start a New Project

 

In the CreatiCode playground, create a new project named “Chat about a PDF file”. Delete the sprite with the dog, as we will only write code in the Empty1 sprite.

 
 

Step 2 - Add a Backdrop

 

To make our project look nice, we can generate a backdrop image using the AI tool. For example, if we use the prompt “a light-colored background with a book, 2D illustration”, we will get a simple image like this:

 
 

Step 3 - Add a Button for Loading the PDF File

 

The user will always start by loading a new PDF file, which will be attached to the user’s chat with the AI. We will provide a button for loading the file from the computer like this:

 
The button will look like this:

 
 

Step 4 - Attach the PDF File to Chat

 

When the user clicks the button, we will run the reporter block “attach files to chat”. It will open a new dialog window for the user to select a PDF file from the computer. The return value of this reporter block will be the file name, and we will store that information in a new variable called “pdf name”:

 
This is how it works:

 
As an example, we will be using the chapter 1 of a free book “US History” from OpenStax. Since that book is too big, we have extracted the first chapter of it into a standalone PDF file. You can download it for testing your program, or you can prepare any other PDF file. Make sure it is not too big, since it will take longer to process a big file.

 
 

Step 5 - Add a Chat Window with the First Message

 

Next, we will add a chat window to the stage. We will also let the user know we have received the file by adding a chat message first. Note that this message is fixed, and we have not started using the AI yet.

 
The result looks like this:

 
 

Step 6 - Add User Message to Chat Window

 

When the user inputs a message and then presses the ENTER key or clicks the SEND button, we will first append that message to the chat window as the user. Note that the “value of chat1” block contains the input from the user.

 
 

Step 7 - Send User Message to AI

 

Next, we will send the user’s message to the AI:

 
We will use the “LLM” block instead of the ChatGPT block, since only the LLM block can read the files attached to the chat. Some notes on the parameters:

• The “small” model is faster, so for most use cases it is good enough;
• The request is simply the user’s input;
• The AI’s response will be stored in the “result” variable;
• We will be waiting for the response to come back before running the next block below;
• The response will be cut off if it is longer than 1000 tokens;
• The AI will use a temperature of 1 for maximum creativity;
• The session type is “continue”: When the first time this block is used after the green flag button is clicked, it will be treated as “new chat”, and all past chat messages and file attachments will be removed; after the chat session has already started, we will continue the same chat, so the AI can “see” previous messages and files.

 
 

Step 8 - Add AI’s Response to Chat

 

Lastly, when we receive the response from AI, we will append it to the chat window as well:

 
Here is the final demo of the project.

 
Note that it might take some time for the PDF to be uploaded to the AI server before it can generate a response, so it may take longer than usual. You can clearly see that the response of the AI is based on the content of the file.

 
 

Additional Challenges

 

Here are some additional challenges for you:

• Display a message like “Please wait while I upload the PDF file …” in the chat window while waiting for the AI’s response, so the user would wait patiently.

• The “attach files to chat” block is actually very powerful. You can attach multiple files so long as their total size is less than 10MB. Besides PDF files, it can also accept Text, Image, CSV, MP3, and WAVE files. Can you try to build another tool to make use of this capability?