Teacher Tool - An AI Chatbot for Checking Student Understanding

[info-creaticode]

Introduction

 

Traditional assessment tools, such as multiple-choice quizzes, often fail to accurately measure a student’s understanding and are highly susceptible to cheating.

As Large Language Models become more intelligent, we can use AI tools to check student understanding in new ways. In this article, we will introduce an example tool, which is a chatbot that can discuss a given program with a student and assess the student’s understanding. All chat messages will be stored in a Google Sheet under each student’s user name, so that the teacher can review them at any time.

This tool project is shared at

play.creaticode.com/projects/67d957d70b962dba34213579

It looks like this when a student runs it:

 

The chat messages will be stored in the Google Sheet like this:

 
 

To use this tool with your students, you need to do the following:

• Remix the project above
• Change the Google Sheet URL
• Change the system instruction.
• Publish the project and send its URL to your students.

Next, I will explain how this tool works and how you can make these changes.

 
 

Overview

 

As shown below, this program only has 3 key modules:

• Initialize Sheet: adds a new tab in the Google Sheet you provide so that the chat messages with this user can be stored there.
• Initialize Chat: set the instruction for AI and then get the first message from AI
• Handle User Message: send new user message to AI and get AI’s response

 
 

Initialize Sheet

 

In this module, a few steps are taken to set up the Google Sheet. You will only need to change the Google Sheet link in the first part:

 

1. Set Sheet URL (TODO)

• All chat messages with all users will be stored in one Google Sheet, and you will need to provide the link to it. The link should be set up so that “anyone with link” can edit it.
• To make sure no one else can see the content of this sheet, you should not share this project with students. Instead, you should publish this project and give its URL to your students, so they can run this project without looking inside its code.

 

2. Set Tab Name

• Each student’s chat messages will be stored in one new tab. To easily identify the student, we will use their “full name” from their profile. To handle duplicate full names, the first 4 letters from their random user ID are appended.
• Note that students must log in to CreatiCode before running this project so their full names in the profile can be used.

 

3. Add a New Tab

• We load all tab names into a list, so we can check if the tab for this student already exists.
• If it does not exist, we will add a new sheet with that name
• We will remove most rows and columns first. This makes sure there are no empty cells. In Google Sheet, there is a limit that the total number of cells in a sheet should be less than 10 million.
• We will add the table headers (Time/Who/Message) to the first row of the new tab

 

4. Add a row of dashed lines

• Sometimes, a student may run this program multiple times. To mark the beginning of a new run, we will add one row with dashed lines.

For example, the new tab will look like this after initialization:

 
 

Initialize Chat

 

In this module, you will need to modify the system instruction based on your requirements.

 

1. Set System Instruction (TODO)

 

 
The system instruction is given to the AI model before the chat begins. It contains all the instructions and requirements. You need to modify it to specify the code that you want the student to discuss and any key points you want to check. Here is an example system instruction:

 

• The pseudocode is enclosed in the “```scratch” tags. You can get this pseudocode for any scratch program by right-clicking on any stack of blocks, and select “copy pseudocode of this script”:

• You can also specify other coding languages after the 3 ticks, such as python, javascript, java, etc.

• Feel free to update the “Key points to check”, so that the AI will try to assess how well the student understand these points about the given program.

• Feel free to update the other instructions below. For example, the example instruction asks AI to conclude the chat with a rating. You can choose to do it differently or not give any rating.

 

2. Display the chat window

 

 

3. Get AI’s initial message

 

 
To make the AI present the given program to the student and ask its initial question, we simply need to send a message like “hi” to the AI. The response from AI will be added to the chat window.

 

4. Store AI’s message

 

Laslty, this message from AI will be stored in the Google Sheet, using the “store message” block. This block will get the current time, then append one row at the bottom of the tab for this student:

 
 

Respond to student message

 

Whenever the student inputs a new message, we will get AI’s response to it. You don’t need to change anything here.

 

1. Display and store student message

• When the student clicks the “send” button, we will append that message (“value of widget [chat1]”) to the chat history.
• We will use the “store message” block to store this message in the Google Sheet as well.

 

2. Get AI’s response

• We will send the user message to the AI model and wait for its response.
• Then we add the response to the chat window
• Lastly, we also store the AI’s response in the Google Sheet.

 
 

Summary

 

In summary, you can reuse the example project by replacing the Google Sheet URL and the system instruction. You can also modify the system instruction to assess the student’s understanding of some concept instead of actual code. You can always review student’s conversation with the AI in the Google Sheet.