• AI for speech to text
• AI for text to speech
• AI for analyzing a sentence
• Using the label widget
• Using variables

Introduction

In this tutorial, we’ll build a chatbot that can take orders from customers (users), similar to the services provided at McDonald’s Drive-Thru.

Step 1 - Remix the Project Template

A template has been prepared for you, which includes all the images you need. Please log into the CreatiCode playground, and remix this project. Feel free to change the backdrop or sprite images.

play.creaticode.com/projects/b748ef8a60fd9ff0f86dd3a6

Step 2 - Greet the Customer

When the project starts, our chatbot should greet the customer with a question like “Hi, what would you like to order?”. Please select the “AI” sprite, and add the green flag block and the “say” block from the “AI” category (premium-subscription required):

Step 3 - Make Key Blocks

Since this will be a relatively large project, it is a good idea to define a few key blocks to organize our code blocks.

For a simple chatbot, we need to get the customer’s input, then analyze the intention of the customer, and then give a response. Therefore, let’s define 3 blocks in the “AI” sprite like this:

Step 4 - Get Customer Input

The customer can talk to our chatbot directly. We can use the "speech recognition“ blocks to listen to the customer.

Note that when we start the speech recognition process, the “text from speech” block will be reset to empty. After the customer has completed a sentence, “text from speech” will be populated with that sentence. So we can wait until “text from speech” is no longer empty, then end the recognition.

Also, let’s use a variable named “input” to store the recognized text. We can convert the text to lower case only, so we do not need to worry about handling both upper and lower case letters.

Now if you run the project, you will hear the question from the chatbot, then a red mic will pop up to get your input. You can say something like “What’s on the menu?”, and you should see that sentence stored in the “input” variable. Note that you should only start talking after the mic icon shows up.

Step 5 - Analyze Customer Intention

Although we can recognize the customer’s speech, we are still far away from understanding the customer’s intention. There are unlimited possibilities for what the customer might say, so there is no way for us to write a program that handles every possibility.

Fortunately, we do not have to. Since we are creating a chatbot for taking orders, we can focus on only a few relevant intentions, such as “checking the menu”, “placing an order”, etc. Then we just ignore all other possible intentions.

To get started, suppose the customer says “what’s on the menu?” or “what’s available?”, then we are pretty sure his/her intention is to check what’s available on the menu. Therefore, we can map these 2 possible inputs to the same intention of “checking the menu”.

We will use a new variable “intention” to store this information. By default, the intention is “unknown”, but we can improve our guess based on the input.

Now if we say “what’s on the menu” or “what’s available”, we will get the same intention of “checking the menu”:

Step 6 - Use Keywords

Currently, when we try to guess the customer’s intention, we are looking for entire sentences like “what’s on the menu”. This is too narrow. For example, if the customer says “Can you give me the menu?”, then our chatbot would not treat this as the intention of “checking the menu”.

To improve this, we should check for keywords instead. For example, if the customer input contains “menu” or “available”, then it’s good enough to set the intention to “checking the menu”.

Now if we say “tell me the menu please”, the chatbot would still be able to get the right intention.

Can you think of other keywords for the “checking the menu” intention?

Step 7 - Respond Based on the Intention

To respond to the customer, we need to check the customer’s intention.

If the intention is “checking the menu”, we simply need to offer the list of items available. If the intention is “unknown”, we have to request the customer to say it again.

We will store our response in a new variable as well, and use the “say” block again to speak it out.

Now our chatbot will be able to respond to us verbally:

Step 8 - Testing vs Production

Before we improve our chatbot to handle more customer inquiries, we need to make our development process more efficient.

Currently, our chatbot says the greeting words and then listens to the customer’s speech. That slows down our development, as we need to test our chatbot a lot.

To solve this problem, we can change our program to run 2 modes: production mode and testing mode. In the production mode, the program will work the same way as now. However, in the testing mode, we will simply set the customer input directly and bypass the “say” and “listen” parts. We also do not need to speak out the response in testing mode.

Here is what we need to do to add the testing mode:

• Add a new variable “is testing”, and set it to 1 when we are working in the testing mode. So we would only “say” the greeting words if we are not testing.

• When we are in testing, set the “input” variable directly instead of getting it via voice

• At the end, we only need to “say” the response if we are not testing

 
Now if you run to run the program in testing mode, it generates the response immediately:

Step 9 - A New Intention “Placing an Order”

Now we are ready to add a new intention type: “placing an order”. For example, the customer might say “I’ll have 2 burgers”, “May I have a hamburger, please?”, or “Can I get a chicken nugget?”.

To map all these inputs to the “placing an order” intention, we can check for the keywords “I’ll have”, “may I have” and “can I get”.

Step 9 - Get the Product Type

When we know the customer’s intention is “placing an order”, we still need to find out exactly which product they are buying. There are 4 product types: burgers, french fries, chicken nuggets and coke.

Please make a new block for “get product”, and use a new variable “product” to store the product type. By default the product type is “unknown”, and we can search for some keywords in the customer input to guess the product type.

Step 10 - Get the Product Count

After getting the product type, we also need to know how many of it the customer needs. This step is more complicated than it appears because there are many ways for the customer to specify the count, such as “two burgers”, “a coke” or “eight large french fries”.

To solve the problem, the key is to find the number word in the customer input. To do that, we can use the “analyze sentence” block, which will tell us the type of every word in the sentence.

Let’s first add a new block “get count”, and run the “analyze sentence” block. Note that we need to create a new table named “result” to store the output of the analysis:

 
Next, set the input variable to “I’ll have two burgers”, and turn on the monitor for the “result” table:

As shown, each word takes one row in the result table, and the word “two” is assigned a type of “Number”. This is very helpful, because we just need to look for the “Number” type to find out the number of “two”. To do that, we can use the table lookup block. Also, if the count is an empty string " ", we will assume the count is 1.

Since the product count we get maybe a word like “two” or “eight”, we need to convert it to an Arabic number:

Now when we run the program, we should see the “count” variable is set to the correct Arabic number:

Step 11 - Labels for Product Counts

We should display the count for each product, so it’s easy for the customer to review the order.

To do that, we need to add a label under each product image when the green flag is clicked. For example, in the burger sprite, we can add the following blocks:

You can copy these blocks to the other 3 products’ sprites as well. Note that the names of the label widgets are all formatted as the product name followed by “label”.

As a result, we should see 4 labels like this:

Step 12 - Respond to “Placing an Order”

Now we are finally ready to respond to the new intention “placing an order”. This should be a new “if” block inside the “respond” block’s definition:

First, we need to change the text of the label for the product being ordered. For example, if the “product” variable is “burger”, we need to set the “burgerlabel” to the value of the “count” variable. We can use the “join” block to get the label name, so we don’t have to write a separate logic for each product:

After that, we need to set the “response” variable to what we will say to the customer. For example, we can say “Sure, 1 burger”. Here we need to compose the response using the “count” and “product” variables:

There is still a small issue. When the count is more than 1 and the product is “burger”, we need to add an “s” at the end of the product name.

Here is the final program:

Now our response captures both the product name and the count:

Step 13 - Publish the Program

Now we have a very simple chatbot that can offer the menu and take new orders. To publish it to users, we need to change the “is testing” variable to 0 (any value that’s not 1).

Also, to repeatedly take customer inputs, we need to add a “forever” loop around our key blocks:

 
Now our chatbot can have a simple conversation with the customer like this:

Chatbot: Hi, what would you like to order?
Customer: What’s on the menu?
Chatbot: You can order burgers, french fries, chicken nuggets or coke
Customer: I’ll have two burgers.
Chatbot: Sure, 2 burgers.
Customer: Can I have a French fries as well?
Chatbot: Sure, 1 fries.

Here is a demo video. Note that you need to wait for the mic icon to appear to start talking.

Creative Ideas

Chatbots can be very complex if you want them to talk to customers like human beings. Here are some new ideas for you to explore:

1. Handle more customer inputs: there are many other ways for the customer to place an order or check the menu. You can make a list of all possible ways, and try to handle them in the “get customer intention” logic.
2. Other Intentions: such as “Changing product count”, “Removing a product”, “Asking for total cost”, etc.
3. Product Size: for example, for french fries and coke, we might offer “large”, “medium” and “small” sizes
4. Other Scenes: you can build a different chatbot that serves other purposes, such as a chatbot at restaurants that handle customer reservations, a FAQ chatbot that answers all kinds of questions on a specific topic, a fun chatbot that cracks jokes or plays Youtube videos.