AI - A Voice Translator (Difficulty: 2)

[info-creaticode]

 

References

• AI for speech to text
• AI for text to speech
• Using the label widget
• Using variables

 

Introduction

 
In this tutorial, you’ll learn how to build an AI program that translates between any two languages!
For example, you can say something in English, and your project will show and speak the translation in Spanish.

Here’s what we’re aiming for:

 
 

Step 1 - Pick a Backdrop

 
Please create a new project, and remove the dog sprite. Select a backdrop for the stage, such as “Stars”.

 
 

Step 2 - Add the First Label

 
Find the “Widgets” category in the palette, then add a label widget. Its text should be “Say something in English”, and please try to make it cover the upper half of the stage:

 
As shown below, you can use the “Widget Position” tool to manually move and resize the label widget. You need to move the mouse slowly to keep dragging the handles. The position and size values in the block will be updated automatically.

 
 

Step 3 - Format the Label

 
Let’s make the label look better:

• Set the background to transparent.
• Make the text larger so it’s easier to read.
   
  
   
  Here are the blocks you need to add. Note that since we have already added the label, you can choose it from the dropdown list.

 
 

Step 4 - Recognize Speech

 
Next, let’s listen to the user on the microphone and recognize what the user is saying for 3 seconds:

 
 

Step 5 - Display the recognized text

 
Next, show the recognized text on the label.

• Use the “text from speech” block to grab the words.
• Then use “set value to” to update the label text.

Now when you speak in English, the words will appear on the stage:

 

 
 

Step 6 - Translate the text

 
Let’s translate the words into Spanish!

First, create two variables:

• “English words” (to store what you said)
• “Spanish words” (to store the translation)
   

 
Step 6 Alternative: Use AI for Translation
 
Instead of using the “translate” block, we can also use the Large Language Model (LLM) block to do the translation:
 

 
Notes:

• In the prompt (request), we ask the AI to only return the translated text and nothing else. Without this requirement, the AI may return some additional words like “Sure, here is the translation:”.
• The session type is “new chat”, so that previous messages would not affect the current request.

 
 

Step 7 - Add another label

 
Now, we need a second label to show the Spanish translation.

• Add a new label.
• Change its style (make it match the project design).
• Then update its text to show the Spanish words.

(You have to add the label first to see it in the dropdown menus for updating its style!)

 
And here are the new blocks you need:
 

 
 

Step 8 - Speak the Spanish Words

 
For the last step, we also need to speak out the Spanish words. You can use the “AI Speaker” for this.

This is the final result:

 
 

Additional Challenges

 
There are many ways you can extend this project. Here are some ideas for your inspiration:

1. Let the user pick two languages
   Add two dropdown menus so the user can pick any two languages to translate between.

2. Translate in both directions
   Add buttons so both people can speak and get translations back and forth — one button for English → Spanish, another for Spanish → English.