AI - Counting Fingers with Hand Detection (Difficulty 3)

[info-creaticode]

 

Key Topics Covered

 

• Using variables
• Reading from tables
• AI for Hand Detection

 
 

Introduction

 

In this project, you’ll build a fun and interactive finger-counting program using AI-powered hand detection. The program will detect how many fingers are stretched out and display that number on the screen.

 
Note: While it’s possible to use the visual capabilities of large language model AIs (see this tutorial), there’s a noticeable delay in receiving the recognition results. For real-time hand tracking, it’s essential to use the dedicated hand detection AI block from this tutorial.

 
 

Step 1 - A New Project

 

Please create a new project in the CreatiCode playground. Remove the sprites named “Sprite1”. We will only need to use the “Empty” sprite.

 
 

Step 2 - Add Number Costumes

 

We’ll show the detected finger count by switching between number costumes.

• Go to the Costumes tab.
• Add the costume “Glow-0” and delete the original costume.

 
Now add “Glow-1” through “Glow-5”, so you have 6 costumes total.

• Costume #1 = Number 0
• Costume #6 = Number 5

 
 

Step 3 - Start Hand Detection

 

Now let’s switch to the “Code” tab, and add the hand detection block from the “AI” category.

 
Click the green flag to activate the camera and begin detecting hands.

 

 
As shown, each detected hand is represented by 21 landmark points: 1 for the wrist, and 4 for each finger.

 
 

Step 4 - Review the Data Table

 

When the hand detection block is added, a new table (named “i”) is created automatically.

• Go to the Variables category.
• Check the box next to table “i” (the name of the output table variable) to show it on the stage.
   
  

 
 
Now the table updates live with hand tracking data.

• Each hand generates 47 rows of data.
• You’ll see a new set of 47 rows added for each additional hand detected.
   
  

 
 

Step 5 - Focus on the “Curl” Angle

 

To determine if a finger is stretched out, we just need to check its curl angle, found in the first 5 rows of the table.

• A stretched finger has a curl angle near 180 degrees.
• A bent finger has a curl angle closer to 0.

Try curling your index finger and watching the change in value:

 
🧠 Tip: For simplicity, if the curl angle is greater than 150, we will treat that finger as stretched.

 
 

Step 6 - Create a Counter Variable

 

To count how many fingers are stretched, create a new variable named “counter”.
This variable will store the total number of stretched fingers.

 
 

Step 7 - Use a Forever Loop

 

We want to continuously update the count in real time. Wrap the logic inside a forever loop, and reset the counter to 0 each time.

 
 

Step 8 - Check If a Hand Is Detected

 

We should only check the data in the table if the camera sees at least one hand.

• Each hand adds 47 rows to the table.
• Use the “row count” block to check if the table has 47 or more rows.
   
  

 
 

Step 9 - Loop Through Rows 1 to 5

 

Create a variable called “row”, and use a for loop to go from 1 to 5, which represent the five fingers.
 

 
 

Step 10 - Read the “curl” Value for Each Row

 

We can read the value in the “curl” column for any row like this:

 
To read all 5 rows one by one, we just need to use the “row” variable to specify the row number in the block:

 
 

Step 11 - Count the Stretched Fingers

 

If the curl value is greater than 150, increase the counter by 1.

 

 
After the loop runs through all 5 rows, “counter” will store the number of fingers that are stretched out.

 
 

Step 12 - Show the Count Using Costumes

 

We’re almost done! Now, just switch to the correct costume based on the counter value.

• Costume 1 = number 0
• Costume 2 = number 1
• Costume 3 = number 2
• … and so on.

So if “counter” is 2, we should show costume 3, which means the costume number is always counter + 1.
 

 
Here is what you should get:

 
 

Additional Challenges

 

This finger-counting program is very simple. There are a lot of places where you can make improvements. Here are some ideas:

• Recognize Higher Numbers: When you show the number “6”, currently the program still reports “2”, since 2 fingers are stretched out. Can you make the program smarter? How about other digits like 7, 8 and 9?

• Detect 2 Hands: You can try to recognize 2-digit numbers with 2 hands

• Recognize Other Gestures: Besides numbers, you can use the “curl” angle to help recognize other hand gestures, such as “OK”, “Peace”, “Thumb Up”.