AI for Body Part Recognition

CreatiCode

Introduction

You can use AI to recognize different parts of a person in front of the camera, such as where are the eyes, nose or hands of the person. This information can be useful in many applications, such as estimating the pose (sitting or standing) of this person.

Introduction

To start running body part recognition, you can run this block:

The image from the front camera of the computer will be displayed on the stage, and the AI engine will start to look for one or multiple persons in the camera view.

Let’s look at the 3 parameters below:

Single Person

If this option is “Yes”, then the AI engine will only report one person’s information, even if there is more than one person in the camera image; if “No”, the AI engine will report all of them. Most of the time, your application is intended to be used by only one user in front of the camera, and you can just select “Yes” here.

Result Table

The recognition result will be stored in the table you have selected here. You can create a new table, or use the default table created for you. The table will contain 6 columns. For each person detected, there will be 21 rows of data added to the table.

The first 17 rows will contain data in the first 4 columns:

The ID is unique for each person detected. You can ignore it if your program only works with a single person.
The part refers to the name of the key body part. Here is the full list: nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle.
The x and y values are the x and y positions of that part in the camera image. For example, if the nose is at the center of the image, then X and Y positions will both be 0.

For each person that’s detected, the 4 rows below the 17 rows will use the first 2 and last 2 columns:

The ID is still the unique ID for each person detected.
The part refers to one of the 4 limbs: “left_arm”, “right_arm”, “left_leg”, “right_leg”.
The curl column (the 5th column) tells us the curling angle of that limb. If that limb is extended in a straight line, then its curling angle is close to 180 degrees. If that limb is curled at the elbow or knee, then this angle will be much smaller.
The dir column (the 6th column) tells us the direction of that limb. For arms, this is the direction of the line from the shoulder to the elbow (the wrist is not considered here). For legs, this is the direction from the hip to the knee (the ankle is not considered).

For example, in this picture of superman, the curling angles are about 90 degrees for the 2 arms and about 180 degrees for the 2 legs. Their directions are all pointing downward.

Debug

If “yes”, then some dots and lines will be drawn over the camera image to mark the recognized body parts; if “no”, then nothing will be drawn over the camera image.

Example - Two People from a Picture

If you set single-person to “no”, multiple people can be detected.

If you move the picture or the camera, the 2 people will be recognized in real-time. The result table will include information for 2 IDs (2 people), each with 21 rows of data.