The Video Sensing Extension

CreatiCode

Prerequisite

Working with Blocks

Introduction

The Video Sensing Extension allows your program to detect object movement in the camera video.

How to detect motion in the camera video?

You can think of the camera video as a sequence of images (called “frames”), just like an animation. For example, suppose we move an apple in front of the camera, and we get these images in slow motion:

Therefore, we can compare the difference between any 2 images. For example, between these 2 images, we can see the apple clearly moved towards the bottom right a little bit.

An image is made of many tiny boxes called “pixels”. For example, the apple in the camera image is made of many small pixels in dark red or brown colors.

By comparing the pixels in these 2 images, we can count how many pixels have moved.

How to add the Video Sensing Extension?

To add the video extension to a project, you click the “Add Extension” button below the blocks categories and then select “Video Sensing” from the extension library. Make sure you give permission to our playground to use your camera.

Camera is Forbidden?

If you do not see the camera video, then you should check if the camera is forbidden by looking at this camera icon above the green flag button:

After you add the video extension, a live video starts displaying on the stage which shows what is in front of your webcam.

Fix Camera Permission Issue

If the camera is somehow forbidden, you can fix it by changing the permission settings and then reloading the page:

The Video Sensing Blocks

There are two stack blocks, one hat block and one reporter block in the video sensing extension.

“Turn video (On/Off/On Flipped)”

This block is used to start or stop the webcam video on stage. There are three options that you can select from the drop-down menu:

On - It turns on the webcam video.
Off - It turns off the webcam video.
On flipped - It turns on the webcam video and also flips it horizontally. That is, things on the left will show up on the right side of the stage and vice versa.

Set Video Transparency

The camera video is below all the sprites but above the stage’s backdrop. You can use this block to change the video’s transparency.

When transparency is 100, you see the backdrop clearly and no video
When transparency is 0, the video will completely block the backdrop
When transparency is between 1 and 99, you see the backdrop through the video.

“Video (motion/direction) on (sprite/stage)”

This reporter block tells the value of “motion” or “direction” sensed in the camera video."

The first input allows you to select if you want to know the value of motion or direction.

Motion: this block will report a number that represents how many pixels (tiny boxes that compose the image) in the camera video have moved between the last frame and the current frame. Note that this is not about the distance or speed of the movement, but the size of the area where some motion is detected.
Direction: this block will tell us the average direction of the detected movement. For example, when most of the pixels are moving to the right, the direction would be around 90 degrees.

The second input allows you to select the current sprite or the entire stage.

Sprite: the computer will only look for movement in the region under the current sprite. Movement outside this sprite will be ignored.

Stage: the computer will look for object movement anywhere in the camera video.

When motion detected in camera video

This block will run the blocks attached below it when it senses enough motion in the camera video.

It takes one input, which is a “threshold number”. If the “motion” detected under this sprite’s area is larger than this threshold number, then the blocks attached below this block will run. Note that this block will ignore motion in other areas of the stage.

Usually, you can set this to 10 to detect any meaningful movement. You should not use 0 as the threshold, since there is always some small movement detected (called “noise”).

In this example below, we can make the apple sprite move up whenever your hand moves under it, and the apple will not move when your hand is not touching it.