The Video Sensing Extension
-
Introduction
The Video Sensing Extension lets your project detect motion using your device’s camera. This means your program can respond when something moves in front of the camera!
How to detect motion in the camera video?
Think of a video as a bunch of pictures shown very quickly — like a flipbook. Each picture is called a frame. When something moves in front of the camera, it looks different in each frame.
For example, when you move an apple slowly across the camera, the frames will show it shifting positions:
We can compare two frames to see which parts of the image have changed. In the two frames below, the apple has moved slightly to the bottom right:
Each image is made up of tiny squares called pixels. The apple is just a group of pixels with red and brown colors.
By checking how many pixels have changed between two frames, we can measure how much motion has occurred.
How to add the Video Sensing Extension?
To add the video extension to a project, you click the “Add Extension” button below the blocks categories and then select “Video Sensing” from the extension library. Make sure you give permission to our playground to use your camera.
Can’t See the Camera Video?If you do not see the camera video, then you should check if the camera is blocked. Try to find this camera icon in the top navigation bar on the right:
Once the extension is added and the camera is working, a live video feed should appear on the stage.
Fix Camera Permission Issue
If the camera is somehow blocked, you can fix it by changing the permission settings and then reloading the page:
The Video Sensing Blocks
The video sensing extension includes:
- 1 hat block (which triggers scripts),
- 1 reporter block (which gives a value).
- 2 stack blocks (blocks you can attach to other blocks),
Let’s look at each of them below.
“Turn video (On/Off/On Flipped)”
This block lets you control whether the camera video appears on the stage:
- On: shows the live video.
- Off: hides the video.
- On flipped: shows the video, but flips it horizontally (like a mirror).
Set Video Transparency
When the camera video is shown, it appears behind your sprites but in front of the backdrop. Use this block to change how see-through the video is:
- 100 = fully transparent (only the backdrop shows),
- 0 = fully solid (the video hides the backdrop),
- 1–99 = a mix—you can see the backdrop through the video.
“Video (motion/direction) on (sprite/stage)”
This reporter block gives back a number based on what the camera sees:
The first input allows you to select if you want to know the value of motion or direction.
-
Motion: this block will report a number that represents how many pixels (tiny boxes that compose the image) in the camera video have moved between the last frame and the current frame. Note that this is not about the distance or speed of the movement, but the size of the area where some motion is detected.
-
Direction: this block will tell us the average direction of the detected movement. For example, when most of the pixels are moving to the right, the direction would be around 90 degrees.
The second input allows you to select the current sprite or the entire stage.- Sprite: checks for motion only under that sprite. Motion outside this sprite will be ignored.
This block runs only if enough motion is detected.
It takes one input: a threshold number. If the amount of motion under the sprite is greater than this number, the block runs.
- Use something like 10 as the threshold.
- Don’t use 0, because the camera always sees a little bit of “noise” or tiny changes.
Example Program
In this example, the apple only moves up when your hand moves under it. If your hand isn’t near it, nothing happens.
