AR - Virtual Costume with Occlusion (Difficulty: 3)

CreatiCode

Key Topics Covered

Introduction

In an earlier tutorial, you learned how to attach a virtual costume to the user’s head.

However, there is a major issue with that project. If the user tilts up his/her head, then we would see the back edge of the hat, so it looks like the entire hat is in front of the face.

You will learn to fix that problem in this tutorial. The basic idea is to occlude (i.e. cover up) the back edge of the costume, as if it is blocked by the head.

ccimg||480||https://cdn.creaticode.com/scratch-gui-projects/forum/30b85131-023c-4127-8e1e-050ec0d06069

Step 1 - Copy the Virtual Costume Project

Please open your project from the “virtual costume” tutorial, and save a new copy of it.

Step 2 - Switch to Emulation Mode

Since we are back to coding, we should switch on the emulation mode and turn on the markers.

Step 3 - Add an Oval Shape for the Head

Now we are ready to do some “magic tricks”. The basic idea is to add a new 3D shape to represent the head, and use this shape to block the back edge of the hat. In addition, we will make this shape transparent, so it will not be visible in the final result. Although a transparent object can not block other objects in the real world, we can make it happen in our virtual world.

To get started, we need to add an oval shape that looks like the head. We can use a sphere shape with sizes of 110/100/120 in the X/Y/Z dimensions, and attach it to the nose bridge:

Note that we will name this oval shape the “head”, so we can select it later.

This is what you should get so far:

Step 4 - Move the Oval Shape to Fit the Hat

Next, we need to move the oval shape to fit inside the hat. You can click the “position” gadget on top of the stage window, then use the arrows to drag the oval shape.

When you are done, the new position will be given in the sprite pane below the stage, and you just need to write them into the “move to” block:

Step 5 - Use the Oval Shape for Transparent Occlusion

Now we are ready to do the magic to make the back edge of the hat disappear. The key block to use is “convert to transparent occlusion”. It will have 2 effects:

The oval head will become transparent, so we won’t see it anymore.
The part of the objects that are blocked by the oval shape will still be blocked (“occluded”), as if the oval shape is not transparent.

To make this happen, you need to do 2 things:

Add the “convert to transparent occlusion” block at the bottom to convert the oval shape for transparent occlusion.
Move the “add sphere” block right after we initialize the scene. This is important because the transparent oval shape would only block other objects that are added after it, including the hat and the markers.
Use a “select” block to select the oval shape, so that we can change it after we add the hat.

Now you will see that the oval shape is gone, but the hat and the markers are blocked by an empty shape when we rotate the face:

Step 6 - Test with the Real Camera

Now we are ready to test our program with real camera images. You just need to hide the markers and turn off the emulation mode:

Now we won’t see the back edge of the hat when the hat is tilted up:

Please adjust the size of the hat and the oval shape to make the hat fit better with your head in the camera.

Creative Ideas

This method of “transparent occlusion” can be used for other virtual objects as well. Here are some ideas for your consideration:

Glasses: you can try to add sunglasses to the head, and make sure the backend of the legs are occluded.
Flying Airplane: you can add an airplane that flies around the head, but make sure it is blocked by the head when it is behind the head.