AR - Virtual Costume (Difficulty: 2)

CreatiCode

Key Topics Covered

Introduction

In this tutorial, you will attach a virtual hat to the face detected in the camera.

Step 1 - Load an “Empty” Scene

Please create a new project on the CreatiCode playground, remove the dog sprite named “Sprite1”, and add these blocks in the “Empty1” sprite.

Load the “Empty” scene using the “initialize 3D scene” block in the “3D Scene” category.

Step 2 - Turn on the AR Face Camera

Now please turn on the AR face camera. Let’s start with the emulation mode, and also show the markers on the face.

You should get markers for a face like these:

Step 3 - Add a Hat Object

Next, let’s add a hat using the “add model” block. Since the head’s height will be about 100, let’s set the target height of the hat to be 80:

When you run the program, you will find the hat far away from the face markers. This is expected since the hat will be added at the origin of the scene, but the face markers are added at a Y position of about 300.

Step 4 - Attach the Hat to a Marker

To make the hat move with the head, we can attach it to one of the markers of the face. For example, we can use the “Nose Bridget” marker. The face object’s name is always “ARHead” when we are using the AR face camera. Please make sure you select the correct sprite from the dropdown, and that the target height is still about 80.

Now the hat is moved to the top of the nose bridge marker. To help you identify which marker is the nose bridge, it is painted green in this screenshot below:

Step 5 - Rotate the Hat

You might have noticed that the hat is facing the wrong direction. Its pointy side should be visible. You can rotate the hat 180 degrees to fix that:

This is a comparison of before and after rotating the hat:

Step 6 - Test with the Real Camera

Now we are ready to test our program with real camera images. You just need to hide the markers and turn off the emulation mode:

Now the hat should follow the real face, but its position will seem a bit off:

Step 7 - Move and Turn the Hat

There seem to be 3 problems with the hat.

First, its position is too forward and too low. We need to move it back and higher.
Second, it is tilting up, so we need to rotate it down around the X-axis.
Third, it is a bit small, so we need to increase the target height.

The exact values might be different for you.

Now the hat looks much more real:

Creative Ideas

You can use this method to attach other virtual objects to the user’s face. Here are some ideas for your consideration:

Costume Switching: you can add a button widget, so when the user clicks on it, you remove the current costume, and add a new one. Note that you can search for **“hat”, “headband” or “crown” in the “add model” library, not just “hat”.
Funny Eyeballs: you can use a soccer ball or basketball to cover up the 2 eyes of the user.