AR - Breathing Fire Effect (Difficulty: 3)

CreatiCode

Key Topics Covered

Introduction

In this tutorial, you will attach a particle emitter to the mouth detected in the camera, and generate a ray of fire when the mouth opens.

Step 1 - Load an “Empty” Scene

Please create a new project on the CreatiCode playground, remove the dog sprite, and add these blocks in the “Empty1” sprite.

Load the “Empty” scene using the “initialize 3D scene” block in the “3D Scene” category.

Step 2 - Turn on the AR Face Camera

Now please turn on the AR face camera. Let’s start with the emulation mode, and also show the markers on the face.

You should get markers like these:

Step 3 - Add a Prebuilt Fire Emitter

To generate a ray of fire, we need to use a prebuilt particle emitter.

Now you should get a fire that shoots forward:

Here are some explanations of the 2 new blocks:

This emitter generates 1000 particles (small png images) of size 25 by 25.
All particles are generated from a small box region of size 1 by 1 by 1.
Each particle travels forward at a speed of 1200 units per second, and it will die within 0.5 seconds.
Each particle will start in a light yellow color and gradually change to orange before it dies.

From this point on, everything we do will be targeted at the fire emitter.

Step 4 - Attach the Emitter to the “Mouth” Marker

To make the fire emitter move with the mouth, we can attach it to the “mouth” marker of the face.

Now the fire is generated from the mouth marker:

Step 5 - Rotate the Fire Emitter

Now we need to rotate the fire emitter so that it shoots the fire out of the mouth. We can rotate the hat 180 degrees to turn it around:

Now the fire looks pretty good:

Step 6 - Enable the Data Table

To detect when the mouth is open, we need to know the position of the upper lip and lower lip, so we can calculate their distance.

To get the lip position data, let’s enable the data table. Note that we can still use the emulation mode to examine the data table, though the numbers will not get updated. You can make a new table, then select it in the dropdown:

Now if you enable the monitor of that table (check its checkbox in the flyout), then you will see what the data looks like. You can find the Z positions of the “upperLip” and “lowerLip” in row 24 and 11:

Step 7 - Calculate the Lip Distance Repeatedly

Now we can read out the z positions of the 2 lips, using the column name “z” and row number 24 or 11. Then we can use a new variable “distance” to store the difference between these 2 numbers.

We need to do this calculation repeatedly, so that if this distance changes at any time, we will know it right away:

Step 8 - Start or Stop the Fire Based on the Distance

Next, we will check if the distance is more than a threshold number to determine whether it is open or closed, so that we can start or stop the fire accordingly:

The example data in the emulation mode comes from a closed mouth, and the lip distance is about 4 (-13.5 vs -17.4). So we can pick a threshold number larger than this value, such as 10. We can refine this number later when we get real data from the live camera.

Note that we have moved the “start emitter” block into the “if-else” block inside the forever loop, since we do not want to start the fire right after adding it (if the mouth is closed).

Step 9 - Test with the Real Camera

Now we are ready to test our program with real camera images. You just need to hide the markers and turn off the emulation mode:

Now you should be able to breathe some fire out if you open your mouth widely.

Note that if the mouth is not open very wide, the lip distance will still be less than 10, so the fire will not start. You can adjust this threshold yourself.

Step 10 - Move and Turn the Fire Emitter

There seem to be 2 problems with the fire:

First, its position is too forward and too low, so the fire does not look like coming from inside the mouth. We need to move it back and higher.
Second, the fire is shooting upwards, so we need to rotate it a bit around the X-axis.

Now the fire looks more like coming from inside the mouth:

Creative Ideas

You can use this method to attach other virtual objects to the user’s face. Here are some ideas for your consideration:

Improve Distance Calculation: currently we are assuming the distance between the 2 lips is the difference in their Z positions. This will not work if the head is tilted to the side. You can improve this by calculating the distance using the lips’ X and Z positions.
Switching Fire Color: you can try to change the fire to a different color when the head is facing left vs right. You just need to use 2 fire emitters, and dynamically decide which one to start based on the head’s facing direction.
Smoke out of the ears: you can add some smoke effects whenever the head is tilting to the left or right.