AR Face Camera
-
Prerequisite
Description
The AR face camera is a fixed camera. It shows the images from the front camera of the device, and recognizes the human face in it. Therefore, this block works on all devices, including computers, laptops, Chromebooks, touchpads and smartphones.
When the AR face camera finds the face in the camera view, it tries to estimate the 3D position of key parts of the face (called “markers”). such as the nose and the mouth. In your program, you can attach other 3D objects to any of these markers.
To turn on the AR face camera, you can use this block:
Parameters
- Show Marker: whether you want to display some white boxes to indicate where are the markers. When you are developing or debugging your program, it is easier if the markers are shown. You should hide them when the project is released to the users.
-
Scale: this value controls the scale of virtual 3D objects vs real objects. For example, when it is 100, a virtual box of size 100 will look like a box of 1 foot in the real world. For another way to think about it: to use a box to cover the entire head in the camera image, then the box’s edge length should be around the value of the scale.
-
Emulation Mode: this value controls if you are running an emulation. If "yes’, then the real camera will not be used. Some markers will be added to the virtual scene for your to test your program with. If “no”, then the real camera will be turned on, and the markers will represent the real face detected in the camera view.
- Data Table: if this option is not set to “none”, then the 3D position of all the markers will be written into this table in real-time.
- Mesh for Face/Eyes/Mouth/Lips: for these 4 options, if you set any of them to “yes”, then that part of the face in the camera will be covered by a 3D mesh surface. Note that “mouth” refers to the mouth opening, not the lips. For example, here is what you get when all options are set to “yes”:
Updating Face Mesh Color or Texture
You can use the “update color” or “update texture” block to change the look of the 4 meshes. The name of the object is always “ARHead”, and the 4 areas are mapped this way: main -> face, second -> eyes, third -> mouth, forth -> lips
For example, this is how to update the 4 areas’ colors:
Attaching objects to markers
After turning on the AR face camera, any 3D object you add to the scene will be displayed over the images from the real camera. However, if you want the 3D object to move or turn with the face in the camera, you need to attach that object to one of the markers using this block:
Parameters
- Part Name: this is the name of the marker you want to attach to. Since this block is also used for avatar bones, not all the options in the list can be used for the AR face camera. You can only use the items listed below “chin”.
-
Face Object Name: you should always write “ARHead” for this input when using the AR face camera. This object is added to the scene automatically when a face has been detected.
-
Sprite with the face object: this should be the name of the sprite that has turned on the AR face camera.
-
Target Height: the desired height of the object you are attaching. For example, if the scale is 100, and you want an object to be half of the height of the head, then its height should be around 50.
Demo - Soccer Nose
In this short program, we turn on the AR face camera in an empty scene, then we attach a soccer ball to the “Nose Tip” marker.
