This guide will provide some tips and guidelines for getting the best results from Faceware’s realtime facial tracking. Things like lighting in the room, camera angle, and objects blocking your face can have a big effect on how well the tracking works, so this guide helps explain what to look for. These tips apply to all Faceware products, both for realtime and non-realtime tracking.
Here are a few things to keep in mind when working with Facial Tracking:
Trim Facial hair so the lips are visible.
Glasses should be worn in a way to avoid interfering with eyebrows or removed entirely for best results.
Turn lights on if it is too dark to see your face in the video feed.
If there are bright highlights on the face, try to dim the lights.
Don’t allow directional lighting to cast deep shadows on the face.
Frame your face so that the top of your forehead and the bottom of your jaw are always visible in the frame.
The closer you are to the camera the better.
Make sure you calibrate with a “Neutral” expression, staring slightly below the camera.
You can always re-calibrate again until you’re happy with how it looks.
Calibration is maybe the single most important factor for achieving good tracking results. When calibrating, it is important to face the camera or tracking device and relax the face into a neutral position while looking at or slightly below the camera. This establishes a baseline for all of the expression values that will be generated. A bad calibration should be immediately obvious as the facial animation will be full of errors, at this point it is always recommended to try calibration again.
Calibration can be used to compensate for slightly incorrect animation results. For example, if your eyelids are normally more heavily lidded than the average person, you may need to squint your eyes slightly when calibrating. This results in the tracking giving you slightly lowered values for blink, and will result in animation where the eye is more fully open. By slightly changing your expression during Calibration you can correct many situations that may result in incorrect animation.
An important part of facial tracking is being able to clearly see the face. While the software can handle some occlusion, the best possible scenario is a face without facial hair, glasses, or long hair that hangs in front of facial features.
Some possible behavior you may expect from occlusion:
Facial Hair
Tracking may mistake moustache for upper lip.
Tracking may grab nose instead of mouth.
Calibration may fail entirely.
Glasses
Tracking may mistake the top of eyeglasses for Brows
Tracking may show incorrect eye position if the lens is especially reflective.
Long Hair or Bangs
Tracking may fail to find brows or Track brows incorrectly if hair hangs in front of brows.
Calibration may fail if hair frames the face in such a way that it no longer resembles a face.
Mike Tyson style Facial Tattoos
Tracking may try to fight you.
Tracking may want to compare notes on Pet Tigers.
For best results, trim beards/mustaches that cover the lips so the technology can clearly see the lips. Don’t allow lighting conditions that cause a great deal of glare on eyeglass lenses or wear eyeglasses pushed too high so that they interfere with your eyebrows. Style long hair back, away from the face so the hair isn’t blocking any facial features.
Framing the face is an important part of seeing smooth, accurate Facial Tracking and Animation. If your face is too far from your camera or tracking device, the animation may be extremely minimized or inaccurate. In the best possible situation, the face will fill the frame from top to bottom, with just enough room for you to open your jaw to its fullest extent and still be able to see from the top of your forehead to the bottom of your chin/jaw.
By framing the image tight to the face, more detailed facial movements with a wider range of motion may be captured. This gives a more natural look to your character’s animation.
Turning your head to the left or right greater than ~30 degrees may cause loss of tracking. The majority of your facial features must be within the frame and not occluded for tracking to succeed and for animation to be accurate and natural.
One of the most important aspects of working with the Faceware tracking feature is proper lighting. In an ideal scenario, your face would be lit with natural, even daylight. This often isn’t possible, so some best practices for proper lighting should be followed.
Auto Exposure
On some cameras, the Auto Exposure function may be too sensitive. Rapid changes in the composition of the images a camera is producing can have negative impact on the Tracking. Where Jitter is apparent, it may benefit the Tracking to disable Auto Exposure.
Tracking is poor in the Dark
Because Tracking needs to see the face to track it, pitch black rooms will cause poor results. While minimal light may work, you will likely see an increased amount of jittery or incorrect Tracking.
Monitor light is generally unsuitable for Tracking. While there will almost definitely be a result, it will be improved by adding additional light.
Half Shadows and Directional Lighting
If a bright light is focused on one side of the face, the shadows being cast to the opposite side may cause Tracking to react poorly or report incorrect results.
Direct, top down lighting may cause shadows beneath the eyes, brows and nose which may negatively affect the Tracking quality.
Rear Lighting
If there is bright light behind the user that shines into the frame of the camera, the Tracking quality may be degraded. Wherever possible try to position lighting in front of the user, preferably with even lighting to either side of the face.
Bright Windows and Lights
Extremely bright light may cause ‘blowouts’. Bright white reflections on the skin can not be Tracked properly. If possible, dim or change direction of any bright directional lights to avoid degradation of Tracking quality.
The basic rule of thumb is that a face must look like a face in order for Studio and Analyzer to track properly. Any lights or shadows that skew or alter your appearance in such a way that the face is distorted will result in unexpected or questionable tracking.