Things you should know before you try to build your Mocap System
When putting together Hacktop Studios, I had to do a lot of research into Motion Capture. I wanted to keep the costs down so I can pass on the savings. I originally thought this would be a simple journey but the more I started researching and working with different vendors the more I learned how little I knew. Hopefully this posting will help anyone who is researching putting together their own motion capture system.
Inertia or Optical Tracking
You have two standard types of motion capture you can invest in. Inertia based systems and Optical systems. Optical systems are ones you typically see in movies where the actor has dots or balls all over them for the recording. Inertia based systems are typically done with track suits or straps and little sensors at each major bone tracking area.
Inertia is typically cheaper as the Opti systems require you to buy a lot of cameras. Optical systems can take longer to get your actor ready but they are great for large areas and having a lot of actors recording at once. Keep in mind the more actors/space you need the more cameras you will also need.
Since Inertia is typically cheaper we will mainly focus on some of the gotchas you will run into when shopping for initia based motion capture suits.
Susceptible to magnets
A lot of inertia based systems are susceptible to magnetic interference. You will want to keep them away from metal desks and cabinets. Try to use wooden or rubber props and have a lot of open space. The sensors have manometers in them and for a lot of cheaper motion capture suits they can be permanently damaged by magnetic contact. You can also see drift over time on some of the cheaper motion capture suits. Make sure you ask about permanent damage to sensors when in contact with magnets before buying a suit.
One thing that may not be obvious is a lot of inertia based systems have a hard time tracking the user as they move through space climb ladders. They use different tricks to try and determine movement and some are pretty good at knowing where the actor has moved to. You often will get a “skating” affect with some systems where the feet will slide around. Some systems will try to solve this with things like shoe insole sensors. Those actually work fairly well. Others will support the use of adding Vive trackers. If the system doesn’t mention insoles of Vive trackers then be very weary as they are likely to have a high slide rate.
Climbing ladders, sitting on bar stools, going up stairs…these are all things that can be hard for inertia based systems. I’ve seen some insole solutions actually work surprising well for these problems but more commonly they are solved with Vive trackers. Be sure to include the expenses of adding Vive trackers and 4 base stations to your overall costs. Vive systems are not included in the cost of a suit and something additional you will need to account for.
Headless Vive Setup
If you are using Vive Trackers for either positioning or for prop tracking, you will need to run SteamVR. SteamVR is designed to run with a full VR headset, controllers and at least 2 base stations. Even if you just want to use Vive trackers, SteamVR still expects you to have a headset and controllers to do the initial room setup. Once you have done the room setup you can use a ‘null’ headset driver so you don’t need the headset or controllers anymore in the future. Using a null headset driver isn’t part of the scope of this posting.
If you do not have a VR headset and can’t do a proper room setup then you can get bad readings that you will need to fix in post. These artifacts are typically your actor appearing below the floor or having elevation issues. These can be fixed after a recording but it is something to be aware of.
If you are only recording the body position (IE. no hands, face or voice recording) then you don’t need to pay attention to time syncing. However, if you ever plan to pair this system with anything else your actor will be doing you will want to sync the recordings. Each system often will use their own software to record and have their own sample rates. This makes splicing all the actions back together, very difficult. To help fix this problem, studios use time syncing. At the minimum ensure your suit can sync the real time into the FBX or exported file format. Ideally the suit should be able to be sync with an external time source such as Tentacle Sync.
You will want to know what your system can work with. Unless you are doing a purely academic project and all you care about is the data. If not, you will want to ensure that whatever you are pairing the suit with or the tools you plan on using work for you. Check the OS and file formats for your current pipeline. FBX and GLTF 2.0 are widely supported formats and at the time of this writing FBX is the most common in MoCap.
Make sure you ask what other vendors the suit can work with. If they have their own options for hand or face tracking, that cold be perfectly fine, but if the quality of their other products are not what you need you maybe stuck. See if they support other vendors hand or face tracking systems and what does the integration look like. There is a big difference in the amount of work if the system supports time syncing or if it has a full integration with a single record button. If you can use other vendors and have 1 place to hit record…that’s money.
You have two types of suits, strap based and track suit. Track suits are typically normal track suits with zipper pockets for sensors and access points, etc. Some suits have additional straps to keep the sensors in place. The suits are mainly designed for wire management. They are not necessary quick or easy to get on and ready. They do typically more aesthetically appealing but are often sold at a higher cost and they are not a one size fits all. If you are using actors you will want a Small, Large and Extra Large size to cover most of your actors.
Strap based systems often come with a shirt or vest to help with cable management. Straps have a more of a universal fit and are typically more versatile. They are also cheaper. You will want to ensure your actor isn’t wearing baggy clothes or something that will be so slick the sensors will move around.
Accuracy of fingers
Many of the finger tracking systems are designed to give general finger tracking but some are designed for more academic research and are very accurate. The academic ones often don’t have a concept of 3 dimensional spacing of the actual hands, so you’ll need to stitch that together with your suit or actor yourself. If you are using a motion capture suit, it helps a lot to have built-in support for your gloves. If not but you really want to use the gloves, ensure the SDK is complete and has the features you need for your project.
Gloves tend to bill themselves as a one size fits all. This is rarely true in practice. Some gloves use straps for fingers which can be harder to get ready but probably do fit the “fits all” category. Other gloves typically just go with a Medium/Large glove and cut off the fingertips. Yes, all hands will fit but the tracking can be weird and having the extra material on the fingers can be a pain.
The reality is, there is a decent amount of down time when recording and your actor will likely want to play with their phone or grab a bite to eat and it would be great if they didn’t have to remove the gloves and potentially change the calibration to check their text messages. If you are only using gloves yourself then you will most likely want to use your keyboard and mouse between takes and the extra material will get in your way. See what sizes they have and if they have any solutions for smaller or larger hands.
How Many Cameras?
This may come as a surprise but when recording an actors face you may want to have more than one camera. You can get better depth with two GoPros than with a single one. However there are nicer smarter cameras that are designed to read depth in faces, such as an iPhone. Yes, you can absolutely just use your phone if it’s good enough and you have software that can properly process it. However there are other things you will want to consider.
Lighting changes will affect the perceived depth of the face and they can mess with Optical tracking motion systems. You should have some dedicated light on the actor but what type may vary on your setup. Infrared light is a great option as it doesn’t blind the actor and isn’t affected by natural lighting changes but you can’t use them with Opti Tracking. If you need to use an LED strip or Opti tracking then you will want to go with a dedicated light or strobing mixed-color light. The idea behind the strobing is that you could get a more consistent illumination.
Most face tracking systems use headgear with a bar in front that holds the camera(s) and lighting system. However, some use chest mounts or do not come with a mounting system at all. You will most likely want a mounting system so check in advance. Chest mounts can be very awkward as the user must turn their whole body and not just their head. The head mounts can still get in the way but are usually the best for most situations.
When an actor is recording their face the are often saying lines. Depending on what you are doing it maybe great to record in a sound studio or build your own. It’s not cheap to do either but the end result will save you a lot of time in trying to make the 3d face match your voice actor if that’s an option.