Planning Your Shoot
Filming with DepthKit is unusual in some ways but it has the benefit of not disrupting the conventional set dynamics. The DP still operates the camera and the DepthKit team can film alongside. To understand the intricacies & conventions of filming with DepthKit Capture on set see notes on filming.
The technology that makes the Kinect work relies on the near-infrared spectrum for illumination. If you use lights containing a lot of infrared content it can disrupt the Kinect, degrading your depth capture.
Unsuitable Light Types
- Direct sunlight
Recommended Light Types
- Fluorescent tube lights
- LED Light Panels
- Indirect sunlight
A rule of thumb is that any lights that are hot to the touch (including our sun) generally introduce some near-infrared pollution, making them not ideal for scanning. But we always recommend that you test your lights!
When considering how to light your scene for DepthKit we recommend considering your final design and designing your lighting for that context. For example, if I intend to place my subject so that they're illuminated from the left by a window in VR, then on set it's best if I put a large softbox to the left of my subject to mimic that light.
The Kinect has a range of around .5m and 4.5m (1'8"-14'6".) We also find that based on the brightness of the IR illuminators on the Kinect V2 that when objects are close to the near range they're often too bright for it to sense. The quality of the depth data falls off as you step away from the sensor, so the general recommendation is that you fill your depth frame with your subject whenever possible to get the optimal resolution.
The Kinect V2 has a depth resolution of 512 x 424 pixels with a field of view (FoV) of 70.6° x 60° resulting in an average of around 7 x 7 depth pixels per degree. Due to the use time-of-flight as the core mechanism for depth capture, each pixel in the depth image contains a real measured depth value (z-coordinate). Read a more thorough breakdown.
Field of View
The depth camera on the Kinect V2 has a FoV of 70.6° x 60° and the color camera has a resolution of 1920 x 1080px with a FoV of 84.1° x 53.8°, resulting in an average of about 22 x 20 color pixels per degree. There is a great interactive Kinect 1 vs Kinect V2 FoV comparison tool here.
We recommend setting the framerate on your camera to your local standard around 24p. The Kinect shoots at a variable framerate around 30fps and DepthKit conforms it to the framerate of the footage provided. We recommend shooting at 23.976 fps / 24p but it will work at any framerate below 30fps.
On-set Material/Garment Constraints
Avoid the use of transparent or reflective materials! This means glasses, mirrors, windows, shiny leather, waxed jeans, etc.
The Kinect relies on cameras and a small infrared emitter to sense depth, so it is susceptible to many of the same optical illusions that trick our eyes. It will struggle to resolve transparent surfaces like glasses or windows. It will also represent reflective surfaces in strange ways – for example if I film something in a mirror, the object will appear to be inside the mirror. It occasionally will struggle with certain fabrics or materials – leather or some kinds of waxed or otherwise treated jeans can appear to be missing. On critical shoots, always test your materials in the Kinect before shooting.
When choosing patterns or colors in clothing, adhere to all the same constraints that govern a typical film shoot – small repeating patterns can cause moiré effects and if garments are too bright or too dark they will indeed appear bright or dark respectively when filmed with DepthKit.
The Kinect must be mounted to and maintain the same relationship to the color camera during the whole shooting process, so make sure you have a strong mounting solution.
In order to calibrate and film with DepthKit you'll need to attach a Kinect firmly to your camera in a way that allows you to adjust the angle of view. We have designed a set of hardware mounts that allow you to attach the leading sensors to a camera – please purchase at left.
Mount kits include one aluminum base and a set of interchangeable arms that can accommodate the Asus/PrimeSense, Kinect V1 or Kinect V2 depending on the mount you buy. The sensor will need to be mounted to the camera for the duration of your shoot – if you change the relationship between camera and sensor you'll have to re-calibrate.
Note: we typically only ship mounts on Mondays. Don't forget to allow for ample time in customs for international orders. If you're on a tight timeframe or have any questions about mounts please contact us directly.
Choosing a space to shoot/Type of shot
When planning a DepthKit shoot, it's important to understand the environment you're shooting in before arriving.
DepthKit works best in locations with diffused or controlled lighting. Have a discussion with the team you plan to work with about the location and your requirements for a space. If possible, head to the space ahead of time and run a test shoot with DepthKit to see how it responds to the lighting and space.
On shots that are critical to DepthKit, it's also important to have a clear dialogue with the person running the color camera if it isn't you. A DepthKit shot requires a much wider lens than normal, so be sure to manage expectations and understand what is needed for a given shot from DepthKit's perspective.