LookUp3D: Data-Drive 3D Scanning

The goal of calibration is to create the unique dictionary $C_i\left(d\right):R^{+}\to \backslash to{R}^3$ between depth and color for each pixel $i$. Our calibration procedure uses a planar calibration target (filled with fiduciary markers) moved by an off-the-shelf linear stage. We project a fixed pattern sequence onto the calibration target and record, per-pixel, the color reading and the depth measurement.

We deviate from traditional structured light: to recover depth, we rely solely on the calibration board and the linear stage, without the need to model the projector defocusing, vignetting, intrinsic and extrinsic parameters. We test this hypothesis with a set-up where the quality of the optics of the projector does not alter the results achieved by LookUp3D, whereas a traditional structured light method reliant on triangulation suffers a lot in quality of reconstruction.

The scanning procedure is as follows: we project the same fixed pattern sequence from calibratioon now onto an object of unknown geometry and capture an image of intensity $I$. Reconstruction compares each pixel in the measured color image $I$ with the calibrated per-pixel lookup table. In its unoptimized form, the operation is a simple, embarrassingly parallel lookup to find the depth whose color is closest to the observed pixel $I_i$, or $$d_i^{*}={argmin}_d\left|C_i\right(d)-I_i{|}_2$$ where the search is done offline on a discrete set of depths. The residual $r_i=\left|C_i\right(d)-I_i{|}_2$ provides a per-pixel confidence measure that can be thresholded to filter unreliable points.

We validate LookUp3D across three hardware configurations: a standard DLP projector, an inexpensive LCD projector, and a custom-built analog projector for high-speed. With the high-speed prototype, we achieve accurate reconstructions at 450 fps and 1 Megapixel resolution. We recover the free-fall of a bunny at 450 fps with relative motion errors below 2%.

LookUp3D achieves reconstruction quality comparable to traditional structured light with the DLP projector, and significantly outperforms it with the LCD projector. While the quality of traditional Gray Code reconstruction deteriorates significantly with the low-quality projector -- due to its reliance on accurate triangulation -- LookUp3D remains robust.

We show below a chess board, a paint brush, a 3D-printed house, and a monkey statue, all reconstructed with 11 channels. These scenes were captured with our DLP Projector with ceiling lights on.

Since we do not need to explicitly model the projector for scanning, we use a custom analog projector that can flicker LEDs at frequencies above 1kHz. We expose a pattern onto a 35-mm color film slide, develop it, and place in front of one of the LEDs, allowing the projection of an RGB patterm at extremely high frequencies. We can then 3D scan slow-motion scenes.