You Can Now Use Your Cellphone As A 3D Modeling Tool

Researchers have created a new technology that uses the camera and accelerometers in an average cellphone to measure real objects in 3D space. Created at Carnegie Mellon University, you can use the system to build “3D models of the world” just by waving your phone around an object or scene.

The accelerometers, called inertial measurement units, roughly tell the phone’s software the position of the phone in space. IMU’s are very noisy and rarely usable to truly assess a phone’s actual orientation with any degree of real accuracy but coupled with the camera you can get far more useful results.

“We’ve been able to get accuracies with cheap sensors that we hadn’t imagined,” said Simon Lucey, associate research professor in the CMU Robotics Institute in a release. “With a face tracker program, we are able to measure the distance between a person’s pupils within half a millimeter.” Such measurements would be useful for applications such as virtual shopping for eyeglass frames.”

The tool allows for better computer vision and could mean that you could create a 3D model of almost anything with your smartphone alone. The researchers expect to use this in self-driving cars, bypassing expensive and “power-hungry” radar. The team used the technology to create something called Smart Fit that finds the perfect glasses frames for your face.

“The trajectory we create with these cheap IMUs will ‘drift’ over time, but the vision element we create is very accurate,” said Lucey. “So we can use the 3-D model to correct for the errors caused by the IMU, even as we use the IMU to estimate the dimensions of the model.”