Real-time 3-D reconstruction of complex scenes from long distances are shaping our present and future

Researchers at Heriot-Watt University, in collaboration with researchers from the University of Toulouse, France, have proposed a novel framework that combines statistical models with highly scalable computational tools from the computer graphics community to accurately extract the 3-D information in real-time (50 frames per second).

The novel method allows for target detection and imaging through cluttered scenes, enabling robust, real-time target reconstruction of complex moving scenes, and a fore-runner for the eventual sensing technologies needed for long-range automotive sensing, a key capability for next generation driverless cars.

Reconstruction of 3-D scenes has many important applications that are shaping our present and future, including autonomous cars, environmental monitoring and defense.

Researchers from Heriot-Watt have announced the remarkable new results on real-time image reconstruction in the prestigious science journal, Nature Communications.

One of these researchers, Prof. Stephen McLaughlin, head of the School of Engineering and Physical Sciences spoke of the research.

He said: "The ability to deliver real time 3-D video reconstruction enables the deployment of new sensing technologies for long-range automotive sensing, a key capability for next generation driverless cars."

Prof. Gerald S. Buller from the Institute of Photonics and Quantum Sciences said: "This work represents the most advanced real-time 3-D image reconstruction of a complex real-world scene yet demonstrated, placing Heriot-Watt firmly at the forefront of international efforts in this emerging field."

Dr. Yoann Altmann, RAEng Research Fellow at the School of Engineering and Physical Sciences said: "This work combining photonics, statistical image processing and computer graphics tools really illustrates how significant leaps forward can be achieved via multidisciplinary research."

Primitive laser-based radar—or lidar—techniques are currently being used in recent generations of cars to assess distance to neighboring vehicles or other potential obstacles.

Heriot-Watt has pioneered the use of the advanced time-correlated single-photon counting lidar approach which several advantages over existing approaches, allowing the use of eye-safe laser sources and enabling excellent resolution at long range (hundreds of meters to kilometers).

Recently, this technique was used to successfully reconstruct high resolution 3-D images in extreme environments such as through fog, with cluttered targets, in highly scattering underwater media, and in free-space at ranges greater than 10 km.

However, until now, a major limitation has been the significant amount of time required for the analysis of the recorded data. Recovering the 3-D information from the individual timed single-photon detection events is a challenging task that requires intensive image processing algorithms. Previous methods required dozens of seconds or even minutes to process a single lidar frame or imposed restrictive assumptions on the scene to recover, hindering practical 3-D imaging applications.

The findings were published today, Friday, November 1 in the scientific journal, Nature Communications.