Amazing work from the University of Tokyo. Dynamic projection mapping on non-rigid and self-occluding surfaces at 1000 fps and 3ms delay.
Projection mapping attracts attentions as an emerging technology to extend the real world. However, almost realized examples have been limited to static or quasi-static environments. This research aims at overcoming this limitation and realizes dynamic projection mapping in which dynamically changing real-world and virtual visual information are completely merged in the level of human visual perception. This high-speed dynamic projection mapping requires a high-speed projector enabling high-frame-rate and low-latency projection. In order to meet this demand, we have developed a high-speed projector “DynaFlash” that can project 8-bit images up to 1,000fps with 3ms delay.
Using these base technologies including DynaFlash and Deformable Dot Cluster Marker, we realize a new dynamic projection mapping onto deforming non-rigid surface. In this demonstration, by drawing the marker on the target with IR ink, we allow the marker to be invisible to human and enable robust sensing independently of the projected images. In our technology, both the projection and sensing are operated at a speed of 1,000 fps. Therefore, it is possible to keep the projection consistent with the deformation and extend the real world as if the projected image is printed or existed as an original (digital) texture on the target. Especially, focusing on new paradigms in the field of user interface and fashion, we have demonstrated dynamic projection mapping onto a deformed sheet of paper and T-shirt. Also we show that projection to multiple targets can be controlled flexibly by using multiple markers.