There is something spooky about being able to see and talk to the pirate Blackbeard while one walks down a dark alley and then stepping right through him as he disappears into thin air. Such entertainment experiences are now possible thanks to a shape-changing fog screen that has been developed at the University of Sussex.

The mid-air display allows users to reach through it and interact with 2D and 3D objects, all the while changing shape and position to optimize visibility, facilitate interaction or flexibly adapt to single or multiple users interacting with the floating content.

While shape-changing displays and fog screens already exist, in labs at least, this is the first time the two technologies have been combined, which opens up new possibilities.

The invention, called MistForm, is being presented next week (Wednesday 10 May) in Denver, USA, at the world’s biggest conference on human-computer interaction, called CHI 2017.

Fog displays scatter light in an uneven way – a different amount of light in different directions. By understanding these scattering patterns and controlling the shape, common visibility and brightness problems can be addressed. Also, by making use of shape reconstruction and 3D projection algorithms, MistForm adjusts its shape to better support user interaction, all while removing any image distortion caused by projecting on moving, curved fog surfaces.

Dr Diego Martinez Plasencia, a Lecturer in the Interact Lab at the University of Sussex’s School of Engineering and Informatics, said: “This has the potential to enable new forms of interaction and collaboration with computers, liberating users from fixed, static screens and opening up whole new interactive spaces.

“This latest study builds upon early concepts to provide a far more enjoyable and reliable user experience, by combining two exciting technologies to combat the issues of distortion and uneven brightness that we often see with fog screens.

“With other 3D display technologies your eyes need to focus on the display surface, even if you see an object “popping out” of the screen. If you then try to touch it, your eyes will need to focus either on your hand or on the display, which soon can lead to eye fatigue (unless the 3D object and your hand are really close to the display surface). MistForm can adapt to these scenarios, moving the display surface so that both the object and the hand remain comfortably visible. With this kind of technique, we can provide comfortable direct hand 3D interaction in all the range your arms can reach.”

MistForm is roughly the size of a 39-inch TV screen and is formed of fog stabilised by curtains of air. The screen can move towards and away from the user and can bend into numerous different shapes. For example, it can curve around two collaborators, providing optimum visibility for both people, or it can take on a triangular shape if those two people need to work on different areas of the screen independently.

The display is projected from above and motion trackers detect the user’s movements and intentions, allowing the display to adapt accordingly.

Further information: http://dl.acm.org/citation.cfm?doid=3025453.3025608