Jester is a communications portal incorporated into a mirror that allows visitors to see people in a similar setting at a distant location and undertake gestural communication with them in real time. More specifically it is a portal between the restrooms of different Cirque du Soleil shows at different locations in the world. A culturally sensitive gesture recognition system translates gestures into written text on the opposite end of the portal. Interface and interaction modality
The user is put in front of a regular mirror, the only difference is in the content. The camera/motion tracker is placed on the top of the mirror and should be less visible as possible, only a hole is displayed. All the system is placed on the back of the mirror. The system contains a full database of all the codified body gestures used in all the cultures and translate them in the language where the image is broadcasted. In this example a swiss man waves his hand to say hello, while a japanese bow himself. Technology
The effectiveness of this system is based upon the use of a panoptic cameras and panoptic displays. This means that the spaces viewed are being captured using three dimensional panoptic camera technology that allows the replay to be viewed from any angle. The panoptic display conveys the image such that the opposite room and the people in it appear in correct perspective regardless of the position of the viewer. User experience
An important aspect to test was to see how people try to express themselves in front of another human figure without using verbal forms of communication. So i setup a fake window with an opaque glass and i gave the user a list of question to ask to the figure using only body languages. The questions are very basic, the only possible answers can be “good” or “bad” and “yes” or “no”. There’s a link to the video in the description. The outcome was that all of them wave their hand to say hello and understand the same movement as a reaction, and to express “well” or “good” they use these two configurations: These gestures are widely used in our western culture but would be interesting to comprare this with others. To observe for example how an indian talks with a north-african or an italian with a japanese. One of the people interviewed started to move his finger on the glass to draw symbols, like a house to indicate the place or the sun for the weather.
