Abstract
“émoi” is both a service and data representation interface. In utilising today’s digital systems (ie. e-commerce, ubiquitous computing, sentiment analysis algorithms), émoi attempts to create an edible gustatory and oral somatosensory representation of the user’s emotion that can support and enrich the already existent interpersonal communication that is carried out through the sharing and exchanging of chocolates as gifts. Such chocolate-sharing scenarios include, but are not limited to, special occasions, holidays and celebrations, expressions of affection, greetings, or condolences. émoi also attempts to explore new and unusual scenarios that emerge as a result within this context.
émoi’s emotion representation language is based on psychological research into the human gustatory and oral somatosensory experience. As a result it attempts to communicate user sentiment using appropriately defined qualities of taste, texture, shape and colour through chocolate.
Using émoi’s mobile, tablet or web app, the user can input a sentimental message or select discrete emotions that they wish to communicate to a recipient. These emotions are then translated into chocolates that embody the flavour, texture and shape of the corresponding emotion. The sender can also define the ambiguity and anonymity of their message and signature as perceived by the recipient. Then, upon receiving the chocolates, the recipient can uncover the communicated emotions via émoi’s emotion-to-chocolate representation interface.
Interface and interaction modality
The first interaction step within émoi’s service takes place through its online/digital interface, where the user can define their emotions and execute their chocolate order. In some parts of the scenario, the user may also interact with shop employees in order to claim their order of chocolates. However, the final interaction event occurs between recipient and émoi’s edible chocolate-based gustatory and oral somatosensory representation interface.
Technology
The émoi service’s online user interface (mobile,tablet, web) was prototyped using designs produced on Adobe Illustrator and Adobe Photoshop. The interactive prototypes were realised using invision, utilising both static screens and animated gif screens produced using Adobe After Effects.
The final product would also utilise a sentiment analysis algorithm system to extract emotions form text and audio messages input by the user. The service infrastructure would be supported by its dedicated server and database to receive and manage orders. The production company would also have their own internal order management system, as well as the related chocolate production machine and system. Product leaflets would be produced via printing. And upon assembly the order would be packaged and delivered to the recipient, or picked-up in store by the customer. Order and package tracking and user/customer notification systems would be implemented throughout this process where necessary.
The resulting edible chocolate interface that is received by the user includes the packaging box, cover and tag, and contains chocolates as well as the corresponding order leaflet. If the order includes an audio message, the message is embedded within the leaflet via a small audio speaker sound module. The recording is activated by pressing a button located beneath the appropriately indicated location on the leaflet surface.
User experience
émoi was designed to provide an interface that is both simple and customisable for the user. There product definition process is meant to be carried out in three simple steps. In the first step the user can provide a message or manually select the emotions they wish to communicate. In the second step the user can define the intensity of each emotion, and the dominance of their selected emotions over one another. In this step the user can also visualise how the selected emotions are translated into chocolate flavour, texture and shape. In the last step, the user can customise the ‘secrecy level’ of their communication, allowing them to choose to be open and candid or anonymous and secretive.
First, the user selects one of three methods to define the emotions they wish to communicate to their recipient. The user can choose to create a message (either written or audio) directed at their recipient, which is in turn used by a émoi’s integrated sentiment analysis algorithm to extract the related emotions. Alternately, the user can manually input their desired emotion by selecting them from a list or searching for them.
Upon defining the selected emotions, the user can then adjust the representation of the related in the emotion in the box of chocolates. For example, the user can control the intensity of their selected emotion and dynamically visualise how this affects the corresponding chocolate’s texture (more intense = more texture). If the user is conveying more than one emotion in the box, the user can define the ‘predominance’ of each of the emotions by adjusting the amount of times it occurs in the box. This predominance value can also be visualised as a percentage in a pie chart. If the user selected positive emotions, the corresponding chocolate shape would be round. If the user selected a negative emotion, its corresponding chocolate shape would be square. While if the user selected a mixed emotion the resulting chocolate would be a sharp-round combination shape. In this section of the interface, the user can also see access a visualisation of the final chocolate outcome, as well as access both the flavour and texture ingredients of their chocolate.
In the following section, the user can define the ‘secrecy level’ of their chocolate-based communication as perceived by their recipient. For example, the sender can choose to sign or withhold their name from their order. The sender can also choose to include or conceal their written or recorded message. The sender can also choose to include or omit the emotion to flavour guide in the leaflet that is enclosed with the chocolates upon receipt.
émoi is suitable for classic scenarios where a sender would like to offer a present in the form of chocolate. These include love (ie. valentines, anniversary, general affection), celebration (ie. graduation, birthday, weddings), empathy/sympathy (ie. get well soon, condolences). However, émoi also has a unique capacity to allow the user to gift chocolate in more personal and even unusual contexts. For example, émoi chocolates could be presented anonymously from a person wishing to express negative feelings to someone else.
émoi aims to allow the user to exercise their own creativity and freedom in creating their own forms of emotional expressions, and in the degree of openness or covertness that they please.
Finally, émoi also allows the recipient to experience a new form of information representation that transcends the visual sensory modality. émoi aims to allow the user to experience data (emotion) representation in the form of a gustatory and oral somatosensory language, utilising flavour, texture, shape and colour as its symbols, icons and information conveying qualities.
Research and development context
The development of this project placed a significant focus on studies into the psychological ongoing of human sensory perception, with a particular focus on the oral and gustatory sensory capacities.
The research concept however first emerged from a desire to address sensory capacities beyond visual perception. This is because as many (ie. Hiroshi Ishii, Mark Weiser) in the field of HCI and technology have often noted, we place an unbalanced preference towards visual and graphic representation in the design of our interfaces today. As a result, I sought to explore other sensory mediums and how they could be used to communicate data and information to the user.
I eventually was drawn towards food based mediums because of their inherent ability to encompass a multimodal sensory experience (ie. taste, tactile texture, visual perception and smell). In addition, the human ability to discern and distinguish between values in the realm of gustatory and oral somatosensory perception is significantly advanced as a result of our ingrained human dependence on food-based nourishment.
I found that the gustatory and oral somatosensory modalities were also particularly interesting and effective mediums because of research that showed how we as humans have an generally inherent ability to attach consistent meanings to tastes and textures in food experiences. Such research as conducted by Dr. Charles Spense from Oxford’s Crossmodal Research Laboratory have shown that bitter flavours, rough textures, dark colours and angular shapes are generally associated with negativity and large physical properties and terms, while sweet flavours, smooth textures, warm colours, and round shapes are associated with positivity and small physical properties in terms. Such research is firmly rooted in the “Bouba/Kiki effect” concept as first proposed German-American psychologist Wolfgang Köhler in 1929.
In addressing the data that is to be communicated by this gustatory representation system, I chose human emotion. This is because the relation between what we eat and what we feel has long been perceived as being intertwined. This strong relationship with emotion is also why I chose chocolate as the particular food based medium. Furthermore, unlike many other foods, chocolate has a unique capacity to be exceptionally malleable in terms of the way it is can be combined with such a diverse variety of ingredients, flavours and textures. This further allows for a diverse palette of qualities to choose from in the information representation design.
In addressing the emotion definition and categorisation system utilised by the service, I referenced some of the most widely used systems by both psychologists and already existing digital sentiment analysis systems (ie. Geneva Emotion Wheel, Robert Plutchik emotion wheel).
Of course, I also researched chocolate production at local artisanal chocolatier, asking them about their user user trends and their internal production process and protocols.