The brain is the ultimate frontier.
Human Computer interaction is dump. Until recently the communication bandwidth between humans and computers was very low in both terms of the richness of information type and information rate (just a few mouse clicks/keyboard strokes). To build the next generation of computing interfaces we need to better understand the cognitive underpinnings of human computer interaction and to substantially enrich that information bandwidth by exploiting user’s rich expressive, cognitive and affective state information. And our brain is where all this information is kept.
The screenshot shows a tool developed for affective evaluation of human computer interaction. It exploits Emotiv EPOC, a 14 electrode consumer EEG device to monitor in real time, expressive (the blue face avatar) and affective information (excitement, engagement and frustration metrics) of a user interacting with a previously developed multimodal iphone application. The tool allows to record, play and analyse user sessions, providing rich information that we can exploit in following iterations of the product design cycle.
Some Details …
Evaluation setting. Depicted counter clockwise is the iphone device, a custom desinged GSR (Galvanic Skin Response) apparatus (arduino and breadboard), the Emotiv device, the audio headset and the PlayStation Eye camera.
Two more screenshots of the Affective Evaluation Studio replaying previously recorded sessions. (a) Standard edition. The two main components depicted are the video and affective plot (see Fig 7.6) widgets. The vertical blue line indicates the playing position in the affective data corresponding to video frame displayed. The user can click on any position of the plot to move in that particular moment in the video stream or vice versa using the video slider. The two widgets in the right of the video widget display the 14 electrode contact quality and the user face expression widget. (b) Research edition. Offers additional EEG processing capabilities such as EEG plot (found below affective plot) and single channel analysis plot and spectrogram (shown when selecting specific channel). It also provides real time scalp plots (next to video widget) which show EEG power distribution for selected spectrum bands animated through time.
Of course, the system can be used not just for HCI evaluation; Following is a sample screenshot of Affective Evaluation Studio used in trailer affective evaluation (Inception movie)
Further analysis includes EEG signal processing using EEGLab. The following two screen-shots show the 14 ICA (independent component analysis) components of a sample session shown as scalp maps. The IC3 component is an eye blink artifact component and will be rejected.
Future work includes cognitive load estimation, emotion analysis, etc.