Symbiotic Mind Computer Interaction for Information Seeking

MindSee is an exciting new research project partially funded by the European Commission 7th Framework programme which looks into advancing symbiotic interaction in the area of information seeking.

In recent years, real time analysis of user state based on signals from the brain and peripheral physiology has made progress, typically within separate branches of research. However, the ability to predict user intention from these inferred states is still a grand challenge in real-world applications.

As a novel solution, MindSee proposes to fuse EEG – as the main sensor – with peripheral physiological sensors (EDR, fEMG, eye gaze and pupillometry) and contextual information for unobtrusive acquisition of implicit measures of perception, cognition and emotion. 

Real-time estimates of these implicit, or hidden, user states will be used to complement keyboard and gestural input in a real-world application of scientific literature search where the information exploration of the user is guided by co-adaptation with the computer. The proposed Symbiotic Information Seeking System will provide a wide range of visualization resources that adapt the information retrieved according to its relevance, cognitive ergonomic complexity, and aesthetic properties. For the target application of scientific literature search, MindSee builds upon a cutting-edge retrieval system that has access to 50 million documents from the main scientific databases.

MindSee will be developed using an iterative approach with three full cycles of implementation and evaluation of increasingly complex symbiotic interactions in information seeking. The iterative evaluation of MindSee technology and validation of underlying methods will be conducted in realistic experiments with user groups that vary in skill and content area.

The MindSee project will develop a new symbiotic information retrieval system capable of more than doubling the performance of information seeking in realistic tasks, compared to mainstream tools. MindSee symbiotic interaction will deliver solutions to increase productivity and creative potential. Several MindSee results are exploitable and applicable to other information seeking contexts beyond scientific literature search.