By detecting specific cognitive states that occur when multitasking, we can build user interfaces that better support task switching, interruption management and multitasking.
Multitasking has become an integral part of work environments, even though people are not well-equipped cognitively to handle numerous concurrent tasks effectively. Systems that support such multitasking may produce better performance and less frustration. However, without understanding the user’s internal processes, it is difficult to determine optimal strategies for adapting interfaces, since all multitasking activity is not identical. We conducted two experiments leading toward a system that detects cognitive multitasking processes and uses this information as input to an adaptive interface. Using functional near-infrared spectroscopy sensors, we differentiate four cognitive multitasking processes. These states cannot readily be distinguished using behavioral measures such as response time, accuracy, keystrokes or screen contents. We then developed a human-robot system as a proof-of-concept that uses real-time cognitive state information as input and adapts in response. This prototype system serves as a platform to study interfaces that enable better task switching, interruption management, and multitasking.
Related Publications
- D. Afergan, E. Peck, E.T. Solovey, A. Jenkins, S. Hincks, E.T. Brown, R. Chang, R.J.K. Jacob. Dynamic Difficulty Using Brain Metrics of Workload. Proc. ACM Conference on Human Factors in Computing Systems CHI ’14, ACM Press (2014). Best Paper Award Honorable Mention.
- E.T. Solovey, P. Schermerhorn, M. Scheutz, A. Sassaroli, S. Fantini, R.J.K. Jacob, “Brainput: Enhancing Interactive Systems with Streaming fNIRS Brain Input,” Proc. ACM Conference on Human Factors in Computing Systems CHI’12, ACM Press (2012). Best Paper Award Honorable Mention.
- E.T. Solovey, K. Chauncey, F. Lalooses, M. Parasi, D. Weaver, M. Scheutz, P. Schermerhorn, A. Sassaroli, S. Fantini, A. Girouard, R.J.K. Jacob, Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface, Proc. ACM Conference on Human Factors in Computing Systems CHI’11, ACM Press (2011). [link]