We are working to prototype, build and evaluate novel systems that integrate knit sensors with virtual and augmented reality. These emerging technologies are complementary and would open up new possbilities for improving human performance as well as creating playful experiences. Recent work has shown that versatile and durable knit touch sensors can be manufactured through standard fabrication methods, with little post-processing. These have potential for being integrated into clothing, hats, furniture, vehicles and anywhere that fabric can be found today. These smart fabrics enable accurate touch input through capacitive sensing, providing an always-available input modality. They also may have additional sensors to monitor aspects of human performance, behavior and health.
The information provided by VR or AR could provide supplemental information (e.g. status updates, confirmation of input), but also could directly augment the fabric to provide indications of available functionality on the touch sensor. For example, a generic fabric-based sensor, could have multiple purposes, depending on the situation, just as a mouse or keyboard currently are highly multi-purpose. By augmenting fabrics with additional virtual information, the touchable areas could have visual affordances, indicating functionality directly on the general purpose fabric sensor. This would enable the same piece of fabric to be used in different ways, mediated by the VR/AR system and software, which changes dynamically.
Publications
McDonald, D.Q., Mahajan, S., Vallett, R., Dion, G., Shokoufandeh, A., Solovey, E.T. (2022). Interaction with Touch-Sensitive Knitted Fabrics: User Perceptions and Everyday Use Experiments. In CHI Conference on Human Factors in Computing Systems (CHI ’22), April 29-May 5, 2022, New Orleans, LA, USA. ACM, New York, NY, USA, 20 pages.
Mahajan, S., McDonald, D.Q., Vallett, R.J., Liu, F., Solovey, E.T. (2022). Exploring Use of AR and Soft Knitted Sensor Technology for Co-located Parent-Child Quality Time. ACM GROUP’22 Workshop on Technologies for Children at Home Exploring Ways To Support Caregivers With Child-friendly Media Technologies For The Home. 5 pages.
McDonald, D.Q., Vallet, R., Solovey, E., Dion, G., Shokoufandeh, A. (2020). Knitted Sensors: Designs and Novel Approaches for Real-Time, Real-World Sensing In Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 4 (4), Article 145 (December 2020), 25 pages.