Home » Mixed Reality Cane

Attribution-NonCommercial-ShareAlike 4.0 International

Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license.

Mixed Reality Cane

Despite advances in virtual reality (VR), people who are blind or have low vision (BLV) cannot fully benefit due to reliance on expensive headsets and accessories. This research study introduces a mobile VR app that enables BLV users to explore virtual environments using an iPhone on a selfie stick, simulating a long cane in VR. The app uses Augmented Reality (AR) to track the iPhone’s movements, providing auditory and vibrotactile feedback when the virtual cane contacts objects. The MR Cane concept can be expanded for navigation, training, and entertainment for BLV users.

Publications:

  • Hong Zhao, Oyewole Oyekoya, and Hao Tang, Enhancing Virtual Mobility for Individuals Who Are Blind or Have Low Vision: A Stationary Exploration Method, ACM Interactive Surfaces and Spaces (ISS), Poster, 27-30 October 2024, Vancouver, BC, Canada, 2024
  • Zhang, L., Wu, K., Yang, B., Tang, H., and Zhu, Z. Exploring Virtual Environments by Visually Impaired Using a Mixed Reality Cane Without Visual Feedback, ISMAR 2020 – International Symposium on Mixed and Augmented Reality, November 9-13, 2020.
  • Zhenchao Xia, Oyewole Oyekoya, and Hao Tang. 2022. Effective Gesture-Based User Interfaces on Mobile Mixed Reality. In Symposium on Spatial User Interaction (SUI ’22), December 1–2, 2022, Online, CA, USA. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3565970.3568189 – pdf