FingerReader

We believe smart devices should be more inclusive, and assistive devices deserve more design and innovation. The low demand for assistive devices has led to high cost and little innovation. While mobile phones have gotten smarter with each passing day, assistive devices like the white cane for the blind, has pretty much remained the same all these years.

In our previous work, we have created a FingerReader-v0 (Finger-worn Assistive Augmentation) with the vision of empowering and enabling people with visual impairments. With FingerReader, blind users are able to simply point at products, text phases etc. and have the results spoken back to them. Our findings suggest that a direct mapping could greatly improve interaction (e.g. easy “re-reading”), as well as scaffold the mental model of a text document effectively, avoiding “ghost text”.

With FingerReader, we aim to transform this proof of concept project into an affordable everyday product that will sustainably change how the visually impaired community can independently access information on the go.

Publications

  • Roy Shilkrot, Jochen Huber, Jürgen Steimle, Suranga Nanayakkara, Pattie Maes. Digital Digits: A Comprehensive Survey of Finger Augmentation Devices. ACM Computing Surveys (CSUR), Vol. 48, 2, Article 30, November 2015. [PDF] [DOI]
  • Roy Shilkrot*, Jochen Huber*, Meng Ee Wong, Pattie Maes and Suranga Nanayakkara [* equal contribution]. "FingerReader: A Wearable Device to Explore Printed Text on the Go", In CHI ’15: Proceedings of the 2015 ACM annual conference on Human Factors in Computing Systems, ACM, 2015. [PDF]
  • Roy Shilkrot, Jochen Huber, Connie K. Liu, Pattie Maes and Suranga Nanayakkara. “FingerReader: A Wearable Device to Support Text-Reading on the Go”. In Proceedings of CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’14). ACM, New York, NY, USA, p. 2359-2364. [DOI], [PDF]
  • Roy Shilkrot, Jochen Huber, Connie K. Liu, Pattie Maes and Suranga Nanayakkara. “A Wearable Text-Reading Device for the Visually-Impaired”. In Proceedings of CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’14 – Video Showcase), p. 193-194. [DOI]

Press and Awards