FingerReader-v0

Accessing printed text in a mobile context is a major challenge for the blind. In this project, we conceptualise and design a finger-worn device, FingerReader, that assists blind users with reading printed text on the go. We introduce a novel computer vision algorithm for local-sequential text scanning that enables reading single lines, blocks of text or skimming the text with complementary, multimodal feedback. This system is implemented in a small finger-worn form factor, that enables a more manageable eyes-free operation with trivial setup. We also look to go beyond usage for persons with a visual impairment, and expect that the FingerReader may be useful to scaffold dyslexic readers, support early language learning for preschool children and reading non-textual languages.

For more information about FingerReader, it's conception, future directions and alternatives please contact us.

The industrial design was done by Jonathan Ng Ming-En and Sanjay Pushparajan.

Publications

  • Roy Shilkrot, Jochen Huber, Jürgen Steimle, Suranga Nanayakkara, Pattie Maes. Digital Digits: A Comprehensive Survey of Finger Augmentation Devices. ACM Computing Surveys (CSUR), Vol. 48, 2, Article 30, November 2015. [PDF] [DOI]
  • Roy Shilkrot*, Jochen Huber*, Meng Ee Wong, Pattie Maes and Suranga Nanayakkara [* equal contribution]. "FingerReader: A Wearable Device to Explore Printed Text on the Go", In CHI ’15: Proceedings of the 2015 ACM annual conference on Human Factors in Computing Systems, ACM, 2015. [PDF]
  • Roy Shilkrot, Jochen Huber, Connie K. Liu, Pattie Maes and Suranga Nanayakkara. “FingerReader: A Wearable Device to Support Text-Reading on the Go”. In Proceedings of CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’14). ACM, New York, NY, USA, p. 2359-2364. [DOI], [PDF]
  • Roy Shilkrot, Jochen Huber, Connie K. Liu, Pattie Maes and Suranga Nanayakkara. “A Wearable Text-Reading Device for the Visually-Impaired”. In Proceedings of CHI ’14 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’14 – Video Showcase), p. 193-194. [DOI]

Press and Awards