Deaf and hard-of-hearing (DHH) students often encounter challenges in their educational settings, particularly in language literacy development, due to limited exposure to written and spoken language. We, therefore, adopted a User-Centered Design (UCD) process collaborated with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School for Deaf in Sri Lanka to develop and utilize a low-cost augmented reality (AR) system incorporated with localized real-time captions, aiming to enhance their learning experience. We designed caption presentation and placement methods, markups for inaccurate ASR transcriptions, and wearer speech visualizations addressing the existing design limitations of AR caption interfaces. User study with 24 DHH participants revealed a strong preference (87.5%) for our prototype and highlighted its potential to enhance the learning experiences.

PUBLICATIONS

SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings

Samaradivakara, Y., Ushan, T., Pathirage, A., Sasikumar, P., Karunanayaka, K., Keppitiyagama, C., Nanayakkara, S.C, 2024. SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA.

SeEar

Deaf and hard-of-hearing (DHH) students often encounter challenges in their educational settings, particularly in language literacy development, due to limited exposure to written and spoken language. We, therefore, adopted a User-Centered Design (UCD) process collaborated with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School for Deaf in Sri Lanka to develop and utilize a low-cost augmented reality (AR) system incorporated with localized real-time captions, aiming to enhance their learning experience. We designed caption presentation and placement methods, markups for inaccurate ASR transcriptions, and wearer speech visualizations addressing the existing design limitations of AR caption interfaces. User study with 24 DHH participants revealed a strong preference (87.5%) for our prototype and highlighted its potential to enhance the learning experiences.

PUBLICATIONS

SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings

Samaradivakara, Y., Ushan, T., Pathirage, A., Sasikumar, P., Karunanayaka, K., Keppitiyagama, C., Nanayakkara, S.C, 2024. SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA.

Deaf and hard-of-hearing (DHH) students often encounter challenges in their educational settings, particularly in language literacy development, due to limited exposure to written and spoken language. We, therefore, adopted a User-Centered Design (UCD) process collaborated with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School for Deaf in Sri Lanka to develop and utilize a low-cost augmented reality (AR) system incorporated with localized real-time captions, aiming to enhance their learning experience. We designed caption presentation and placement methods, markups for inaccurate ASR transcriptions, and wearer speech visualizations addressing the existing design limitations of AR caption interfaces. User study with 24 DHH participants revealed a strong preference (87.5%) for our prototype and highlighted its potential to enhance the learning experiences.

PUBLICATIONS

SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings

Samaradivakara, Y., Ushan, T., Pathirage, A., Sasikumar, P., Karunanayaka, K., Keppitiyagama, C., Nanayakkara, S.C, 2024. SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA.

SeEar

Deaf and hard-of-hearing (DHH) students often encounter challenges in their educational settings, particularly in language literacy development, due to limited exposure to written and spoken language. We, therefore, adopted a User-Centered Design (UCD) process collaborated with 8 DHH students and 2 Teachers of the Deaf (ToDs) from a School for Deaf in Sri Lanka to develop and utilize a low-cost augmented reality (AR) system incorporated with localized real-time captions, aiming to enhance their learning experience. We designed caption presentation and placement methods, markups for inaccurate ASR transcriptions, and wearer speech visualizations addressing the existing design limitations of AR caption interfaces. User study with 24 DHH participants revealed a strong preference (87.5%) for our prototype and highlighted its potential to enhance the learning experiences.

PUBLICATIONS

SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings

Samaradivakara, Y., Ushan, T., Pathirage, A., Sasikumar, P., Karunanayaka, K., Keppitiyagama, C., Nanayakkara, S.C, 2024. SeEar: Tailoring Real-time AR Caption Interfaces for Deaf and Hard-of-Hearing (DHH) Students in Specialized Educational Settings. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA.