EyeKnowYou is a DIY toolkit to estimate cognitive load and actual screen time using a head-mounted webcam. The do-it-yourself guide makes the set-up of the hardware and software easy. It uses videos of the eye to analyze corneal surface reflections to recognize screen usage and eye movements to determine cognitive load. We collected data from 17 participants to train a neural network to predict screen usage and cognitive load simultaneously. Our initial model, which is a 3D Convolutional Neural Network, achieves up to 70% overall accuracy for both tasks. The accuracy can go up to 90% when a personalized calibration is carried out. We encapsulated the model into a software toolkit that can be used by non-AI-experts. Both the software toolkit and our model are open-sourced for the community to further develop and build upon. Our aim is to collect more data over time, using this software toolkit, to improve model performance.

PUBLICATIONS

EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam

Kaluarachchi, T., Sapkota, S., Taradel, J., Thevenon, A., Matthies, D.J.C., and Nanayakkara, S.C., 2021. EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam. In MobileHCI ’21 Extended Abstracts: The ACM International Conference on Mobile Human Computer Interaction, Sept. 27- Oct. 1, 2021, Touluse, France.

EyeKnowYou

EyeKnowYou is a DIY toolkit to estimate cognitive load and actual screen time using a head-mounted webcam. The do-it-yourself guide makes the set-up of the hardware and software easy. It uses videos of the eye to analyze corneal surface reflections to recognize screen usage and eye movements to determine cognitive load. We collected data from 17 participants to train a neural network to predict screen usage and cognitive load simultaneously. Our initial model, which is a 3D Convolutional Neural Network, achieves up to 70% overall accuracy for both tasks. The accuracy can go up to 90% when a personalized calibration is carried out. We encapsulated the model into a software toolkit that can be used by non-AI-experts. Both the software toolkit and our model are open-sourced for the community to further develop and build upon. Our aim is to collect more data over time, using this software toolkit, to improve model performance.

PUBLICATIONS

EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam

Kaluarachchi, T., Sapkota, S., Taradel, J., Thevenon, A., Matthies, D.J.C., and Nanayakkara, S.C., 2021. EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam. In MobileHCI ’21 Extended Abstracts: The ACM International Conference on Mobile Human Computer Interaction, Sept. 27- Oct. 1, 2021, Touluse, France.

EyeKnowYou is a DIY toolkit to estimate cognitive load and actual screen time using a head-mounted webcam. The do-it-yourself guide makes the set-up of the hardware and software easy. It uses videos of the eye to analyze corneal surface reflections to recognize screen usage and eye movements to determine cognitive load. We collected data from 17 participants to train a neural network to predict screen usage and cognitive load simultaneously. Our initial model, which is a 3D Convolutional Neural Network, achieves up to 70% overall accuracy for both tasks. The accuracy can go up to 90% when a personalized calibration is carried out. We encapsulated the model into a software toolkit that can be used by non-AI-experts. Both the software toolkit and our model are open-sourced for the community to further develop and build upon. Our aim is to collect more data over time, using this software toolkit, to improve model performance.

PUBLICATIONS

EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam

Kaluarachchi, T., Sapkota, S., Taradel, J., Thevenon, A., Matthies, D.J.C., and Nanayakkara, S.C., 2021. EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam. In MobileHCI ’21 Extended Abstracts: The ACM International Conference on Mobile Human Computer Interaction, Sept. 27- Oct. 1, 2021, Touluse, France.

EyeKnowYou

EyeKnowYou is a DIY toolkit to estimate cognitive load and actual screen time using a head-mounted webcam. The do-it-yourself guide makes the set-up of the hardware and software easy. It uses videos of the eye to analyze corneal surface reflections to recognize screen usage and eye movements to determine cognitive load. We collected data from 17 participants to train a neural network to predict screen usage and cognitive load simultaneously. Our initial model, which is a 3D Convolutional Neural Network, achieves up to 70% overall accuracy for both tasks. The accuracy can go up to 90% when a personalized calibration is carried out. We encapsulated the model into a software toolkit that can be used by non-AI-experts. Both the software toolkit and our model are open-sourced for the community to further develop and build upon. Our aim is to collect more data over time, using this software toolkit, to improve model performance.

PUBLICATIONS

EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam

Kaluarachchi, T., Sapkota, S., Taradel, J., Thevenon, A., Matthies, D.J.C., and Nanayakkara, S.C., 2021. EyeKnowYou: A DIY Toolkit to Support Monitoring Cognitive Load and Actual Screen Time using a Head-Mounted Webcam. In MobileHCI ’21 Extended Abstracts: The ACM International Conference on Mobile Human Computer Interaction, Sept. 27- Oct. 1, 2021, Touluse, France.