Emotion recognition with simulated phosphene vision
until further notice
New York, NY : Association for Computing Machinery (ACM)
InLiu, X.; Min, R.; McDaniel, T. (ed.), MAHCI '19: Proceedings of the 2nd Workshop on Multimedia for Accessible Human Computer Interfaces, pp. 1-8
MM '19: The 27th ACM International Conference on Multimedia (Nice, France, 21-25 October 2019)
Article in monograph or in proceedings
Display more detailsDisplay less details
SW OZ DCC AI
Liu, X.; Min, R.; McDaniel, T. (ed.), MAHCI '19: Proceedings of the 2nd Workshop on Multimedia for Accessible Human Computer Interfaces
SubjectBiophysics; Cognitive artificial intelligence
Electrical stimulation of retina, optic nerve or cortex is found to elicit visual sensations, known as phosphenes. This allows visual prosthetics to partially restore vision by representing the visual field as a phosphene pattern. Since the resolution and performance of visual prostheses are limited, only a fraction of the information in a visual scene can be represented by phosphenes. Here, we propose a simple yet powerful image processing strategy for recognizing facial expressions with prosthetic vision, supporting communication and social interaction in the blind. A psychophysical study was conducted to investigate whether a landmark-based representation of facial expressions could improve emotion detection with prosthetic vision. Our approach was compared to edge detection, which is commonly used in current retinal prosthetic devices. Additionally, the relationship between the number of phosphenes and accuracy of emotion recognition was studied. The landmark model improved accuracy of emotion recognition, regardless of the number of phosphenes. Secondly, the accuracy improved with an increasing number of phosphenes up to a saturation point. The performance saturated with fewer phosphenes with the landmark model than with edge detection. These results suggest that landmark-based image pre-processing allows for a more efficient use of the limited information that can be stored in a phosphene pattern, providing a route towards more meaningful and higher-quality perceptual experience in subjects with prosthetic vision.
Upload full text
Use your RU credentials (u/z-number and password) to log in with SURFconext to upload a file for processing by the repository team.