Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device

Brown, David J, MacPherson, Tom and Ward, Jamie (2011) Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device. Perception, 40 (9). pp. 1120-1135. ISSN 0301-0066

[img]
Preview
PDF - Published Version
Download (406kB) | Preview

Abstract

Sensory substitution devices convert live visual images into auditory signals, for example with a web camera (to record the images), a computer (to perform the conversion) and headphones (to listen to the sounds). In a series of three experiments, the performance of one such device (‘The vOICe’) was assessed under various conditions on blindfolded sighted participants. The main task that we used involved identifying and locating objects placed on a table by holding a webcam (like a flashlight) or wearing it on the head (like a miner’s light). Identifying objects on a table was easier with a hand-held device, but locating the objects was easier with a head-mounted device. Brightness converted into loudness was less effective than the reverse contrast (dark being loud), suggesting that performance under these conditions (natural indoor lighting, novice users) is related more to the properties of the auditory signal (ie the amount of noise in it) than the cross-modal association between loudness and brightness. Individual differences in musical memory (detecting pitch changes in two sequences of notes) was related to the time taken to identify or recognise objects, but individual differences in self-reported vividness of visual imagery did not reliably predict performance across the experiments. In general, the results suggest that the auditory characteristics of the device may be more important for initial learning than visual associations.

Item Type: Article
Schools and Departments: School of Psychology > Psychology
Depositing User: Tom MacPherson
Date Deposited: 09 Aug 2012 10:45
Last Modified: 07 Mar 2017 06:37
URI: http://sro.sussex.ac.uk/id/eprint/13912

View download statistics for this item

📧 Request an update