When one sees what the other hears: Crossmodal attentional modulation for gazed and non-gazed upon auditory targets

Abstract

Three experiments investigated the nature Of visuo-auditory crossmodal cueing in a triadic setting participants had to detect all auditory signal while observing another agent's head facing one of the two laterally positioned auditory sources Experiment I showed that when the agent's eyes were open, sounds originating on the side of the agent's gaze were detected faster than sounds originating on the side Of the agent's visible ear, when the agent's eyes were closed this pat-tern of responses was reversed. Two additional experiments showed that the results were sensitive to whether participants could infer a hearing function oil the part of the agent When no ear was depicted oil the agent, only a gaze-side advantage was observed (Experiment 2), but when the agent's ear was covered (Experiment 3), all ear side advantage was observed only when hearing could still be inferred (i e, wearing the hat) but not when hearing was inferred to be diminished (i e, wearing a helmet) The findings are discussed in the context of inferential and simulation processes and joint attention mechanisms.