^April May – 2009 ^June

Affective computing is a field of AI and HCI that aims to incorporate the behavioural cues of affective states (emotions, mental states, moods and attitudes) that are common in human-human communication into human-computer and human-robot interfaces, in order to improve the usability and performance of these interfaces. The talk will focus on recognition of subtle affective states from their non-verbal expressions in speech, their analysis and implementation to HCI. Temporal abstractions of paralinguistic speech events, borrowed from various disciplines such as musicology, engineering and linguistics, were extracted from speech signals and used as attributes for the recognition. The recognition was based on a novel multiclass and semi-blind multi-label inference system which was designed to infer co-occurring affective states. The system was used for analysis of expressions during sustained human-computer interaction. The results were validated through correlation to various indicators (multi-modal analysis) and comparison to human performance. Inference results of over 500 different affective states allowed analysis of the relations between these affective states, serving as a tool for verification of taxonomies. The implication is that the system and its architecture can be generalised to new affective states, speakers, languages and scenarios with no additional training.