Regression of Multiple Conversation Aspects Using Dyadic Physiological Measurements

Dyadic physiological responses are correlated with the quality of interpersonal processes - for example, the degree of "connectedness" in education and mental health counseling. Pattern recognition algorithms could be applied to such dyadic responses to identify the states of specific dyads, but such pattern recognition has primarily focused on classification. This paper instead uses regression algorithms to estimate three conversation aspects (valence, arousal, balance) from heart rate, skin conductance, respiration, and skin temperature. Data were collected from 35 dyads who engaged in 20 minutes of conversation, divided into 10 two-minute intervals. Each interval was rated with regard to conversation valence, arousal, and balance by an observer. When regression algorithms (support vector machines and Gaussian process regression) were trained on other data from the same dyad, they were able to estimate valence, arousal and balance with lower errors than a simple baseline estimator. However, when algorithms were trained on data from other dyads, errors were not lower than those of the baseline estimator. Overall, results indicate that, as long as training data from the same dyad are available, autonomic nervous system responses can be combined with regression algorithms to estimate multiple dyadic conversation aspects with some accuracy. This has applications in education and mental health counseling, though fundamental issues remain to be addressed before the technology is used in practice.