Inferring interactivity from gaze patterns during triadic person-object-agent interactions

Observing others’ gaze informs us about relevant matters in the environment. Humans’ sensitivity to gaze cues and our ability to use this information to focus our own attention is crucial to learning, social coordination and survival. Gaze can also be a deliberate social signal which captures and directs the gaze of others towards an object of interest.
In the current study, we investigated whether the intention to actively communicate one’s own attentional focus can be inferred from the dynamics of gaze alone. We used a triadic gaze interaction paradigm based on the recently proposed classification of attentional states and respective gaze patterns in person-object-person interactions, the so-called “social gaze space” (SGS). Twenty-eight participants interacted with a computer controlled virtual agent while they assumed to interact with a real human (Fig. 1A). During the experiment, the virtual agent engaged in various gaze patterns (Fig. 1B) which were determined by the agent’s attentional communicative state, as described by the concept of SGS. After each interaction, participants were asked to judge whether the other person was trying to deliberately interact with them.

Jording 2019
Figure 1. Illustration of the technical Setup and the participants’ perspective during the experiment. A: Illustration of a participant interacting with the agent controlled by the platform TriPy. B: The behavior of the agent created by TriPy as seen from the perspective of the participant.

Results show that participants were able to infer the communicative intention solely from the agent’s gaze behavior (Fig. 2). The results substantiate claims about the pivotal role of gaze in social coordination and relationship formation.
Our results further reveal that social expectations are reflected in differential responses to the displayed gaze patterns and may be crucial for impression formation during gaze-based interaction. To the best of our knowledge, this is the first study to document the experience of interactivity in continuous and contingent triadic gaze interactions.

Jording 2019
Figure 2. Illustration of the participants gaze behavior and instances of eye contact and joint attention between participant and agent in connection to the participants rating of the agents interactivity, separately for an agent behaving non-interactively (light blue) vs. interactively (dark blue). A: Boxplots of relative fixation durations as the portion of time spent on the AoIs Eyes, Face and Objects per trial; B: Frequencies of eye contact instances per trial; C: Mean rates (circles and triangles) and model predictions with 95% confidence intervals (lines and ribbons) of interactivity ratings for differing numbers of eye contact instances per trial. D: Frequencies of joint attention instances per trial; E: Mean rates (circles and triangles) and model predictions with 95% confidence intervals (lines and ribbons) of interactivity ratings for differing numbers of joint attention instances per trial.

Publication:

Jording, M., Hartz, A., Bente, G., Schulte-Rüther, M., & Vogeley, K. (2019). Inferring interactivity from gaze patterns during triadic person-object-agent interactions. Frontiers in Psychology, 10, 1913.

Correspondence to:

Mathis Jording

Letzte Änderung: 06.04.2024