An End-to-End Conversational Style Matching Agent

We present an end-to-end voice-based conversational agent that is
able to engage in naturalistic multi-turn dialogue and align with
the interlocutor’s conversational style. The system uses a series of
deep neural network components for speech recognition, dialogue
generation, prosodic analysis and speech synthesis to generate
language and prosodic expression with qualities that match those
of the user. We conducted a user study (N=30) in which participants
talked with the agent for 15 to 20 minutes, resulting in over 8
hours of natural interaction data. Users with high consideration
conversational styles reported the agent to be more trustworthy
when it matched their conversational style. Whereas, users with
high involvement conversational styles were indifferent. Finally,
we provide design guidelines for multi-turn dialogue interactions
using conversational style adaptation.