The R3D3 project combined research in the areas of computer vision, data processing, and spoken language interaction in a real-life context. It also addressed practical challenges in robot development.  Inspired by Star Wars’ R2D2, our R3D3 takes the form of a Receptionist Robot that can have an intelligent dialogue with the visitors to a museum, shop or office building. These dialogues are ‘double’ in the sense that the user (or a group of users) interacts with both the robot and a virtual character shown on the screen carried by the robot. The virtual character speaks, while the robot interacts nonverbally through head movements, gaze and emotions.

For an overview of the project results, see this blog post by R3D3 project partner VicarVision. R3D3’s interaction capabilities are shown in the video below.

Involved parties are the departments of Human Media Interaction and Robotics and Mechatronics of the University of Twente, the Dutch Police Academy, VicarVision, Sentient, NEMO Science Museum and smartrobot.solutions.

Contact: Mariët Theune, m.theune@utwente.nl

Publications

• J. Linssen, M. Berkhoff, M. Bode, E. Rens, M. Theune, and D. Wiltenburg (2017). You can leave your head on – attention management and turn-taking in multi-party interaction with a virtual human/robot duo. In Proceedings of Intelligent Virtual Agents (IVA 2017).
• J. Linssen and M. Theune (2017). R3D3: the Rolling Receptionist Robot with Double Dutch Dialogue. In Proceedings of the Companion of HRI ’17, pp. 189–190.
• M. Theune, D. Wiltenburg, M. Bode, and J. Linssen (2017). R3D3 in the Wild: Using A Robot for Turn Management in Multi-Party Interaction with a Virtual Human. In Proceedings of the IVA 2017 workshop on Interaction with Agents and Robots: Different Embodiments, Common Challenges.