What Should Autonomous Robots Verbalize and What Should They Not?

Empowered by the development of multimodal information processing and large language model-related technologies, robots that can operate autonomously in human living spaces are being realized. When such robots communicate with human users, they need a framework to appropriately select what to say to the user among the recognition results, memories, and decisions obtained from various multimodal environments. In this study, we constructed a framework that enables robots to appropriately select events to be verbalized by calculating the mutual information between the robot’s verbalization texts and its own memory or common sense knowledge. User evaluation results using crowdsourcing suggested that the proposed framework improves the necessity and sufficiency of the robot’s speech. This ability will contribute to improving the usability of the robot.