Testing Conversational Interfaces

Share on FacebookTweet about this on TwitterShare on Google+Share on LinkedInEmail this to someone

Usability Testing on Chatbots

Chatbots are a great innovation and certainly an indicator as to where the industry is heading. But they are also in their relative infancy and provide plenty of scope for errors – and even full-blown disasters – to occur in their deployment and interaction with users. There have been several high-profile and highly publicised incidents of chatbots going wrong in recent months; one to the extent that the chatbot was withdrawn from service. So, usability testing before putting your offering into the public domain is a critical factor in development.

usability testing chatbots

But how do you go about this usability testing and what sorts of issues do you need to test for?

There are several key factors in how well a chatbot works and you need to be acutely aware of these when designing and testing your prototype:

  • Understanding of Context: Your bot needs to appreciate the environment from which the question is coming and consider user expectations and needs. The bot should be able to construct meaning from sometimes confusing sentences or requests by looking at the context of the conversation.
    You can test for this by beginning a conversation and then throwing in some slightly offkey questions or potentially confusing words and see how the bot deals with them. If you are constantly getting an inability to respond or nonsense replies, then the bot’s contextual understanding is insufficient.

  • Awareness of User: Your bot needs to be aware of whom it is speaking to and frame its language and responses accordingly. A bot is likely to be used by several people even in the same location and it needs to be able to recognise an user and deal with them appropriately. You wouldn’t expect a child to be addressed in the same way as an adult for example. The bot has therefore to be able to establish identity from previous inputs and realise the wants and characteristics of the person it is speaking to.
  • Intelligence: Finally a bot has to have the required level of artificial intelligence to process, understand and respond accurately to requests. The ability to learn and adapt will; be crucial in providing a service that doesn’t frustrate users. Usability testing on relevant target groups can help determine whether your bot is likely to be helpful or simply a nuisance and give you some hints on where it needs to be enhanced and smoothed to make it more acceptable.

This is an area with many potential problems but it can provide great dividends if executed properly. If you are interested in finding out more about designing and usability testing chatbot conversations, why not give us a ring on +44(0)800 024624 or email us at info@usability247.com for a conversation of our own!

Leave a comment

*