Achieving better business results by tuning your assistant in Rasa X/Enterprise

If you’ve ever built a conversational AI assistant, I’m sure not many would disagree when we say it’s hard. Surprisingly that hard part is often not what you might expect; in talking with many Rasa customers and users over the years, they often tell us the same things that building the prototype is the easy part. The real challenge comes when you want to go from a prototype to something you want to ship and put in front of your customers/users and ensure a best-in-class brand experience.

Hearing about these challenges led to the creation of Rasa’s methodology Conversation Driven Development (CDD). You can read more about it here in this blog post. This methodology has two main goals:

1. Help us all build better conversational AI assistants
2. Save newcomers to Rasa from having to learn the hard way how to build better assistants

Let's take a moment to look at the key principles that make up Conversation Driven Development:

• Prototyping & Early Feedback - Conversations are valuable data. Learn what users say to the assistant and how they interact with your assistant.
• Continued iteration of your assistant - Turning real messages into training examples to improve your assistant further

Adopting an enterprise-focused solution built and maintained by industry experts is something worth considering. To that end, allow me to introduce you to Rasa X/Enterprise: A no-code solution for multidisciplinary teams that will help you get more out of your conversational AI Assistant. With Rasa X/Enterprise your teams can:

• Share the assistant at various stages of development with real users
• Collect and review conversations

Sharing your assistant

Assistants cannot be built in a vacuum and internal users cannot accurately reflect the experiences of real customers, so sharing your assistant as early as possible is one of the most recommended steps we at Rasa encourage everyone to do. The moment your assistant can handle the "happy path" for your key use cases, consider sharing it with internal users to gather insights about how people interact with the Assistant. Learnings from the journey outside of the happy path are key to making improvements and more resilient AI Assistants.

Within the Rasa X/Enterprise UI generating a URL is a very straightforward process that allows you to provide your test users direct access to your AI assistant for you to start collecting feedback on the performance of your AI assistant.

Reviewing Stories & Conversations

Tagging conversations is a meaningful way to keep track of conversations in your assistant and as usage of your assistant grows this will be considered essential not only will this make annotation easier but also could be used for the division of labour between new and experienced members of your team.

View larger version

Alternatively, flag conversations as you are reviewing them, its a great way to highlight important issues without distracting you from the work you are doing.

In the early development of your assistant creating/maintaining/managing tags manually is a great way to start. As your assistant continues to grow and you begin to review conversations on a regular basis you will want to take advantage of the APIs available within Rasa X/Enterprise is a great method to simplify and accelerate your processes.

Here is an example repo in GitHub to provide some direction on how you might approach this but there are a few of ways you could approach this. For example this particular script can be used to automatically tag conversations based on your own logic, e.g.

• tag all conversations where slot CUSTOMER_IS_ANGRY is filled, so you can easily filter those conversations in Rasa X and review them

The Rasa X/Enterprise UI also provides you the ability to view ALL conversations your assistant has ever had, additionally, you’ll also have access to advanced filtering capabilities to look at conversations that have the most relevance to you. Whether that is looking at conversations that received low-confidence predictions or conversations that involved a specific user or intent. The UI is specifically designed to help you find what you need and act accordingly to improve your assistant's performance.

Annotation

Annotation is the process of adding context to conversations to aid in the training of your assistant, through it you could for instance label customer conversations as having positive/negative sentiments. It is a crucial step in creating a great customer experience and an area that you should not neglect.

It begins after you have already shared your assistant with some test/real users and you have actual conversations to work with. It’s how you ensure it continues to evolve to better serve your end users, make better predictions and provide accurate more complete responses to customer queries.

You can already train your assistant using the command line but Rasa X/Enterprise provides an intuitive interface to simplify this critical step in your assistant's evolution.

The ‘Training’ page of Rasa X/Enterprise allows you to manage your assistant's training, manage or create new training examples as well as train your models. Through it you can view conversations held with your bot and see how intents were classified for each utterance and where needed it allows you to make the appropriate adjustments.

Take advantage of the UI’s Stories page to create and visualise your stories and see the various directions the stories you have written can take your end users.

Here at Rasa, we help you build assistants your customer will love, with the technology loved by conversational teams. Why not select a solution that not only meets your needs today but also the needs of tomorrow.

If you want to know more about how we at Rasa use Rasa X/Enterprise, get in contact and let us share our knowledge and expertise with you.