Hi everyone! Welcome back to the Rasa
Masterclass. I think the time has come for us to talk about what comes after
you built a simple version of your assistant and how you can take it to the
next level by learning from real users. It’s a big topic and we will cover
different aspects of it over the course of the next few episodes. In this one, you
will learn: Why it is important to improve your assistants by learning from real
users? What is the Rasa X and how it empowers you to improve your assistants?
How to set up Rasa X on a server and connect your assistant to it? How to use Rasa X integrated version control? And last but not least how Rasa X
works and what are the most important features you should keep in mind and use
when improving your assistants. We have a lot to cover so let’s get started. When
developing conversational AI assistants one of the main challenges is getting
good training data. Often times it all starts with a developer manually
generating some example conversations to kick-start the development process. Just
like we did with our medicare locator assistant.
However relying on only one or a group of developers to generate the training
data makes it close to impossible to build models that generalize. And here is
why: It’s very difficult to come up with a wide variety of NLU training
examples. And similar for dialogue – you can rely on real users to take the
unanticipated paths but by definition you can’t anticipate those paths yourself.
Also when it comes to generating training data,
humans subconsciously tend to stick to specific patterns in examples they come
up with. And on top of all of this – hypothetical conversations don’t really
reflect the real-world conversations and they may drastically differ from how
real users would talk to your assistant. That’s why you should aim to involve real
users into the development process as soon as possible because the
conversations they have your assistant is the best training data can possibly
get and use to improve your assistant. So when is the best time to start inviting
real users to talk to your assistant? As soon as your assistant is capable of handling the most important happy path stories, it’s the
best time to start getting real-world feedback and use it to steer the
direction of the development. Take our Medicare Locator assistant as an example.
It’s still quite a simple application but it’s good enough to collect some
required information and run specific actions to fulfill users’ requests. Getting
some real user feedback would help us immensely to take this assistant to the
next level – improve the models using the real-world data, add new skills or maybe
introduce some changes to the existing ones. The best way to achieve all that is
by introducing Rasa X into our development workflows. So what is Rasa X? Rasa X is a toolset which allows you to improve the assistants built with
Rasa open source. It enables you to easily share your assistants with real
users, collect the conversations they have your assistant, then review them and
decide on which minor, major or architectural improvements you should
work on. Since Rasa X is a toolset for improving existing assistants rather
than building them from scratch, the only prerequisite for using Rasa X is an
existing assistant built with Rasa Open Source. Let’s start with setting up the
Rasa X. For running Rasa X on a server Rasa supports docker-compose but you
can also find Helm Charts to deploy Rasa X on Open Shift or Kubernetes. We
recommend deploying Rasa X on Open Shift or Kubernetes if you are planning
to ship your application into production because they allow you to set up the
CI/CD. For smaller applications using docker-compose is a
great way to go. In this video we will show you how to set up projects using
docker-compose and we’ll cover the setup using Helm Charts on Kubernetes in
upcoming videos or the Masterclass. You can choose what server you want to
deploy Rasa X on – for example Google Could, AWS or your own server but make
sure that it complies with the minimum suggested requirements as shown in
Rasa’s documentation. It’s recommended to use a Linux operating system to set
up the Rasa X but you can also install rasa X on other operating
systems as long as Docker support is available. We will install Rasa X on a
Google Cloud Compute Engine. So first let’s set up the virtual
machine instance. Let’s call our instance rasax–server. We can change the region
closer to where we are based which in our case is Belgium. Let’s choose the
machine type and pick the one with two CPUs to meet at least the minimum
requirements. Next let’s specify the boot disk. There is a variety of different
Linux distributions you can use. We will select Ubuntu 16.04 and increase the disk
size to 100 gigabytes. Let’s allow the HTTP and HTTPS traffic to allow the
traffic from outside of our network and hit create.
Once the instance is set up, we can login to our instance and start the Rasa X
installation. We will connect to our instance using the gcloud command so if
you want to follow, make sure to install Google Cloud SDK on your machine. So
let’s go to the terminal on our local machine and connect to the instance. Perfect! Now we are ready to set up
Rasa X on our server. At this moment our instance is quite
empty so let’s download and install Rasa X using the following command
Here 0.23.3 refers to the Rasa X version to be installed. You can specify
this, but we recommend to install the latest version of the Rasa X which at
the point of making this video is 0.23.3. The installation process usually
takes a few minutes and you will be asked to agree with terms and conditions
of using the Rasa X. By default the files will be installed in the etc/rasa/
directory on your server (this can be customized as well). Those files consist
of everything that rasa X needs to run properly and allow you to build great
assistants. So for example, those files are things like: credentials, models
directory, database which will be used to store the conversations between the real
users and your assistant, logs where you can find the logs of the container,
docker-compose file, and some others. So now, let’s start Rasa X. Rhis will also
take a few minutes. Once the container is up and running you
should set the password for the admin user of the Rasa X instance. And
that’s it! We are ready to open up the Rasa X, connect our assistant to it and
work on the improvements. To do that, let’s open our server using the provided
IP address. For now, we will have to change our URL to http. Let’s type in the
password we created for the admin user and login to the Rasa
X UI. Rasa X comes with an easy-to-use UI which allows technical and
non-technical people to work on the improvements of the AI assistant. Before
diving into the usage of the Rasa X let’s first connect our assistant to the
Rasa X server. Rasa X comes with an integrated version control feature which
allows you to easily connect your assistant to rasa X and version control
your assistant on GitHub (or any other remote git server). For that, we have to make sure that we have our
assistant on the remote GitHub repository with all the data and
configuration files just like we had on our local machine. Once the assistant is
on GitHub, we can enable version control feature on Rasa X. At the moment, it’s an
experimental feature but you can already test it out by navigating to your user’s
icon, selecting Experimental features and enabling integrated version control. Once
enabled, you should see the icon of this feature appearing next to your user’s
icon. Integrated version control is a usual workflow in software engineering
which allows developers to version the data and action code on Git,
review the changes before they go into production and run integrated tests on
the proposed changes. It’s the workflow which will recommend Rasa developers
to follow as well when building their conversational AI assistants. With Rasa’s integrated version control you can: Automate data synchronization with
your Git repository. Autoamtically stay up to date with the latest version
of your remote repository on Git server and annotate new data and push the changes
to Git. In addition to that, Rasa’s integrated version control makes it
easier to connect an existing assistant to the Rasa server and work on the
improvements. The main requirement for using Rasa’s integrated version control
with an existing assistant is the project structure. It should be very
similar to what you get when you run rasa init function: data files should
be placed in a data directory while files like domain and config should be
present and placed in the main directory of your project. Once that’s cleared,
lets authenticate our server. Here is how it’s done. First let’s generate the
SSL key for our remote repository using the following command. This command will
generate a private and a public key. We have to provide the public key to our
assistant’s GitHub repo to authenticate it with Rasa X. Let’s copy the key stored
in a git-deploy-key.pub file (which corresponds to a public key) and paste it
in the Deploy Keys section of the GitHub repository settings. Let’s not forget to
give the key write permissions. Once that’s done we have to add our
repository to Rasa X. Here’s how it’s done. We have to create a repository.json file with the following parameters: repository_url is a SSH URL to our
remote repository. Tt can be found once you click Clone or Download your GitHub
repository and choose use SSH option. ssh_key is the private key which we just
generated. It’s a long key and remember that you have to provide a full key
including the lines “BEGIN RSA PRIVATE KEY” and “END RSA PRIVATE KEY”. target_branch is a branch which we will point our Rasa X server to and this parameter
can be “master”. Let’s update all details to fit our case and save the file. Once
that’s done we have to verify the integration by running the following
command which is provided in the Rasa documentation. It will send a request to
our Rasa X server and create the repository. To run this command we need
to provide our server host URL (which in our case will be the server IP) and an
API token. The easiest way to find the API token is to navigate to Rasa X Models tab and click on Upload Model button. Here you will find the API token
which you can copy and provide in this command. Here is how the command with the
updated details look like for our assistant. If the authentication is
successful, Rasa X will pull the current version of the files we have in
our assistant’s repository. We can check it by opening the Rasa
X UI. At the bottom left we can now see that GitHub integration icon is
suggesting that our Rasa X instance is connected to the master branch of our
assistant’s GitHub repository. There is one more thing we have to configure. Tt’s
our assistant’s custom action server. The code of custom actions have to be placed
in the actions directory of the server. For that reason, we will create a
directory called “actions” with the file actions.py
where we will paste the code of our custom action. After saving it, we have to
create a docker-compose.override.yml file to let the docker-compose know that
in addition to Rasa X, there should be a custom action server spinned up. The
parameters here mean that we are using Rasa SDK image to run the custom action,
that the action server will listen to a port 5055 and that it depends on the
rasa-production service responsible for running the training model used for
parsing intent messages and predicting actions in conversations with users over
the input channel or Rasa X UI. After saving the file we can spin up
the docker container once again and have our assistant fully functioning on Rasa
X server. So let’s have a look! Rasa X UI consists of four main tabs: “Talk to your
bot” where you can test your assistant and improve it by talking to it in an
interactive learning session. “Conversations” where you will find the
conversations that real users have your assistant. “Models” where you will find and
manage the models. And “Training” where you can find all the current files of your
assistant including training data, domain, templates, configuration and you can also
annotate new training examples, and last but not least – train new models after
making certain improvements. Currently our Rasa x instance contains all the
files it pulled from the GitHub repository. Let’s hit the train
button and train the model with the current training data and configurations.
Once the model is trained we will find it in a “Models” tab and we can select it as
an active production model. We can keep on improving the model just by talking
to our assistant. In “Talk to your bot” tab of the Rasa
X UI, we can talk to our assistant in two modes. A simple chat mode means that we
can test the performance of the assistant just by simply having a
conversation. It’s a nice way to test the assistant and at the same time generate
some NLU and stories training examples. An interactive learning mode means that
while we talk to our assistant, it will ask us to validate the intent
classification action prediction at each step. This way, we can keep on training
our assistant and generate new NLU training examples as well as stories for
our dialogue management model. In the “Training” tab of the Rasa X UI we can
find lots of very important sections of the Rasa X. NLU training section
contains the training examples of the NLU model. If we click on “Training data”
tab we will find all NLU inputs we initially created as well as all inputs
we sent to our assistant during the interactive learning session. We can keep
the predictions here or we can change them to whatever we think fits better. If
we head to “Annotate new data” section we can create new NLU training examples.
Rasa X makes it easier by suggesting the most probable intent label to those
inputs already. We can annotate those examples and use them when training new
NLU models. And what about the stories? We can find them in “Stories”
section of the “Training” tab. Next to each story we will find a visualization of
the dialogue flow covered by that story. It’s a very nice way to see the visual
representation of what dialogue turns the story covers. We can edit every story
by simply making the changes here. We can also write new stories, import stories in
bulk or generate more training data points by talking to our
in an interactive learning mode. In the “Training” tab we can find all the other
files necessary for our assistant to run: like model configurations, domain,
response templates and we can make changes to all of them here. Most
importantly, we can train new models with the new data points and any other
changes we made in the configuration. To do that all we have to do is hit the
Train button. In a few seconds a new model is trained and we should be ready
to test. We can find the new model in the model section of the Rasa
X UI. Here we can choose the freshly trained model as an active production model and
we can test it again by heading back to “Talk to your bot” section of the Rasa X
UI. Usually this process of building a simple assistant, then improving it by
talking to it in an interactive learning session, training new models and testing
it again should repeat it until you feel confident enough to share your bot with
real testers. Those real testers can be your colleagues, a specific group of
people picked for a user test or pretty much anyone you would like to test your
assistant. The feedback you will get and the conversations you will collect
between your assistant and those users will help you to make immediate
improvements on your assistant. We will cover the entire process of sharing your
assistant with real testers in the next episode of the Rasa Masterclass. So stay
tuned for that and I’ll see you then.