notice

This is unreleased documentation for Rasa Documentation Main/Unreleased version.
For the latest released documentation, see the latest version (3.x).

Version: Main/Unreleased

Customizing LLM based Components

Rasa Pro Only
Rasa Labs
Rasa Labs access - New in 3.7.0b1

Rasa Labs features are experimental. We introduce experimental features to co-create with our customers. To find out more about how to participate in our Labs program visit our Rasa Labs page.

We are continuously improving Rasa Labs features based on customer feedback. To benefit from the latest bug fixes and feature improvements, please install the latest pre-release using:

pip install 'rasa-plus>3.6' --pre --upgrade

The LLM components can be extended and modified with custom versions. This allows you to customize the behavior of the LLM components to your needs and experiment with different algorithms.

Customizing a component

The LLM components are implemented as a set of classes that can be extended and modified. The following example shows how to extend the LLMIntentClassifier component to add a custom behavior.

For example, we can change the logic that selects the intent labels that are included in the prompt to the LLM model. By default, we only include a selection of the available intents in the prompt. But we can also include all available intents in the prompt. This can be done by extending the LLMIntentClassifier class and overriding the select_intent_examples method:

from rasa_plus.ml import LLMIntentClassifier
class CustomLLMIntentClassifier(LLMIntentClassifier):
def select_intent_examples(
self, message: Message, few_shot_examples: List[Document]
) -> List[str]:
"""Selects the intent examples to use for the LLM training.
Args:
message: The message to classify.
few_shot_examples: The few shot examples to use for the LLM training.
Returns:
The list of intent examples to use for the LLM training.
"""
# use all available intents for the LLM prompt
return list(self.available_intents)

The custom component can then be used in the Rasa configuration file:

config.yml
pipeline:
- name: CustomLLMIntentClassifier
# ...

To reference a component in the Rasa configuration file, you need to use the full name of the component class. The full name of the component class is <module>.<class>.

All components are well documented in their source code. The code can be found in your local installation of the rasa_plus python package.

Common functions to be overridden

Below is a list of functions that could be overwritten to customize the LLM components:

LLMIntentClassifier

select_intent_examples

Selects the intent examples to use for the LLM prompt. The selected intent labels are included in the generation prompt. By default, only the intent labels that are used in the few shot examples are included in the prompt.

def select_intent_examples(
self, message: Message, few_shot_examples: List[Document]
) -> List[str]:
"""Returns the intents that are used in the classification prompt.
The intents are included in the prompt to help the LLM to generate the
correct intent. The selected intents can be based on the message or on
the few shot examples which are also included in the prompt.
Including all intents can lead to a very long prompt which will lead
to higher costs and longer response times. In addition, the LLM might
not be able to generate the correct intent if there are too many intents
in the prompt as we can't include an example for every intent. The
classification would in this case just be based on the intent name.
Args:
message: The message to classify.
few_shot_examples: The few shot examples that can be used in the prompt.
Returns:
The intents that are used in the classification prompt.
"""

closest_intent_from_training_data

The LLM generates an intent label which might not always be part of the domain. This function can be used to map the generated intent label to an intent label that is part of the domain.

The default implementation embedds the generated intent label and all intent labels from the domain and returns the closest intent label from the domain.

def closest_intent_from_training_data(self, generated_intent: str) -> Optional[str]:
"""Returns the closest intent from the training data.
Args:
generated_intent: the intent that was generated by the LLM
Returns:
the closest intent from the training data.
"""

select_few_shot_examples

Selects the NLU training examples that are included in the LLM prompt. The selected examples are included in the prompt to help the LLM to generate the correct intent. By default, the most similar training examples are selected. The selection is based on the message that should be classified. The most similar examples are selected by embedding the incoming message, all training examples and doing a similarity search.

def select_few_shot_examples(self, message: Message) -> List[Document]:
"""Selects the few shot examples that should be used for the LLM prompt.
The examples are included in the classification prompt to help the LLM
to generate the correct intent. Since only a few examples are included
in the prompt, we need to select the most relevant ones.
Args:
message: the message to find the closest examples for
Returns:
the closest examples from the embedded training data
"""

LLMResponseRephraser

rephrase

Rephrases the response generated by the LLM. The default implementation rephrases the response by prompting an LLM to generate a response based on the incoming message and the generated response. The generated response is then replaced with the generated response.

def rephrase(
self,
response: Dict[str, Any],
tracker: DialogueStateTracker,
) -> Dict[str, Any]:
"""Predicts a variation of the response.
Args:
response: The response to rephrase.
tracker: The tracker to use for the prediction.
model_name: The name of the model to use for the prediction.
Returns:
The response with the rephrased text.
"""

IntentlessPolicy

select_response_examples

Samples responses that fit the current conversation. The default implementation samples responses from the domain that fit the current conversation. The selection is based on the conversation history, the history will be embedded and the most similar responses will be selected.

def select_response_examples(
self,
history: str,
number_of_samples: int,
max_number_of_tokens: int,
) -> List[str]:
"""Samples responses that fit the current conversation.
Args:
history: The conversation history.
policy_model: The policy model.
number_of_samples: The number of samples to return.
max_number_of_tokens: Maximum number of tokens for responses.
Returns:
The sampled conversation in order of score decrease.
"""

select_few_shot_conversations

Samples conversations from the training data. The default implementation samples conversations from the training data that fit the current conversation. The selection is based on the conversation history, the history will be embedded and the most similar conversations will be selected.

def select_few_shot_conversations(
self,
history: str,
number_of_samples: int,
max_number_of_tokens: int,
) -> List[str]:
"""Samples conversations from the given conversation samples.
Excludes conversations without AI replies
Args:
history: The conversation history.
number_of_samples: The number of samples to return.
max_number_of_tokens: Maximum number of tokens for conversations.
Returns:
The sampled conversation ordered by similarity decrease.
"""