Configuring Nemo-Guardrails Your Means: An Various Methodology for Giant Language Fashions | by Masatake Hirono | Sep, 2023
The center of Nemo-Guardrails lies in its configuration information, sometimes in .yml format. These information help you specify which LLM to make use of, what sort of habits you anticipate from it, and the way it ought to work together with different companies. For instance, a easy configuration may seem like this:
fashions:
- kind: foremost
engine: openai
mannequin: text-davinci-003
This configuration specifies that the OpenAI’s text-davinci-003 mannequin must be used as the principle LLM. The .yml information are extremely customizable, permitting you to outline varied varieties of guardrails, actions, and even hook up with totally different LLM suppliers.
Whereas .yml information are a handy and easy method to configure your LLMs, they aren’t the one possibility. That is notably related in the event you’re eager about utilizing LLM suppliers aside from OpenAI, reminiscent of Azure. Some customers have reported challenges when making an attempt to configure these suppliers utilizing .yml information alone.
One various is to leverage LangChain’s Chat mannequin. This strategy means that you can instantly move the LLM configuration to Nemo-Guardrails. It’s particularly helpful for individuals who want to use LLM suppliers that will not but be absolutely supported in .yml configurations. For instance, in the event you’re utilizing Azure as your LLM supplier, LangChain’s Chat mannequin gives a method to combine it seamlessly.
Now that you’ve got a foundational understanding of Nemo-Guardrails and its capabilities, you’re well-prepared for the following part. The upcoming tutorial will solely deal with various strategies for configuring your LLM, notably helpful for individuals who wish to use suppliers, reminiscent of Azure. This can give you a extra versatile and probably superior setup to your conversational fashions.
On this tutorial, we’ll present you an alternate method to arrange your chatbot, which is especially helpful in the event you’re not utilizing OpenAI. We’ll deal with constructing a chatbot for an insurance coverage buyer help heart that retains the dialog centered on insurance coverage subjects.
If you happen to haven’t already put in the NeMo-Guardrails toolkit, please check with the official installation guide.
Vital Notice: No matter whether or not your machine is GPU-enabled, keep away from utilizing torch model 2.0.1. This model has identified issues with Nemo-Guardrails attributable to its lack of dependency on CUDA libraries, probably inflicting a ValueError associated to libnvrtc.so.
Create a brand new folder to your venture, naming it ins_assistant. Inside this folder, create one other folder known as config.
Your folder construction ought to seem like this:
ins_assistant
└── config
In conventional setups, you’d specify the LLM mannequin instantly within the config.yml file. Nevertheless, on this various strategy, you don’t must specify the mannequin. If you want to make use of the context to information the chatbot’s habits, you are able to do so.
Create a brand new config.yml file inside your config folder and add the next line:
directions:
- kind: common
content material: |
You're an AI assistant that helps staff at an insurance coverage firm's buyer help heart.
By doing this, you’re setting the stage for the chatbot, instructing it to deal with insurance-related buyer help.
Create a brand new file beneath the config folder and identify it off-topic.co. That is the place you’ll outline the canonical varieties and dialog flows which are particular to your insurance coverage buyer help heart chatbot.
Add the next content material to off-topic.co:
outline consumer ask off matter
"How's the climate at this time?"
"Are you able to advocate a very good restaurant close by?"
"What's your opinion on the most recent political information?"
"How do I cook dinner spaghetti?"
"What are one of the best vacationer sights in Paris?"outline bot clarify cant off matter
"I can not reply to your query as a result of I am programmed to help solely with insurance-related questions."
outline circulate
consumer ask off matter
bot clarify cant off matter
Be happy so as to add extra pattern off-topic inquiries to the consumer ask off matter canonical type in the event you’d wish to make the chatbot extra strong in dealing with quite a lot of off-topic queries.
Navigate again to the ins_assistant folder and create a brand new Python file named cli_chat.py. This script will allow you to work together along with your chatbot through the CLI.
Right here’s a pattern code snippet for cli_chat.py:
import os
from langchain.chat_models import AzureChatOpenAI
from nemoguardrails import LLMRails, RailsConfig# Studying setting variables
azure_openai_key = os.environ.get("AZURE_OPENAI_KEY")
azure_openai_model = os.environ.get("AZURE_OPENAI_MODEL")
azure_openai_endpoint = os.environ.get("AZURE_OPENAI_ENDPOINT")
# Outline LLM and parameters to move to the guardrails configuration
chat_model = AzureChatOpenAI(
openai_api_type="azure",
openai_api_version="2023-03-15-preview",
openai_api_key=azure_openai_key,
deployment_name=azure_openai_model,
openai_api_base=azure_openai_endpoint
)
# Load configuration
config = RailsConfig.from_path("./config")
# Configuration of LLMs is handed
app = LLMRails(config=config, llm=chat_model)
# pattern consumer enter
new_message = app.generate(messages=[{
"role": "user",
"content": "What's the latest fashion trend?"
}])
print(f"new_message: {new_message}")
To work together along with your chatbot, open your terminal, navigate to the ins_assistant folder, and run:
python cli_chat.py
You need to see the chatbot’s responses within the terminal, guiding any off-topic conversations again to insurance-related subjects.
Be happy to edit the content material in new_message to move totally different consumer inputs to the LLM. Get pleasure from experimenting to see how the chatbot responds to varied queries!
As we’ve seen, guardrails supply a strong method to make LLMs safer, extra dependable, and extra moral. Whereas .yml information present an easy methodology for configuration, various approaches just like the one demonstrated on this tutorial supply higher flexibility, particularly for these utilizing LLM suppliers aside from OpenAI.
Whether or not you’re constructing a chatbot for buyer help in an insurance coverage firm or every other specialised software, understanding the right way to successfully implement guardrails is essential. With instruments like Nemo-Guardrails, attaining this has by no means been simpler.
Thanks for becoming a member of me on this deep dive into the world of LLM guardrails. Glad coding!
Masatake Hirono is an information scientist based mostly in Tokyo, Japan. His various skilled expertise consists of roles at world consulting companies the place he specialised in superior analytics. Masatake has led quite a lot of tasks, from ML-driven demand forecasting to the event of recommender engines.
He holds a Grasp’s Diploma in Increased Training Institutional Analysis from the College of Michigan, Ann Arbor. His ability set encompasses Econometrics, machine studying, and causal inference, and he’s proficient in Python, R, and SQL, amongst different instruments.
