Structured LLM Output Storage and Parsing in Python


Generative AI is at present getting used extensively everywhere in the world. The power of the Massive Language Fashions to know the textual content supplied and generate a textual content primarily based on that has led to quite a few purposes from Chatbots to Textual content analyzers. However typically these Massive Language Fashions generate textual content as is, in a non-structured method. Typically we wish the output generated by the LLMs to be in a buildings format, let’s say a JSON (JavaScript Object Notation) format. Let’s say we’re analyzing a social media put up utilizing LLM, and we’d like the output generated by LLM throughout the code itself as a JSON/python variable to carry out another activity. Attaining this with Immediate Engineering is feasible but it surely takes a lot time tinkering with the prompts. To resolve this, LangChain has launched Output Parses, which could be labored with in changing the LLMs output  storage to a structured format.

Studying Goals

  • Decoding the output generated by Massive Language Fashions
  • Creating customized Information Constructions with Pydantic
  • Understanding Immediate Templates’ significance and producing one formatting the Output of LLM
  • Discover ways to create format directions for LLM output with LangChain
  • See how we are able to parse JSON information to a Pydantic Object

This text was revealed as part of the Data Science Blogathon.

What’s LangChain and Output Parsing?

LangChain is a Python Library that permits you to construct purposes with Massive Language Fashions inside no time. It helps all kinds of fashions together with OpenAI GPT LLMs, Google’s PaLM, and even the open-source fashions accessible within the Hugging Face like Falcon, Llama, and lots of extra. With LangChain customising Prompts to the Massive Language Fashions is a breeze and it additionally comes with a vector retailer out of the field, which might retailer the embeddings of inputs and outputs. It thus could be labored with to create purposes that may question any paperwork inside minutes.

LangChain permits Massive Language Fashions to entry info from the web by brokers. It additionally gives output parsers, which permit us to construction the info from the output generated by the Massive Language Fashions. LangChain comes with completely different Output Parses like Record Parser, Datetime Parser, Enum Parser, and so forth. On this article, we’ll look by the JSON parser, which lets us parse the output generated by the LLMs to a JSON format. Beneath we are able to observe a typical stream of how an LLM output is parsed right into a Pydantic Object, thus making a prepared to make use of information in Python variables

Langchain and output parsing | LLM Output Storage

Getting Began – Establishing the Mannequin

On this part, we’ll arrange the mannequin with LangChain. We will probably be utilizing PaLM as our Massive Language Mannequin all through this text. We will probably be utilizing Google Colab for our surroundings. You’ll be able to substitute PaLM with some other Massive Language Mannequin. We’ll begin by first importing the modules required.

!pip set up google-generativeai langchain
  • This may obtain the LangChain library and the google-generativeai library for working with the PaLM mannequin.
  • The langchain library is required to create customized prompts and parse the output generated by the massive language fashions
  • The google-generativeai library will allow us to work together with Google’s PaLM mannequin.


To work with the PaLM, we’ll want an API key, which we are able to get by signing up for the MakerSuite web site. Subsequent, we’ll import all our vital libraries and move within the API Key to instantiate the PaLM mannequin.

import os
import google.generativeai as palm
from langchain.embeddings import GooglePalmEmbeddings
from langchain.llms import GooglePalm

os.environ['GOOGLE_API_KEY']= 'YOUR API KEY'

llm = GooglePalm()
llm.temperature = 0.1

prompts = ["Name 5 planets and line about them"]
llm_result = llm._generate(prompts)
print(llm_result.generations[0][0].textual content)
  • Right here we first created an occasion of the Google PaLM(Pathways Language Mannequin) and assigned it to the variable llm
  • Within the subsequent step, we set the temperature of our mannequin to 0.1, setting it low as a result of we don’t need the mannequin to hallucinate
  • Then we created a Immediate as a listing and handed it to the variable prompts
  • To move the immediate to the PaLM, we name the ._generate() methodology after which move the Immediate checklist to it and the outcomes are saved within the variable llm_result
  • Lastly, we print the consequence within the final step by calling the .generations and changing it to textual content by calling the .textual content methodology

The output for this immediate could be seen beneath

Output | LLM Output Storage

We will see that the Massive Language Mannequin has generated a good output and the LLM additionally tried so as to add some construction to it by including some strains. However what if I need to retailer the data for every mannequin in a variable? What if I need to retailer the planet identify, orbit interval, and distance from the solar, all these individually in a variable? The output generated by the mannequin as is can’t be labored with immediately to realize this. Thus comes the necessity for Output Parses.

Making a Pydantic Output Parser and Immediate Template

On this part, focus on pydantic output parser from langchain. The earlier instance, the output was in an unstructured format. Have a look at how we are able to retailer the data generated by the Massive Language Mannequin in a structured format.

Code Implementation

Let’s begin by wanting on the following code:

from pydantic import BaseModel, Discipline, validator
from langchain.output_parsers import PydanticOutputParser

class PlanetData(BaseModel):
    planet: str = Discipline(description="That is the identify of the planet")
    orbital_period: float = Discipline(description="That is the orbital interval 
    within the variety of earth days")
    distance_from_sun: float = Discipline(description="This can be a float indicating distance 
    from solar in million kilometers")
    interesting_fact: str = Discipline(description="That is about an fascinating truth of 
    the planet")
  • Right here we’re importing the Pydantic Bundle to create a Information Construction. And on this Information Construction, we will probably be storing the output by parsing the output from the LLM.
  • Right here we created a Information Construction utilizing Pydantic known as PlanetData that shops the next information
  • Planet: That is the planet identify which we’ll give as enter to the mannequin
  • Orbit Interval: This can be a float worth that accommodates the orbital interval in Earth days for a specific planet.
  • Distance from Solar: This can be a float indicating the space from a planet to the Solar
  • Attention-grabbing Truth: This can be a string that accommodates one fascinating truth concerning the planet requested

Now, we intention to question the Massive Language Mannequin for details about a planet and retailer all this information within the PlanetData Information Construction by parsing the LLM output. To parse an LLM output right into a Pydantic Information Construction, LangChain gives a parser known as PydanticOutputParser. We move the PlanetData Class to this parser, which could be outlined as follows:

planet_parser = PydanticOutputParser(pydantic_object=PlanetData)

We retailer the parser in a variable named planet_parser. The parser object has a way known as get_format_instructions() which tells the LLM the right way to generate the output. Let’s attempt printing it

from pprint import pp
LLM Output Storage

Within the above, we see that the format directions comprise info on the right way to format the output generated by the LLM. It tells the LLM to output the info in a JSON schema, so this JSON could be parsed to the Pydantic Information Construction. It additionally supplies an instance of an output schema. Subsequent, we’ll create a Immediate Template.

Immediate Template

from langchain import PromptTemplate, LLMChain

template_string = """You're an professional with regards to answering questions 
about planets 
You'll be given a planet identify and you'll output the identify of the planet, 
it is orbital interval in days 
Additionally it is distance from solar in million kilometers and an fascinating truth



planet_prompt = PromptTemplate(
    partial_variables={"format_instructions": planet_parser
  • In our Immediate Template, we inform, that we’ll be giving a planet identify as enter and the LLM has to generate output that features info like Orbit Interval, Distance from Solar, and an fascinating truth concerning the planet
  • Then we assign this template to the PrompTemplate() after which present the enter variable identify to the input_variables parameter, in our case it’s the planet_name
  • We additionally give in-the-format directions that now we have seen earlier than, which inform the LLM the right way to generate the output in a JSON format

Let’s attempt giving in a planet identify and observe how the Immediate appears earlier than being despatched to the Massive Language Mannequin

input_prompt = planet_prompt.format_prompt(planet_name="mercury")
LLM Output Storage

Within the output, we see that the template that now we have outlined seems first with the enter “mercury”. Adopted by which might be the format directions. These format directions comprise the directions that the LLM can use to generate JSON information.

Testing the Massive Language Mannequin

On this part, we’ll ship our enter to the LLM and observe the info generated. Within the earlier part, see how will our enter string be, when despatched to the LLM.

input_prompt = planet_prompt.format_prompt(planet_name="mercury")
output = llm(input_prompt.to_string())
Testing the large language model | LLM Output Storage

We will see the output generated by the Massive Language Mannequin. The output is certainly generated in a JSON format. The JSON information accommodates all of the keys that now we have outlined in our PlanetData Information Construction. And every key has a worth which we anticipate it to have.

Now now we have to parse this JSON information to the Information Construction that now we have accomplished. This may be simply accomplished with the PydanticOutputParser that now we have outlined beforehand. Let’s take a look at that code:

parsed_output = planet_parser.parse(output)
print("Planet: ",parsed_output.planet)
print("Orbital interval: ",parsed_output.orbital_period)
print("Distance From the Solar(in Million KM): ",parsed_output.distance_from_sun)
print("Attention-grabbing Truth: ",parsed_output.interesting_fact)

Calling within the parse() methodology for the planet_parser, will take the output after which parses and converts it to a Pydantic Object, in our case an Object of PlanetData. So the output, i.e. the JSON generated by the Massive Language Mannequin is parsed to the PlannetData Information Construction and we are able to now entry the person information from it. The output for the above will probably be

We see that the key-value pairs from the JSON information had been parsed accurately to the Pydantic Information. Let’s attempt with one other planet and observe the output

input_prompt = planet_prompt.format_prompt(planet_name="venus")
output = llm(input_prompt.to_string())

parsed_output = planet_parser.parse(output)
print("Planet: ",parsed_output.planet)
print("Orbital interval: ",parsed_output.orbital_period)
print("Distance From the Solar: ",parsed_output.distance_from_sun)
print("Attention-grabbing Truth: ",parsed_output.interesting_fact)

We see that for the enter “Venus”, the LLM was in a position to generate a JSON because the output and it was efficiently parsed into Pydantic Information. This fashion, by output parsing, we are able to immediately make the most of the data generated by the Massive Language Fashions

Potential Functions and Use Instances

On this part, we’ll undergo some potential real-world purposes/use circumstances, the place we are able to make use of these output parsing methods. Use Parsing in extraction / after extraction, that’s once we extract any kind of knowledge, we need to parse it in order that the extracted info could be consumed by different purposes. A number of the purposes embrace:

  • Product Grievance Extraction and Evaluation: When a brand new model involves the market and releases its new merchandise, the very first thing it desires to do is verify how the product is performing, and among the finest methods to guage that is to research social media posts of shoppers utilizing these merchandise. Output parsers and LLMs allow the extraction of data, resembling model and product names and even complaints from a shopper’s social media posts. These Massive Language Fashions retailer this information in Pythonic variables by output parsing, permitting you to put it to use for information visualizations.
  • Buyer Assist: When creating chatbots with LLMs for buyer assist, one essential activity will probably be to extract the data from the shopper’s chat historical past. This info accommodates key particulars like what issues the shoppers face with respect to the product/service. You’ll be able to simply extract these particulars utilizing LangChain output parsers as a substitute of making customized code to extract this info
  • Job Posting Info: When creating Job search platforms like Certainly, LinkedIn, and so forth, we are able to use LLMs to extract particulars from job postings, together with job titles, firm names, years of expertise, and job descriptions. Output parsing can save this info as structured JSON information for job matching and proposals. Parsing this info from LLM output immediately by the LangChain Output Parsers removes a lot redundant code wanted to carry out this separate parsing operation.


Massive Language Fashions are nice, as they’ll actually match into each use case attributable to their extraordinary text-generation capabilities. However most frequently they fall quick with regards to really utilizing the output generated, the place now we have to spend a considerable period of time parsing the output. On this article, now we have taken a glance into this drawback and the way we are able to resolve it utilizing the Output Parsers from LangChain, particularly the JSON parser that may parse the JSON information generated from LLM and convert it to a Pydantic Object.

Key Takeaways

A number of the key takeaways from this text embrace:

  • LangChain is a Python Library that may be create purposes with the prevailing Massive Language Fashions.
  • LangChain supplies Output Parsers that allow us parse the output generated by the Massive Language Fashions.
  • Pydantic permits us to outline customized Information Constructions, which can be utilized whereas parsing the output from the LLMs.
  • Other than the Pydantic JSON parser, LangChain additionally supplies completely different Output Parsers just like the Record Parser, Datetime Parser, Enum Parser, and so forth.

Steadily Requested Questions

Q1. What’s JSON?

A. JSON, an acronym for JavaScript Object Notation, is a format for structured information. It accommodates information within the type of key-value pairs.

Q2. What’s Pydantic?

A. Pydantic is a Python library which creates customized information buildings and carry out information validation. It verifies whether or not each bit of knowledge matches the assigned kind, thereby validating the supplied information.

Q3. How can we generate information in JSON format from Massive Language Fashions?

A. Do that with Immediate Engineering, the place tinkering with the Immediate would possibly lead us to make the LLM generate JSON information as output. To ease this course of, LangChain has Output Parsers and you need to use for this activity.

This fall. What are Output Parsers in LangChain?

A. Output Parsers in LangChain enable us to format the output generated by the Massive Language Fashions in a structured method. This lets us simply entry the data from the Massive Language Fashions for different duties.

Q5. What are the completely different output parses does LangChain has?

A. LangChain comes with completely different output parsers like Pydantic Parser, Record Parsr, Enum Parser, Datetime Parser, and so forth.

The media proven on this article will not be owned by Analytics Vidhya and is used on the Creator’s discretion.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button