What Are Hugging Face Brokers?. A step-by-step tutorial on how you can use… | by Farzad Mahmoodinobar | Could, 2023

6.2. Query Answering

Query answering is self-explanatory however let’s make it extra fascinating. As an alternative of offering a paragraph to the agent and asking questions concerning the supplied info, let’s present a picture and ask the agent concerning the content material of the picture.

I wrote just a few strains in a phrase doc after which saved it as a *.jpg picture in my native. Let’s first use PIL to see the picture as follows:

from PIL import Picture
picture = Picture.open('/content material/jack.jpg')
picture.present()

Outcomes:

As you may see, the picture has just a few sentences about my imaginary buddy in Seattle, named Jack. Subsequent, let’s ask a query from our agent and see the way it responds. I wish to ask the agent about Jack’s favourite colour, as follows:

process = "within the following 'doc', what's Jack's favourite colour?"
agent_starcoder.run(process, doc=picture)

Outcomes:

As soon as once more, let’s take a look at the final sentence within the backside — That’s fairly good! We will see that textual content extraction isn’t good, for instance, it extracted Tesla to ‘tesia’ however nonetheless, the agent returned the related portion of the picture, which answered our query.

Let’s see what instruments precisely the agent used:

process = "within the following 'doc', what's Jack's favourite colour?"
agent_starcoder.run(process, doc=picture, return_code=True)

Outcomes: