How A lot Does it Price to Use an LLM? by @ttunguz

How a lot does AI value? It relies on a just a few completely different dimensions, however for a typical use case the place a consumer enters a question & expects a couple of 200 phrase response, the fee varies from $0.03 to $3.60.

Completely different fashions value completely different quantities. Additionally, the dimensions of the context window is a crucial issue.

The context window is a flowery approach of claiming the amount of background info the consumer can present to the AI.

When asking an AI to put in writing a two-paragraph essay within the type of Hemingway, having just a few sentences of his writing gained’t assist a lot. However maybe importing one-third of the Outdated Man & the Sea imbues Ernest’s type throughout the prose – no adverbs, easy direct sentences.

Longer tracts value extra. A 500 phrase essay totals $8.40 with the fanciest model of GPT-4 & $0.07 with Llama2, Fb’s open-source mannequin.

There’s a 120x distinction in value between the highest of the road AI & the free model.

Which is healthier? It relies on the use case & the quantity of optimization one is keen to decide to the mannequin.

These costs aren’t fastened. Earlier this week, OpenAI introduced a 3x discount in prices – a development that can proceed as engineers optimize the chips, the algorithms, & the techniques for AI.

Relative to different infrastructure prices, these charges might be important & understanding the gross-margin affect of this new layer of software program shall be important, & will seemingly push startups from the dearer bigger fashions to extra capital-efficient AI over time.

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button