How a lot does AI value? It relies on a just a few completely different dimensions, however for a typical use case the place a consumer enters a question & expects a couple of 200 phrase response, the fee varies from $0.03 to $3.60.
Completely different fashions value completely different quantities. Additionally, the dimensions of the context window is a crucial issue.
The context window is a flowery approach of claiming the amount of background info the consumer can present to the AI.
When asking an AI to put in writing a two-paragraph essay within the type of Hemingway, having just a few sentences of his writing gained’t assist a lot. However maybe importing one-third of the Outdated Man & the Sea imbues Ernest’s type throughout the prose – no adverbs, easy direct sentences.
Longer tracts value extra. A 500 phrase essay totals $8.40 with the fanciest model of GPT-4 & $0.07 with Llama2, Fb’s open-source mannequin.
There’s a 120x distinction in value between the highest of the road AI & the free model.
Which is healthier? It relies on the use case & the quantity of optimization one is keen to decide to the mannequin.
These costs aren’t fastened. Earlier this week, OpenAI introduced a 3x discount in prices – a development that can proceed as engineers optimize the chips, the algorithms, & the techniques for AI.
Relative to different infrastructure prices, these charges might be important & understanding the gross-margin affect of this new layer of software program shall be important, & will seemingly push startups from the dearer bigger fashions to extra capital-efficient AI over time.
