Is this whole token thing really just the same as paid in-game currencies, just designed to get you to not think about the fact that you’re spending money? Or is there another use for it?
Tokens are an AI context of units of text. Could be a character or a word. Why use the term tokens instead of words? Who knows. Maybe because some units aren’t words? The tech industry delights in creating stupid terms for things that already had terms.
There are to many words to have each be it’s own input, but bundling frequently used snippets is more efficient than using characters.
Then the investors came and decided to measure success by wasted energy and the rest is history.
A token can range from individual letters to several words in a stock phrase, from what little I understand. It’s a piece of whatever they’re doing behind the scenes leaking into the public-facing terminology.
According to my SO, it’s safe enough to just say “a token is a word” as long as you remember this is only an approximation.
Tokens are pretty much the compute used for one word, approximately, or some other information of a similar size. (Long words are two or three tokens each actually)
Then, yes those tokens are paid for with “credits” which is them gamifying the purchase lol
Is this whole token thing really just the same as paid in-game currencies, just designed to get you to not think about the fact that you’re spending money? Or is there another use for it?
Tokens are an AI context of units of text. Could be a character or a word. Why use the term tokens instead of words? Who knows. Maybe because some units aren’t words? The tech industry delights in creating stupid terms for things that already had terms.
Well, yeah. You answered yourself?
There are to many words to have each be it’s own input, but bundling frequently used snippets is more efficient than using characters.
Then the investors came and decided to measure success by wasted energy and the rest is history.
A token can range from individual letters to several words in a stock phrase, from what little I understand. It’s a piece of whatever they’re doing behind the scenes leaking into the public-facing terminology.
According to my SO, it’s safe enough to just say “a token is a word” as long as you remember this is only an approximation.
Tokens are pretty much the compute used for one word, approximately, or some other information of a similar size. (Long words are two or three tokens each actually)
Then, yes those tokens are paid for with “credits” which is them gamifying the purchase lol
Thanks!