Token
AI
A token is the basic unit of text that an AI model processes. A token can be a whole word, part of a word, or punctuation, depending on how the model breaks text into smaller pieces.
AI models convert text into tokens so they can process it. The number of tokens a model can work with at one time is called its context window. See more here.
When a marketer asks an AI tool to write ad copy, both the prompt and the response are measured in tokens. The more tokens used, the more information the AI can process, which can also affect cost.
Tokens determine how much information an AI model can process at once, influencing response length, performance, and, for many AI services, pricing.