Glossary · Technical
Token
The basic unit of text a model reads and generates — roughly a word, part of a word, or a punctuation mark — not a full word or a single character.
What it is
Models don't process raw letters or whole words; they process tokens, which are chunks of text produced by a tokenizer. A common word might be one token, while a rarer word or name can be split into two or three.
Why it matters for AI visibility
Context windows, usage costs, and generation length are all measured in tokens, not words or characters. A brand name that splits into unusual token combinations — due to invented spelling or unconventional capitalization — can be represented less consistently by the model than a name that tokenizes cleanly.
See more terms in the full glossary, or read the AI Visibility 101 guide for the full picture.