Tokens (in AI), explained
A token is a chunk of text (roughly a word or part of a word) that an AI reads and writes in. Models measure input, output, and cost in tokens.
AI models do not read letters or whole words exactly; they read tokens. A token is a piece of text, often about three-quarters of a word, so a sentence might be a dozen or so tokens. The model breaks your input into tokens, predicts tokens back, and that is the unit everything is measured in.
Tokens matter for two practical reasons: cost and length. Paid APIs charge per token, and every model has a limit on how many tokens it can consider at once (its context window). Longer prompts and longer answers cost more and eat into that limit.
This is also why models sometimes stumble on things like spelling or counting letters in a word: they see tokens, not individual characters. Knowing that explains a lot of otherwise baffling behavior.
Go deeper
Wield's AI Foundations track covers this hands-on, in plain English, with real examples and a copy-paste prompt to try it yourself.
Learn it, or have it done for you
Understanding the term is step one; using it well is the course. Start the course free and build a working AI habit yourself — or, if you'd rather skip to the outcome, MCF Agentic builds the AI workflows into your business directly.