AI Token Counter
Approximate token counts for the major model families, with cost.
Why tokens are not characters or words
Language models read subword units. Common English words are usually a single token; longer or rarer words split into two or three, and punctuation, spaces and numbers each consume tokens of their own. A rough rule of thumb for English prose is one token per four characters, or about 0.75 tokens per word — but the ratio shifts with content. Code tokenises far less efficiently because identifiers and symbols split into many pieces; JSON and log data are worse still. Non-Latin scripts usually cost more tokens per character of meaning, which is why the same document in Japanese can cost noticeably more than in English.
How the estimate is produced
This tool counts words, characters and whitespace separately, then applies a per-family multiplier calibrated on typical English text: GPT-style byte-pair encoding at roughly 1.3 tokens per word, Claude slightly higher, and Llama-family tokenisers close to GPT. It then adjusts for punctuation density, because heavy punctuation pushes the count up. Results normally land within about ten percent of the official tokeniser for prose. That is accurate enough for planning a prompt or estimating a monthly bill, and not accurate enough for a precise per-request budget — for that, call your provider's tokeniser endpoint, which is free and exact.
Why the number matters
- Context limits. Models cap input plus output at a fixed window. If your prompt plus retrieved documents exceed it, the call fails or gets silently truncated.
- Cost. Providers bill per token, input and output priced separately. A prompt that is 20% longer costs 20% more on every single call, and at millions of requests a month that is real money.
- Latency. Time to first token scales with output length, and processing time scales with input length. Shorter prompts feel faster.
- Quality. Very long contexts degrade attention: models handle information at the start and end of a prompt better than material buried in the middle.
Reducing token usage
The biggest wins are structural rather than stylistic. Trim retrieved context to the passages that actually answer the question rather than passing whole documents. Remove boilerplate, repeated instructions and formatting you do not need. Cache static prompt prefixes where the provider supports it, since cached tokens are usually billed at a fraction of the normal rate. Set an explicit max_tokens so a runaway response cannot generate an expensive wall of text. Then check the result with the cost calculator to see what those changes are worth per month, and the context window calculator to confirm what fits.
Input and output are priced differently
Output tokens almost always cost several times more than input tokens, because generating them requires far more compute. A chatbot that reads a short question and writes a long answer can therefore spend most of its budget on output despite the prompt looking large. When estimating, count both sides separately — and remember that a "cheap per million input" headline often hides an output rate five times higher.