In this article
You build a small chatbot, test it for an afternoon, and it costs almost nothing. Then real users arrive, and the bill looks nothing like your estimate. The reason is rarely a mystery: AI services meter usage in tokens, and tokens behave in ways that are easy to misjudge. Once you understand three things, what a token is, why output costs more, and why history keeps growing, you can predict the bill before you ship.
What a token is
Language models do not read letters or whole words. They read tokens, pieces of text chosen so that common fragments get a single token. “the” is one token. A long or unusual word may be split into two or three pieces. Spaces, punctuation and numbers also count. The exact split depends on the model’s tokenizer, so different services count the same sentence slightly differently.
For plain English, two rules of thumb work well enough for budgeting: about 4 characters per token, or about three quarters of a word per token. They are estimates. Code, numbers and rare words use more tokens. Text in scripts other than the Latin alphabet, such as Hindi in Devanagari, often needs noticeably more tokens for the same meaning, so a Hindi or mixed-language chatbot can cost more than the same bot in English.
Tokens ≈ words ÷ 0.75. Use these only for planning. For exact counts, run your text through a token counter.
Input, output and the price per million
Providers publish prices per million tokens, with separate rates for input and output. Output costs more because the model has to produce it one token at a time, which uses far more computing than reading the input in one pass. A common pattern is output at around three to five times the input rate.
Take a customer-support chatbot. Each conversation sends about 800 tokens in (instructions plus the customer’s message) and gets 300 tokens back. The cost of one conversation is (800 × 3 + 300 × 15) ÷ 1,000,000 = $0.0069, under one cent. That feels free, until you multiply.
A tenth of a cent here and there becomes real money at scale. Always multiply by your expected volume.
- Input tokens (800 per chat)$72
- Output tokens (300 per chat)$135
Output is only 27% of the tokens (300 of 1,100) but 65% of the cost. That is why the length of the answer is the first thing to look at.
Why chat history makes the bill grow
A model has no memory between requests. To make a chat feel continuous, the app sends the entire conversation so far each time. On turn 1 you send only the first message. On turn 5 you send turns 1 to 4 again, plus the new one. The new text each turn stays small, but the input keeps climbing.
Suppose each user message is 200 tokens and each reply 300 tokens. Turn 1 sends 200 tokens. Turn 2 sends 200 + (200 + 300) = 700. Turn 3 sends 1,200. And so on, 500 more every turn.
By turn 10 you are sending 4,700 tokens to ask a 200-token question. The grey line is what it would be if history were free; the blue line is what you pay.
Total input for the 10 turns is 24,500 tokens against only 2,000 of genuinely new text, 12 times as much. The output stays at 3,000 tokens either way, so the history effect hits the input side.
The context window: a ceiling on the sum
Every model has a context window, the maximum number of tokens it can handle in one request, counting input and output together. A window of 128,000 tokens holds roughly 96,000 words, close to a full-length novel. If your history plus the new question plus the expected answer exceeds the window, something must be dropped or the call fails.
- 1InstructionsThe system prompt that tells the model how to behave.
- 2History and documentsEarlier turns, retrieved notes and any files you pasted in.
- 3The new messageWhat the user just asked.
- 4Room for the answerOutput tokens count against the same window.
Plan the window like a suitcase: everything has to fit at once. The context window calculator shows how much space is left.
If a document is too big, split it into pieces and process them one by one with an AI text chunker, rather than forcing it into a single call.
How big prompts and big answers change the cost
The next table shows the cost of 100,000 calls at the example prices for different prompt and answer sizes. Moving down means a bigger prompt, moving right means a longer answer.
| Input tokens | 100 out | 500 out | 1,500 out |
|---|---|---|---|
| 500 in | $300 | $900 | $2,400 |
| 2,000 in | $750 | $1,350 | $2,850 |
| 8,000 in | $2,550 | $3,150 | $4,650 |
The ringed cell, 2,000 tokens in and 500 out, costs $1,350 for 100,000 calls. A long answer hurts more than a long prompt at the same size: compare 500 in, 1,500 out ($2,400) with 8,000 in, 100 out ($2,550), where the prompt is 16 times larger yet costs about the same.
Four ways to cut the cost
Because the bill is a simple formula, each lever is easy to test. Using the support-bot example (800 in, 300 out, $6.90 per 1,000 conversations), here is what each change does.
Halving the answer saves 33%, halving the prompt only 17%, because output costs five times as much per token. Halving both cuts the bill by half.
- Ask for shorter answers. A line such as “answer in under 100 words unless asked for more” can cut output dramatically. Set a maximum output length as a safety net.
- Trim the history. Keep the last few turns and a short summary of the rest, instead of the whole transcript.
- Tighten the instructions. A 2,000-token system prompt sent on every call is a fixed tax. Remove examples and rules the model does not need. The prompt tools on this site, such as the prompt generator, help you write compact ones.
- Cache and batch where offered. Some providers discount repeated prompt prefixes and offer cheaper rates for jobs that can wait. Check the documentation, since terms vary.
- Use the right-sized model. Easy tasks such as classification or short rewrites rarely need the most expensive model. Test a smaller one on a sample.
A back-of-envelope method you can reuse
- 1Estimate tokens per callPrompt plus history in, answer out. Use words ÷ 0.75 if you have only word counts.
- 2Multiply by the priceInput tokens × input rate plus output tokens × output rate, each per million.
- 3Multiply by volumeCalls per day × 30.
- 4Add a safety marginAdd 20 to 30 percent for retries, long chats and growth.
- 5Test with real trafficRun a small pilot and compare the real token counts with your guess.
A single AI call is nearly free. A million of them is a line item in the budget. The skill is simply in multiplying early, watching the output length, and keeping history under control. If you like thinking in compounding terms, the same “small things times many” idea appears in the minimum payment trap and the appliance cost guide. Put your own numbers in the calculator above and see what your app will cost before you launch it.
Questions people ask
What is a token in AI?
A token is a small chunk of text, such as a short word or part of a longer word, which language models use as their unit of reading and writing. In English, one token is about four characters or three quarters of a word.
How many tokens is 1,000 words?
About 1,300 to 1,400 tokens in English. Code, numbers and non-English text usually use more.
Why do output tokens cost more than input tokens?
The model writes output one token at a time, which takes much more computing than reading the input. Providers usually price output at several times the input rate.
Why does a long chat cost more per message?
The app resends the earlier messages with each new one, so the input grows every turn. Total input grows roughly with the square of the chat length.
What is a context window?
The maximum number of tokens, input and output combined, that a model can handle in a single request.
How can I lower my AI API bill?
Ask for shorter answers, trim the conversation history, shorten the instructions, use caching or batch discounts where offered, and pick a smaller model for easy tasks.