# AI tokens and API cost: why the bill depends on the answer length and the chat history

> AI services charge by the token, not by the word or the message. Output tokens usually cost several times more than input, and a long conversation resends its whole history every turn. This guide explains tokens with simple rules of thumb, works through a monthly-bill example, and shows four ways to cut the bill without hurting quality.

Published 2026-10-08 by Toolcloak. 6 minute read. Original: https://toolcloak.pages.dev/articles/ai-tokens-and-api-cost-why-the-bill-depends-on-output-and-chat-history
Related free tool: [AI API Cost Calculator](https://toolcloak.pages.dev/tools/llm-cost-calculator)

You build a small chatbot, test it for an afternoon, and it costs almost nothing. Then real users arrive, and the bill looks nothing like your estimate. The reason is rarely a mystery: AI services meter usage in tokens, and tokens behave in ways that are easy to misjudge. Once you understand three things, what a token is, why output costs more, and why history keeps growing, you can predict the bill before you ship.

> **In short**
> - A **token** is a chunk of text, roughly **4 characters or 0.75 words** in English. 1,000 words is about **1,300 tokens**.
> - You pay for **input tokens** (what you send) and **output tokens** (what the model writes). Output is usually priced at several times the input rate.
> - In a chat, every turn **resends the earlier messages**, so input grows turn by turn. A 10-turn chat can use **12 times** the input of its new text alone.
> - To cut cost: **shorten answers, trim history, keep prompts tight, and cache or batch** where the provider allows.

## What a token is

Language models do not read letters or whole words. They read **tokens**, pieces of text chosen so that common fragments get a single token. “the” is one token. A long or unusual word may be split into two or three pieces. Spaces, punctuation and numbers also count. The exact split depends on the model’s tokenizer, so different services count the same sentence slightly differently.

For plain English, two rules of thumb work well enough for budgeting: about **4 characters per token**, or about **three quarters of a word per token**. They are estimates. Code, numbers and rare words use more tokens. Text in scripts other than the Latin alphabet, such as Hindi in Devanagari, often needs noticeably more tokens for the same meaning, so a Hindi or mixed-language chatbot can cost more than the same bot in English.

**Roughly how many tokens different texts use (English)**

|  | Tokens |
| --- | --- |
| A tweet (280 characters) | ≈ 70 |
| A 100-word email | ≈ 133 |
| A 500-word page | ≈ 667 |
| A 2,000-word article | ≈ 2,670 |
| A 25,000-word report (about 50 pages) | ≈ 33,000 |

*Tokens ≈ words ÷ 0.75. Use these only for planning. For exact counts, run your text through a [token counter](https://toolcloak.pages.dev/tools/token-counter).*

Try it: [AI API Cost Calculator](https://toolcloak.pages.dev/tools/llm-cost-calculator). Enter prompt size, answer size, price per million tokens and number of calls to estimate the cost.

## Input, output and the price per million

Providers publish prices per **million tokens**, with separate rates for input and output. Output costs more because the model has to produce it one token at a time, which uses far more computing than reading the input in one pass. A common pattern is output at around three to five times the input rate.

> **Cost = (input tokens × input price + output tokens × output price) ÷ 1,000,000:** In this article we use example prices of **$3 per million input tokens** and **$15 per million output tokens**. These are made-up round numbers for the arithmetic. Check your provider’s current price list before budgeting.

Take a customer-support chatbot. Each conversation sends about 800 tokens in (instructions plus the customer’s message) and gets 300 tokens back. The cost of one conversation is (800 × 3 + 300 × 15) ÷ 1,000,000 = **$0.0069**, under one cent. That feels free, until you multiply.

**A support bot with 1,000 conversations a day**

|  | Value | Note |
| --- | --- | --- |
| Per conversation | $0.0069 | 800 tokens in, 300 out |
| Per 1,000 conversations | $6.90 | One day of traffic |
| Per month | $207 | 30 days of the same load |

*A tenth of a cent here and there becomes real money at scale. Always multiply by your expected volume.*

**Where the $207 a month goes**

|  | Share |
| --- | --- |
| Input tokens (800 per chat) | $72 |
| Output tokens (300 per chat) | $135 |

*Output is only 27% of the tokens (300 of 1,100) but 65% of the cost. That is why the length of the answer is the first thing to look at.*

## Why chat history makes the bill grow

A model has no memory between requests. To make a chat feel continuous, the app sends the **entire conversation so far** each time. On turn 1 you send only the first message. On turn 5 you send turns 1 to 4 again, plus the new one. The new text each turn stays small, but the input keeps climbing.

Suppose each user message is 200 tokens and each reply 300 tokens. Turn 1 sends 200 tokens. Turn 2 sends 200 + (200 + 300) = 700. Turn 3 sends 1,200. And so on, 500 more every turn.

**Input tokens sent on each turn of a 10-turn chat**

| Turn number | Input actually sent (history + new message) | New text only |
| --- | --- | --- |
| 1 | 200 | 200 |
| 2 | 700 | 200 |
| 3 | 1200 | 200 |
| 4 | 1700 | 200 |
| 5 | 2200 | 200 |
| 6 | 2700 | 200 |
| 7 | 3200 | 200 |
| 8 | 3700 | 200 |
| 9 | 4200 | 200 |
| 10 | 4700 | 200 |

*By turn 10 you are sending 4,700 tokens to ask a 200-token question. The grey line is what it would be if history were free; the blue line is what you pay.*

**Cost of one 10-turn chat, with and without the history effect (example prices)**

|  | Input | Output |
| --- | --- | --- |
| If only new text counted | $0.006 | $0.045 |
| With history resent | $0.0735 | $0.045 |

*Total input for the 10 turns is 24,500 tokens against only 2,000 of genuinely new text, 12 times as much. The output stays at 3,000 tokens either way, so the history effect hits the input side.*

> **Long chats get expensive, not just long:** The total input of a chat grows roughly with the **square** of its length: doubling the number of turns more than doubles the input. That is why “start a new conversation” is good cost advice as well as good usability advice.

## The context window: a ceiling on the sum

Every model has a **context window**, the maximum number of tokens it can handle in one request, counting input and output together. A window of 128,000 tokens holds roughly 96,000 words, close to a full-length novel. If your history plus the new question plus the expected answer exceeds the window, something must be dropped or the call fails.

**What has to fit in one request**

1. **Instructions.** The system prompt that tells the model how to behave.
2. **History and documents.** Earlier turns, retrieved notes and any files you pasted in.
3. **The new message.** What the user just asked.
4. **Room for the answer.** Output tokens count against the same window.

*Plan the window like a suitcase: everything has to fit at once. The [context window calculator](https://toolcloak.pages.dev/tools/ai-context-window-calculator) shows how much space is left.*

If a document is too big, split it into pieces and process them one by one with an [AI text chunker](https://toolcloak.pages.dev/tools/ai-text-chunker), rather than forcing it into a single call.

## How big prompts and big answers change the cost

The next table shows the cost of **100,000 calls** at the example prices for different prompt and answer sizes. Moving down means a bigger prompt, moving right means a longer answer.

**Cost of 100,000 calls (US dollars, example prices)**

| Input tokens | 100 out | 500 out | 1,500 out |
| --- | --- | --- | --- |
| 500 in | $300 | $900 | $2,400 |
| 2,000 in | $750 | $1,350 | $2,850 |
| 8,000 in | $2,550 | $3,150 | $4,650 |

## Four ways to cut the cost

Because the bill is a simple formula, each lever is easy to test. Using the support-bot example (800 in, 300 out, $6.90 per 1,000 conversations), here is what each change does.

**Cost per 1,000 conversations after each change (example prices)**

|  | Cost |
| --- | --- |
| Today | $6.90 |
| Halve the prompt (400 in) | $5.70 |
| Halve the answer (150 out) | $4.65 |
| Halve both | $3.45 |

*Halving the answer saves 33%, halving the prompt only 17%, because output costs five times as much per token. Halving both cuts the bill by half.*

1. **Ask for shorter answers.** A line such as “answer in under 100 words unless asked for more” can cut output dramatically. Set a maximum output length as a safety net.
2. **Trim the history.** Keep the last few turns and a short summary of the rest, instead of the whole transcript.
3. **Tighten the instructions.** A 2,000-token system prompt sent on every call is a fixed tax. Remove examples and rules the model does not need. The prompt tools on this site, such as the [prompt generator](https://toolcloak.pages.dev/tools/prompt-generator), help you write compact ones.
4. **Cache and batch where offered.** Some providers discount repeated prompt prefixes and offer cheaper rates for jobs that can wait. Check the documentation, since terms vary.
5. **Use the right-sized model.** Easy tasks such as classification or short rewrites rarely need the most expensive model. Test a smaller one on a sample.

> **Do not forget the retries:** Failed calls, retries and “try again” buttons are billed like any other call. If 10% of requests are retried, your real volume is 10% higher than your user count suggests.

## A back-of-envelope method you can reuse

**Estimate the monthly bill in five steps**

1. **Estimate tokens per call.** Prompt plus history in, answer out. Use words ÷ 0.75 if you have only word counts.
2. **Multiply by the price.** Input tokens × input rate plus output tokens × output rate, each per million.
3. **Multiply by volume.** Calls per day × 30.
4. **Add a safety margin.** Add 20 to 30 percent for retries, long chats and growth.
5. **Test with real traffic.** Run a small pilot and compare the real token counts with your guess.

A single AI call is nearly free. A million of them is a line item in the budget. The skill is simply in multiplying early, watching the output length, and keeping history under control. If you like thinking in compounding terms, the same “small things times many” idea appears in [the minimum payment trap](https://toolcloak.pages.dev/articles/the-minimum-payment-trap-what-a-credit-card-balance-really-costs) and [the appliance cost guide](https://toolcloak.pages.dev/articles/what-your-appliances-cost-per-month-watts-hours-and-units-explained). Put your own numbers in the calculator above and see what your app will cost before you launch it.

## Frequently asked questions

### What is a token in AI?

A token is a small chunk of text, such as a short word or part of a longer word, which language models use as their unit of reading and writing. In English, one token is about four characters or three quarters of a word.

### How many tokens is 1,000 words?

About 1,300 to 1,400 tokens in English. Code, numbers and non-English text usually use more.

### Why do output tokens cost more than input tokens?

The model writes output one token at a time, which takes much more computing than reading the input. Providers usually price output at several times the input rate.

### Why does a long chat cost more per message?

The app resends the earlier messages with each new one, so the input grows every turn. Total input grows roughly with the square of the chat length.

### What is a context window?

The maximum number of tokens, input and output combined, that a model can handle in a single request.

### How can I lower my AI API bill?

Ask for shorter answers, trim the conversation history, shorten the instructions, use caching or batch discounts where offered, and pick a smaller model for easy tasks.