Token accounting with result.metrics

The one new thing: result.metrics.plan_tokens is a TokenUsage with input_tokens, output_tokens, and total_tokens. It covers the planning call only. Executing the plan is pure Python and uses no tokens.

Read the token counts#

result = agent.run(task)
t = result.metrics.plan_tokens
print(t.input_tokens, t.output_tokens, t.total_tokens)

That is the whole API. The rest of this tutorial is about what the numbers tell you.

Six tasks, simple to complex#

The agent is a calculator with add, subtract, multiply, divide, factorial, and fibonacci primitives. Six tasks run from a one-step addition to a multi-step combination.

# main.py
from calc import Calc
from opensymbolicai.llm import LLMConfig

TASKS = [
    "What is 7 + 3?",
    "What is 12 * 15 - 47?",
    "What is 8 factorial?",
    "What is the 10th Fibonacci number?",
    "What is 6 factorial plus the 8th Fibonacci number?",
    "What is (factorial of 5) divided by (fibonacci of 6), then add 12?",
]

llm = LLMConfig(provider="ollama", model="qwen2.5-coder:7b")

rows = []
for task in TASKS:
    agent = Calc(llm=llm)
    result = agent.run(task)
    t = result.metrics.plan_tokens
    rows.append((task, result.result, t.input_tokens, t.output_tokens, t.total_tokens))

uv run main.py

Output:

Provider : ollama
Model    : qwen2.5-coder:7b

Task                                           Result   In tokens  Out tokens  Total
------------------------------------------------------------------------------------
What is 7 + 3?                                     10        421          19    440
What is 12 * 15 - 47?                             133        427          28    455
What is 8 factorial?                            40320        419          15    434
What is the 10th Fibonacci number?                 34        423         138    561
What is 6 factorial plus the 8th Fibonacci…       741        426          45    471
What is (factorial of 5) divided by (fib…        27.0        437          61    498

Totals                                                       2553         306   2859

What the numbers show#

Input tokens are nearly constant across all six tasks, around 420. The input is the system prompt plus the list of primitives. The task question adds only a handful of tokens, so the total input barely moves no matter how complex the question is.

Output tokens vary by plan complexity. A one-liner plan for 7 + 3 costs 19 tokens. A multi-step plan for the 10th Fibonacci number costs 138. Simpler questions produce shorter plans even when the computation is harder: factorial(8) is one call, so its output is only 15 tokens.

Execution costs nothing. All six primitives run in pure Python after the plan is written. There is no second model call for execution, regardless of how many steps the plan has.

What to notice#

• Input cost is predictable. It scales with the number of primitives and decompositions in the prompt, not with the task. Adding more primitives to the agent raises the input cost for every task equally.
• Output cost reflects plan length. If your use case favors short plans, simpler primitives that do more work each are cheaper than many small ones.
• Totals across runs aggregate naturally. Sum plan_tokens.total_tokens over a batch to get the cost of the whole batch. Nothing else needs tracking.