Intermediate data lives in Python, not the prompt

When one primitive produces a large list and another consumes it, the list travels between them as a Python variable in the plan's namespace. The model never sees those values: not when the list is produced and not when it is consumed. This tutorial shows that in action with 100 entries and 100,000.

The agent: `ReturnAnalyst`#

The agent holds a series of synthetic monthly returns and exposes arithmetic over them as @primitive methods.

python

# analyst.py
import random
import statistics
from opensymbolicai.blueprints import PlanExecute
from opensymbolicai.core import primitive


def generate_monthly_returns(n: int, seed: int = 42) -> list[float]:
    rng = random.Random(seed)
    return [round(rng.gauss(0.008, 0.04), 6) for _ in range(n)]


class ReturnAnalyst(PlanExecute):
    def __init__(self, n: int, **kwargs) -> None:
        super().__init__(**kwargs)
        self._monthly = generate_monthly_returns(n)

    @primitive(read_only=True)
    def monthly_returns(self) -> list[float]:
        """Monthly return series for the full dataset."""
        return self._monthly

    @primitive(read_only=True)
    def mean_of(self, values: list[float]) -> float:
        """Arithmetic mean of a list of floats."""
        return round(sum(values) / len(values), 6)

    @primitive(read_only=True)
    def count_positive(self, values: list[float]) -> int:
        """Number of positive values in the list."""
        return sum(1 for v in values if v > 0)

    @primitive(read_only=True)
    def len_of(self, values: list[float]) -> int:
        """Number of entries in the list."""
        return len(values)

    @primitive(read_only=True)
    def max_of(self, values: list[float]) -> float:
        """Largest value in the list."""
        return round(max(values), 6)

    @primitive(read_only=True)
    def min_of(self, values: list[float]) -> float:
        """Smallest value in the list."""
        return round(min(values), 6)

    @primitive(read_only=True)
    def subtract(self, a: float, b: float) -> float:
        """Subtract b from a."""
        return round(a - b, 6)

    @primitive(read_only=True)
    def divide(self, a: float, b: float) -> float:
        """Divide a by b."""
        return round(a / b, 6)

The model sees only the signatures: monthly_returns() -> list[float], mean_of(values: list[float]) -> float, and so on. The data itself is invisible to it.

Run three tasks at two scales#

python

# main.py
from analyst import ReturnAnalyst
from opensymbolicai.llm import LLMConfig

TASKS = [
    "What is the mean monthly return?",
    "What fraction of months had a positive return?",
    "What is the range between the best and worst monthly return?",
]

llm = LLMConfig(provider="ollama", model="qwen2.5-coder:7b")

for task in TASKS:
    print(f"\nTask: {task}\n")
    for n in [100, 100_000]:
        agent = ReturnAnalyst(n=n, llm=llm)
        result = agent.run(task)
        print(f"  n={n:>7,}  result = {result.result}")
        print(f"  plan:")
        for line in result.plan.splitlines():
            print(f"    {line}")
        print()

bash

uv run main.py

Output (values vary by run):

text

Task: What is the mean monthly return?

  n=    100  result = 0.010329
  plan:
    returns = monthly_returns()
    mean_return = mean_of(values=returns)
    return mean_return

  n=100,000  result = 0.00809
  plan:
    mean_monthly_return = mean_of(monthly_returns())
    return mean_monthly_return


Task: What fraction of months had a positive return?

  n=    100  result = 0.63
  plan:
    positive_count = count_positive(monthly_returns())
    total_months = len_of(monthly_returns())
    fraction_with_positive_return = divide(positive_count, total_months)
    return fraction_with_positive_return

  n=100,000  result = 0.58112
  plan:
    ...


Task: What is the range between the best and worst monthly return?

  n=    100  result = 0.198228
  plan:
    best_return = max_of(monthly_returns())
    worst_return = min_of(monthly_returns())
    range_between_best_and_worst = subtract(a=best_return, b=worst_return)
    return range_between_best_and_worst

  n=100,000  result = 0.376076
  plan:
    ...

The plan text is the same at n=100 and n=100,000. The model wrote two lines of code. The scale of the data does not change that.

Why the plan doesn't change with data size#

The plan is generated once from the signatures. monthly_returns() returns list[float] whether that list has 100 items or 100,000. The model plans against the type, not the data.

When the plan executes, monthly_returns() returns the real list, and mean_of(values=returns) receives it directly as a Python variable. Neither call serializes the list into a string or a message.

text

OSAI -- data as a Python variable
--------------------------------------------------
  monthly_returns()
        |
        |  100,000 floats
        |  (Python namespace -- model never sees this)
        v
  mean_of(values=returns)  -->  result
--------------------------------------------------
  model sees: 2 lines of code, 0 floats

Compare: tool-calling loops#

In a ReAct / tool-calling loop, every primitive return becomes a message. The list must travel through the context window before the next tool can be called.

text

Tool-calling loop -- data through the context window
--------------------------------------------------
  monthly_returns()
        |
        v
  context: [0.023, -0.041, 0.012, 0.008, ...]  <- 100,000 floats
        |
        |  model copies them into the next call
        v
  mean_of(values=[0.023, -0.041, 0.012, ...])  <- 100,000 floats again
--------------------------------------------------
  model sees: 100,000 floats x 2

At n=100 this is awkward. At n=100,000 it exceeds most context windows. OSAI's plan runs the same code without touching the model either way.

What to notice#

The plan text is identical at both sizes. Read the two plan outputs for the same task and confirm they say the same thing.
Execution time scales with data size; plan time does not. The model call happens once regardless of how many floats are in the list.
The pipe is just a variable name. returns = monthly_returns() is ordinary Python assignment. The framework adds no special mechanism here: it runs the plan in a namespace, so intermediate values are naturally in scope.

The agent: ReturnAnalyst#

Run three tasks at two scales#

Why the plan doesn't change with data size#

Compare: tool-calling loops#

What to notice#

The agent: `ReturnAnalyst`#