Expectations, Values, Recipes, and “Where Does \(E\) Live?” in Dolo

This note consolidates the key “forward-looking recipe” insights that came up in our Dec 22 discussion and the follow-up debugging/exploration:

• how recipes.yaml defines what timed objects exist (signatures),
• why values: [V] and expectations: [m] are different kinds of objects,
• why the YAML syntax does not tell you whether you are computing \(f(E[V])\) vs \(E[f(V)]\),
• where the actual integration / expectation operator lives in Dolo algorithms,
• and how this resolves the “variable vs function” ambiguity (e.g., V[t+1]).

Links to prior notes: - 06-time-indices-compilation.md (how [t±1] becomes function arguments) - 03-function-compilation.md (how equation blocks become compiled functions)

---

1. The Three Layers: SYNTAX → COMPILATION → ALGORITHM

It helps to separate Dolo into three conceptual layers:

1. YAML syntax (what you write): timed symbols like c[t+1], V[t+1], m[t].
2. Compilation (what functions get generated): recipes.yaml determines signatures and which timed objects are passed as positional arguments.
3. Algorithms (what is integrated / iterated): solvers decide where to loop over shocks/integration nodes and thus where the expectation operator \(E\) lives.

The main confusion is usually mixing (1) and (3): the YAML notation looks like it contains probability/expectation semantics, but in Dolo that semantics mostly lives in the algorithms.

---

2. Recipes Define Signatures (Not Expectation Semantics)

In packages/dolo/dolo/compiler/recipes.yaml, the dtcc recipe lists which symbol groups exist and what each equation block “expects” as inputs.

For example (abridged):

dtcc:
  symbols: [exogenous, states, controls, poststates, rewards, values, expectations, shocks, parameters]

  specs:
    value:
      target: [values, 0, v]
      eqs:
        - [exogenous, 0, m]
        - [states,    0, s]
        - [controls,  0, x]
        - [values,    0, v]
        - [exogenous, 1, M]
        - [states,    1, S]
        - [controls,  1, X]
        - [values,    1, V]
        - [parameters,0, p]

    expectation:
      target: [expectations, 0, z]
      eqs:
        - [exogenous, 1, M]
        - [states,    1, S]
        - [controls,  1, X]
        - [parameters,0, p]

What this means: - A compiled value function is allowed to reference both “current” objects (m,s,x,v) and “future” objects (M,S,X,V). - A compiled expectation function is an “integrand factory”: it can only reference future objects (M,S,X) and parameters.

What it does not mean: - It does not say “take expectations of V[t+1]”. There is no explicit E[...] operator in the recipe. - Recipes define data plumbing (what arguments get passed), not the probabilistic operator.

---

3. Time Indices Are Compile-Time Directives

In Dolo syntax, [t], [t+1], [t-1] are best understood as compile-time directives that determine which argument group a symbol belongs to.

Example:

equations:
  arbitrage:
    - 1 - beta*(c[t]/c[t+1])^sigma*(1-delta+rk[t+1])

This is compiled into a function that receives both “current” arrays and “future” arrays (see 06-time-indices-compilation.md).

Key point:

A time lead [t+1] by itself does not mean “take expectation”.

It only means “this equation refers to future-period objects, so the compiled function must accept the future argument blocks”.

---

4. values vs expectations: Two Different Roles

Both values and expectations are “forward-looking” in economics, but Dolo treats them differently because they play different roles in the computational pipeline.

Symbol group intuition

Symbol group	What it is (computationally)	Typical example	How it is produced
values	A function object represented on a grid (via DecisionRule) and updated by a fixed-point / Bellman step	V	value iteration / evaluation of Bellman recursion
expectations	A moment / intermediate object (also tabulated on grid) used to simplify policy updates	m (“expected marginal value”)	computed as an expectation of an integrand, then fed into direct_response / EGM-style steps

Why V[t+1] is not in expectations

In a Bellman recursion, V[t+1] is not “a new expectation variable” — it is a recursive reference to the same function \(V(\cdot)\) evaluated at a different argument (next state), and typically integrated over shocks:

\[ V(s) \;=\; u(s,x(s)) \;+\; \beta\,E\big[V(s')\big] \]

So values: [V] signals “there is a fixed-point object to solve for”.

By contrast, expectations: [m] is usually a derived object like:

\[ m(s) \equiv E\big[g(s',x(s'))\big] \]

and then the policy update uses m as an input.

---

5. Where Does the Expectation Operator \(E\) Actually Live?

Dolo does not place an explicit E[...] operator in YAML. Instead, the expectation is implemented by algorithms that:

1. discretize the exogenous process into nodes (inode) and weights (iweight),
2. evaluate an integrand at each node,
3. sum (or average) across nodes.

This is why, in practice, “expectation is taken with respect to the whole block output”: the algorithm wraps the compiled block evaluation inside a quadrature/transition loop.

(A) Time iteration (Euler residuals)

In time_iteration, the residuals are accumulated as a weighted sum across integration nodes: the algorithm computes something like \(E[\text{EulerResidual}(t+1)]\) as part of the residual evaluation.

Concrete example (abridged from packages/dolo/dolo/algos/time_iteration.py):

for I_ms in range(dprocess.n_inodes(i_ms)):
    M = dprocess.inode(i_ms, I_ms)
    prob = dprocess.iweight(i_ms, I_ms)
    S = g(m, s, xm, M, parms)
    XM = dr.eval_ijs(i_ms, I_ms, S)
    rr = f(m, s, xm, M, S, XM, parms)      # integrand evaluation
    res[i_ms, :, :] += prob * rr           # expectation / quadrature

(B) EGM-style models

In egm.py, the compiled expectation block function h(...) is treated as the integrand, and the algorithm does:

\[ z(m,s) = \sum_{i_M} w_{i_M}\; h(M, S(M), X(M)) \]

Then z (an expectations object) is fed into a policy update (direct_response_egm).

Concrete example (abridged from packages/dolo/dolo/algos/egm.py):

for i_M in range(dp.n_inodes(i_m)):
    w = dp.iweight(i_m, i_M)
    M = dp.inode(i_m, i_M)
    S = gt(m, a, M, p)
    X = drfut(i_M, S)
    z[i_m, :, :] += w * h(M, S, X, p)      # h(...) is the integrand

(C) Value iteration (Bellman problems)

Value iteration effectively has two places where expectations appear:

1. Policy evaluation: compute the value function implied by a fixed policy (integration occurs during value evaluation).
2. Policy improvement: when maximizing over x, the objective includes \(\beta\,E[V(s')]\), so the integrator is inside the objective evaluation.

So it is normal to see expectation/integration both “before” and “inside” the maximization loop: policy iteration interleaves evaluation and improvement.

Two concrete snippets from packages/dolo/dolo/algos/value_iteration.py:

1) Inside the maximizer objective (expectation depends on candidate x):

cont_v = 0.0
for I_ms in range(dprocess.n_inodes(i_ms)):
    M = dprocess.inode(i_ms, I_ms)
    prob = dprocess.iweight(i_ms, I_ms)
    S = transition(m, s, x, M, parms)
    V = drv(I_ms, S)[0]
    cont_v += prob * V
return felicity(m, s, x, parms) + beta * cont_v

2) During policy evaluation (expectation inside the value-function update):

for I_ms in range(n_mv):
    M = dprocess.inode(i_ms, I_ms)
    prob = dprocess.iweight(i_ms, I_ms)
    S = g(m, s, xm, M, parms)
    XM = dr.eval_ijs(i_ms, I_ms, S)
    VM = drv.eval_ijs(i_ms, I_ms, S)
    rr = val(m, s, xm, vm, M, S, XM, VM, parms)
    res[i_ms, :, :] += prob * rr

---

6. The \(f(E[V])\) vs \(E[f(V)]\) Question (and Why YAML Doesn’t Decide)

Suppose you write an expression like:

value:
  - V[t] = u[t] + beta * f(V[t+1])

In stochastic Dolo algorithms, the integration loop typically wraps evaluation of the RHS as an integrand. That means the solver computes:

\[ \beta \, E\!\left[f(V(s'))\right] \]

not \( \beta\, f(E[V(s')]) \).

Why? Because the algorithm is:

1. loop over shock nodes,
2. compute next state \(s'\),
3. evaluate \(V(s')\),
4. compute \(f(V(s'))\),
5. weight and sum across nodes.

There is no syntax-level E[...] operator in YAML that would let you place expectation inside or outside f.

Implication: if you truly need \(f(E[V])\), you would need to introduce an intermediate object representing \(E[V]\) and then apply \(f\). In stock Dolo/dtcc, that is not expressible without extending the recipe signatures (because the expectation: block does not take values[1] as an input by default).

---

7. Resolving the Function-vs-Variable Ambiguity

The core ambiguity is:

In YAML, V[t] looks like a scalar time-indexed variable.
In computation, V is a function over the state space (and possibly over exogenous nodes).

Dolo resolves this implicitly by its data structures: - Controls are represented as a DecisionRule \(x = x(m,s)\). - Values are represented as a DecisionRule \(V = V(m,s)\). - A reference like V[t+1] means “evaluate the value-function approximation at the next state”.

So mentally, you should read:

• V[t] as “\(V(m_t, s_t)\)”
• V[t+1] as “\(V(m_{t+1}, s_{t+1})\)” (with \(s_{t+1}\) coming from transition)

and then the algorithm decides whether to integrate over \(m_{t+1}\) (and/or other shocks) and where that integration sits in the computational flow.

---

8. Implications for DDSL-SYM / T core (from Dec 22)

This Dolo behavior motivates the “expectation location” design principle we recorded on Dec 22:

• Dolo’s YAML syntax stays implicit and economist-friendly (good DDSL-SYM target).
• The true semantics of expectation placement live in the algorithmic layer (what we want to make explicit in T core).

In other words:

In Dolo, expectation is primarily an algorithmic operator, not a syntactic operator.
T core should make that operator explicit; DDSL-SYM can keep the implicit [t+1] style.

Christopher Carroll and Akshay Shanker — repository in development, please do not cite.