Bellman calculus: technical overview¶

For computer scientists and software engineers

The Problem Domain¶

Bellman equations are the mathematical foundation of dynamic programming and reinforcement learning. They describe how to make optimal sequential decisions under uncertainty—where today's choice affects tomorrow's options, and future outcomes are probabilistic.

In economics and operations research, these problems model everything from household savings decisions to inventory management to robotic control. The mathematical structure is well-understood, but translating theory into reliable numerical code remains challenging: it requires expertise in both the domain mathematics and numerical methods.

Bellman calculus addresses this gap. It is a domain-specific language that lets users specify the mathematical structure of a dynamic program declaratively, then compile it into executable objects without entangling the mathematical model with solver internals.

Architecture Overview¶

The system follows a compiler architecture with clear separation between specification and implementation:

┌─────────────────────────────────────────────────────────────────────┐
│                        BELLMAN CALCULUS PIPELINE                     │
│                                                                     │
│   ┌──────────┐     φ      ┌──────────┐     Υ      ┌──────────┐    │
│   │   SYM    │ ─────────► │  CORE    │ ─────────► │   Math   │    │
│   │ (YAML)   │            │   IR     │  meaning   │  Object  │    │
│   └──────────┘            └──────────┘            └──────────┘    │
│                           │                                        │
│                           │   \mathbf{\mathrm{Ρ}} (repr.)          │
│                           └───────────────────────────────►        │
│                                                   ┌──────────┐     │
│                                                   │  Code /  │     │
│                                                   │  Arrays  │     │
│                                                   └──────────┘     │
│                                                                     │
│   Human-authored          Typed Bellman-          Executable        │
│   declarative syntax      calculus stage IR       numerical code    │
└─────────────────────────────────────────────────────────────────────┘

Key translations: - φ (phi): SYM \(\leftrightarrow\) CORE (bijection on a normalized AST) - Υ (Upsilon): assigns mathematical meaning to CORE (a modular dynamic program object) - \(\mathbf{\mathrm{Ρ}}\) (Rho): compiles a calibrated + methodized CORE object into executable representations

In practice, we currently author SYM as dolo-plus / Dolang+ YAML (with implicit rules), normalize to a well-typed CORE stage, and then compile.

The ADC Pattern¶

The core abstraction is the Arrival → Decision → Continuation (ADC) pattern, which decomposes each decision point into three information states:

                           STAGE
    ┌─────────────────────────────────────────────────┐
    │                                                 │
    │    [<]   ────────►    ·    ────────►   [>]     │
    │                                                 │
    │   arrival          decision        continuation │
    │   state            point           state        │
    │                                                 │
    │   Information      Agent           Resulting    │
    │   available        chooses         state after  │
    │   on entry         action          action       │
    └─────────────────────────────────────────────────┘

Each position (perch) represents an information set—a σ-algebra in the underlying probability space. This explicit tracking of information flow distinguishes Bellman calculus from simpler state-machine formalisms.

Stages compose to form complete decision problems. Within a period, adjacent stages are wired by connectors (field renaming between stage interfaces); across periods, successive period instances are wired by twisters.

Key Abstractions¶

Concept	Description	Type Signature
Stage	Single decision point with ADC structure	`(arvl, dcsn, cntn) → Stage`
Perch	Information boundary within a stage	σ-algebra / filtration step
Mover	Operator transforming functions between perches	\(\mathscr B(\mathsf X^{\succ}) \to \mathscr B(\mathsf X^{\prec})\) (factored via \(\mathbf{B},\mathbf{I}\))
Connector	Links stages (continuation → arrival)	rename adapter \(\mathsf X^{\succ,(j)} \to \mathsf X^{\prec,(j+1)}\)
Twister	Links periods (period continuation → next arrival)	rename adapter across period instances

Movers are the computational core—functional operators that transform value functions (backward iteration) or probability distributions (forward simulation).

Model Specification¶

Models are specified in SYM YAML (dolo-plus / Dolang+) with explicit typing and perch-aware objects. A minimal sketch:

symbols:
  prestate: {m: @in R+}      # arrival state
  states: {w: @in R+}        # decision state
  poststates: {a: @in R+}    # continuation state
  controls: {c: @in R+}      # decision control

equations:
  dcsn_to_cntn_transition: |
    a[>] = w - c
  cntn_to_dcsn_mover:
    Bellman: |
      V[-] = max_{c}(u(c) + β*V[>])

The [<], unmarked, [>] annotations are perch tags—measurability assertions declaring which information set each object belongs to. (Aliases [_arvl]/[_dcsn]/[_cntn] and [<-]/[-]/[->] may be accepted for compatibility.)

Three-File Separation¶

Full models separate concerns across three files:

File	Contains	Purpose
`stage.yml`	Symbols, equations	Pure mathematical specification
`methodization.yml`	Algorithm choices	Quadrature, optimization, interpolation schemes
`calibration.yml`	Parameter values	Concrete numbers for parameters and solver settings

This mirrors the distinction between interface and implementation in software design.

Project Structure¶

bellman-ddsl/
├── packages/
│   ├── dolo/              # Model objects + solvers (VFI, EGM, etc.)
│   └── dolang/            # Expression parser + perch-tag handling
├── docs/
│   ├── overview/          # Entry points (this page, guides)
│   ├── theory/            # Foundations of Bellman calculus + MDP theory
│   ├── dolo-plus-spec/    # Dolang+ syntax/semantics + reference tables
│   └── development/       # Implementation specs + research notes
└── explore/               # Notebooks + experimental scripts

Implementation Status¶

The project implements a backwards-compatible extension to Dolo, an existing economic modeling framework. Current capabilities include:

SYM authoring in dolo-plus YAML (perch tags, implicit rules)
Translation to vanilla Dolo objects (for existing solver pipelines)
Methodization + calibration attachment (representation-facing configuration)
EGM/VFI workflows for core examples, including bounded-control constraints

See the dolo-plus spec and the implementation specs for current status, and the SYM/CORE architectural note in the theory section.