Overview

dolo-plus (ADC dialect) — spec_0.1 overview (exploratory)¶

This "spec_0.1" work is an exploratory exercise to iterate on dolang and to some extent dolo to create a native DDSL representation kit. Our objective is to keep changes to dolang and dolo clinical and minimal, and learn what minimal dolo-plus structure is needed to run an ADC-stage model through an existing Dolo solver and map results back to ADC objects.

Hard constraint: minimal/no changes to vanilla Dolo¶

This spec is written to keep changes to dolang/dolo minimal and ensure the original functionality of dolang/dolo is unbroken:

v0.1 stance (this repo): we will make clinical, additive changes to dolang/dolo's YAML importer to recognize dolang-plus stage files and load them in a parse-only mode (no compilation), while keeping all legacy behavior unchanged for plain Dolo files.
Any change inside Dolo must:
be narrowly scoped and behind an opt-in switch (stage mode),
preserve all existing behavior for plain Dolo models,
come with regression tests showing "old models still import/compile/solve".

In particular, existing Dolo examples and the existing explore/* scripts must continue to work unchanged.

Target outcome (end of spec_0.1)¶

By the end of spec_0.1 we want a working end-to-end round trip:

1) Parse dolo-plus stage YAML into a dolo-plus-native model representation (perch tags preserved). 2) Decorators + calibration + methods plumbing (parameter registration): attach optional symbol metadata (e.g. β @in (0,1)) without changing legacy symbol names, and validate/calibrate in a way that keeps old Dolo behavior unchanged. 3) Translate dolo-plus → Dolo model representation (the "horse" interface): rewrite perch tags and dolo-plus structure into something vanilla Dolo can compile/solve. 4) Solve with vanilla Dolo (e.g. EGM) and obtain Dolo solver outputs (e.g. DecisionRule, grids, etc.). 5) Translate solver outputs back to ADC/dolo-plus objects, so we recover stage/perch objects (policy/value representations) in the ADC vocabulary.

Milestone split¶

This table is the single source of truth for dev-spec status. Keep it in sync with actual spec files.

Spec	Title	Status	Spec file	Description
0.1a	Parse ADC	✅ Done	`spec_0.1a-parse-ADC.md`	Produce dolo-plus-native `equations` object, preserve perch tags
0.1b	Translator	✅ Done	`spec_0.1b-doloplus-to-dolo-translator.md`	Rewrite perch tags to numeric indices, emit dtcc/EGM horse
0.1c	Calibration	✅ Done	`spec_0.1c-calibration.md`	Metadata-first decorator system, parameter registration, calibration functor
0.1d	Methodization functor	✅ Done	`spec_0.1d-methodization.md`	Methodization functor for solver binding. Method schemes registry.
0.1e	Back-translation	Planned	—	Map Dolo outputs back to ADC/dolo-plus objects
0.1eA	Semiconjugate solve	Planned	`spec_0.1eA-stage-operators-to-dolo.md`	Stage operators to dolo translation for solver integration
0.1f	Constraints	✅ Done	`spec_0.1f-constraints.md`	`⊥` bound declarations in value block (VFI control bounds)
0.1g	Bare symbol resolution	✅ Done	`spec_0.1g-bare-symbol-resolution.md`	Native perch inference for bare symbols
0.1h	Periods and nests	✅ Done	`spec_0.1h-periods-nests.md`	Dict-based periods + connectors, backward accretion
0.1hA	Discount stage	Planned	`spec_0.1hA-discount-stage.md`	Extract β into dedicated terminal stage (`disc`). Enables port-only periods. (CDC #11)
0.1hB	Solution lifecycle	Planned	`spec_0.1hB-solution-lifecycle.md`	Solutions attached to periods (lifecycle enrichment). Single interleaved nest. (CDC #10)
0.1i	Functions and operators	Planned	—	Define reward or Q functions as first-class objects, not just string replacements
0.1j	Run horse solver	Planned	—	Feed translated horse into Dolo solver (EGM)
0.1k	RAG assistant	✅ Done	`spec_0.1k-rag-assistant.md`	Matsya RAG index and query pipeline for project documentation
0.1l	Branching stages	In progress	`spec_0.1l-branching.md`, `spec_0.1l-branching-theory.md`	Multi-branch compositions, discrete choice, DAG periods/nests
0.1lA	Connector unification	Planned	`spec_0.1lA-connector-unification.md`	Unify "connector"/"twister" → "connector" with scope qualifiers. `:=:` syntax. (CDC #8, #9)
0.1m	Constraints (extended)	Planned	—	More careful consideration of constraints beyond VFI bounds
0.1o	Infinite horizon / fixed points	Planned	—	Loops, stationarity, fixed-point semantics
0.1p	Intermediate-variable indexing	Planned	`spec_0.1p-intermediate-variable-indexing.md`	`c[_a]`, `m_d[_a]` syntax for symbols indexed by transition intermediates (EGM endogenous grid)
0.1q	`solve_{}` operator + `d_{}` + user functions	✅ Done	`spec_0.1q-solve-operator.md`	`solve_{c[>]}{eq=0}` implicit system operator, `d_{a}V` derivative variables, `y(z)` user-defined function calls, branch-keyed sub-equations in compiler
0.2a	Compiler pipeline architecture	Planned	`spec_0.2a-compiler-pipeline-architecture.md`	Pipeline stages, intermediate representations, operator registry, environment threading (Lean 4-inspired)

0.1a — Parse-only (no dolang/Dolo interface, no compilation)¶

Status: ✅ Implemented

Produce a dolo-plus-native equations object (with sub-equations), read label → symbol metadata, preserve perch tags like [_arvl].

Spec: spec_0.1a.md

0.1b — Interface/translation to "old dolang/Dolo"¶

Status: ✅ Implemented

Rewrite perch tags to numeric slot/time indices and emit a dtcc/EGM horse that Dolo can compile.

Primary specs:
spec_0.1b-doloplus-to-dolo-translator.md

0.1c — Decorators + calibration + methods (focus: parameter registration)¶

Status: ✅ Implemented

Introduce a metadata-first decorator system for parameters/settings that: - preserves legacy symbols: parameters: [β, ...] lists and legacy calibration semantics, - stores decorators separately from symbol names (so old Dolo code paths remain runnable), - optionally validates calibrated values (e.g. β ∈ (0,1)), - threads method/config blocks through the translation to the horse unchanged.

Spec: spec_0.1c.md

0.1d — Methodization functor¶

Status: ✅ Implemented Methodization functor for solver binding — binds solution method configuration to calibrated stage.

Spec: spec_0.1d.md

0.1e — Back-translation (horse outputs → ADC outputs)¶

Status: Planned

Map Dolo's outputs back into dolo-plus/ADC objects (e.g. a decision-perch policy, continuation-grid representation if needed, and any value/shadow-value objects we choose to recover).

0.1f — VFI control bounds via `⊥` declarations¶

Status: ✅ Implemented

Enable VFI solvers to read control bounds directly from the model YAML by allowing ⊥ (up tack) bound declarations within the value equation block. This is a vanilla Dolo enhancement (not dolo-plus pipeline specific).

Spec: spec_0.1f.md

Implementation: - Grammar: packages/dolang/dolang/grammar.lark (bound_constraint rule) - Parser: packages/dolo/dolo/compiler/recipes.yaml (value block bounds extraction)

0.1g — Bare symbol resolution (native perch inference)¶

Status: ✅ Implemented

Allow bare symbols (without explicit perch indices) in adc-stage equations. Infer perch from symbol declarations: - prestate symbols → _arvl - states symbols → _dcsn - poststates symbols → _cntn - controls symbols → _dcsn

Spec: spec_0.1g-bare-symbol-resolution.md

Implementation: - Grammar: packages/dolang/dolang/grammar.lark (bare symbol in assignment LHS) - Resolver: packages/dolang/dolang/perch_resolver.py (AST walker for native perch inference) - Integration: packages/dolo/dolo/compiler/model.py (calls resolver for adc-stage dialect) - Also added: argmax_{} grammar support mirroring max_{} structure

0.1h — Period instantiation (dict-based, no new classes)¶

Status: ✅ Implemented

Provide a minimal instantiation layer for periods + twisters that: - does not introduce new classes (artifacts are plain dicts + existing SymbolicModel objects), - uses twisters-as-composition (inter-period wiring objects), - uses connectors for within-period wiring, - binds calibration/methodization/settings at the stage level, - reuses existing dolo-plus functor implementations (calibrate, configure, methodize).

Spec: spec_0.1h-periods-nests.md

Related: - Semantic layer: ../syntax-semantic-rules/05-periods-models.md - Theory: ../../theory/ddsl-foundations/periods-nests-theory.md

Period template (timeless):

# Period template (timeless)
name: cons_port_period

# Ordered within-period stage list
stages:
  - cons_stage
  - port_stage

# Connectors only needed when adjacent stage interface names don't match
# (identity connector is omitted)
connectors:
  - from: cons_stage
    to: port_stage
    rename: {a: k}

0.1i — Functions and operators¶

Status: Planned

Option to add or include functions and operators within a stage (for example, define reward or Q functions as not just string replacements).

0.1j — Run horse solver¶

Status: Planned

Feed the translated horse into dolo.yaml_import(...) + a solver (EGM) and capture outputs.

Spec: spec_0.1d.md

0.1k — AST Resolution Checks and Finalization¶

Status: Planned

Document and finalize the structure of the dolang AST in context of Backus-Bellman foundations and Bellman equation formalization. Includes wiring validation/resolution checks into the parsing pipeline.

Key outstanding items (see todo.md for full list): - Resolution check for @in X decorators (validate X is in registry or symbols.spaces) - Wire registry into strict parsing mode - Finalize AST node types for perch-tagged expressions

0.1l — Perch semantics¶

Status: Planned

Perch-indexed symbols as function evaluations.

Files (current "source of truth" set)¶

File	Description
`spec_0.1a.md`	Parse-only milestone
`spec_0.1b-doloplus-to-dolo-translator.md`	Translator to vanilla Dolo
`spec_0.1c.md`	Decorators/calibration/methods
`spec_0.1d.md`	Methodization functor / Run horse solver
`spec_0.1f.md`	VFI control bounds
`spec_0.1g-bare-symbol-resolution.md`	Bare symbol resolution
`spec_0.1h-periods-nests.md`	Period + nest instantiation
`spec_0.2a-compiler-pipeline-architecture.md`	Compiler pipeline architecture (Lean 4-inspired)
`ambiguities.md`	Known ambiguities and resolutions

| todo.md | Outstanding tasks |

Example stage files: - packages/dolo/examples/models/doloplus/consumption_savings_iid_egm_doloplus/stage.yaml - packages/dolo/examples/models/doloplus/port-with-shocks/library/ (cons_stage, port_stage, noport_stage)

Explore scripts (explore/stage-pipeline-demos/): - stage_explorer.py — inspect stage files - stage_pipeline.py — run methodize/calibrate/configure pipeline - stage_pipeline.ipynb — notebook demonstrating accretion approach - egm_cons_stage_pipeline.py / .ipynb — EGM cons-stage pipeline demo - parse_doloplus_models.ipynb — parse dolo-plus models demo - spec_0.1c_demo.ipynb — calibration/decorator demo

Explore scripts (explore/cons-port/): - noport-cons/lifecycle_cons_period.py — cons-only period lifecycle - noport-cons/lifecycle_cons_port_period.py — cons-port period lifecycle - noport-cons/lifecycle_port_cons_period.py — port-cons period lifecycle - noport-cons/semiconjugate_solve.py / .ipynb — semiconjugate solve demo - noport-cons/lifecycle_cons_period_walkthrough.ipynb — cons period walkthrough notebook - port-cons/solve_port_cons.py — solve port-cons model - port-cons/solve_port_cons_ffp.ipynb — FFP accretive solve (EGM vs VFI, Matsya de-sugaring)

Conceptual overview: - Meaning map to code (Υ → Ρ) — how V[>] de-sugars into a callable, the whisperer pattern, and method independence

Naming convention: label vs symbol¶

Label: the YAML key under equations: (human-facing; editable).
Symbol: the stable identifier used by dolo-plus IR and translation (e.g. g_ad, T_ed), recorded in metadata as:
dolo_plus.equation_symbols: {<label>: <symbol>, ...}

Perch-tag indices¶

In dolo-plus stage-mode equation payloads we keep perch tags like [_arvl], [_dcsn], [_cntn]. They are rewritten only in the interface step (0.1b/0.1) when producing a Dolo/dolang-compatible horse representation.

Overview

dolo-plus (ADC dialect) — spec_0.1 overview (exploratory)¶

Hard constraint: minimal/no changes to vanilla Dolo¶

Target outcome (end of spec_0.1)¶

Milestone split¶

0.1a — Parse-only (no dolang/Dolo interface, no compilation)¶

0.1b — Interface/translation to "old dolang/Dolo"¶

0.1c — Decorators + calibration + methods (focus: parameter registration)¶

0.1d — Methodization functor¶

0.1e — Back-translation (horse outputs → ADC outputs)¶

0.1f — VFI control bounds via ⊥ declarations¶

0.1g — Bare symbol resolution (native perch inference)¶

0.1h — Period instantiation (dict-based, no new classes)¶

0.1i — Functions and operators¶

0.1j — Run horse solver¶

0.1k — AST Resolution Checks and Finalization¶

0.1l — Perch semantics¶

Files (current "source of truth" set)¶

Naming convention: label vs symbol¶

Perch-tag indices¶

0.1f — VFI control bounds via `⊥` declarations¶