Skip to content

Related

Foundations: Periods & Nests Theory, Core DDSL Concepts

Semantic Rules: 05 Periods & Models, 01 Variable Declaration, 02 Stage Structure, 03 Equations

Literature: Backus 1978 — Can Programming Be Liberated from the von Neumann Style?

Spec 0.1h — Period and nest instantiation

Abstract

This specification defines a minimal instantiation representation: plain Python dicts together with existing stage objects for periods and multi-stage problems (nests) using DDSL. Period and stage templates are timeless; any "time indexing" is derived from position in the nest and appears only at the inter-period composition. It also describes an optional abbreviated notation for constructing finite-horizon models as a nest—an ordered family of period instances wired by composed by twisters. A nest is not a list of template names or references; it is defined and instantiated as a collection of stages (with or without calibration/methodization/settings attached). #definitions/stage #RCRK

Ambiguities / open questions (from notes + discussion)

This spec is intentionally minimal. The items below are unresolved design choices surfaced by the running #ambiguity / #todo notes in this spec and recent design discussion. They should be tracked and resolved before declaring the representation "locked".

  • Reference granularity (template vs instance): should surface syntax ever name period instances / stage instances, or remain purely positional (index-by-list-position)? (#ambiguity #naming-stages)

    [!tip] Current decision Periods are indexed, stages are named instances of string expressions or SymbolicModel class. #decision-point

  • Period namespace semantics ("fields" vs string names): is the period namespace merely a set of unique strings, or a mapping from economic "fields" to string identifiers? (see §3.1 remark)

    [!tip] Current decision Period namespaces are economic fields (or we can informally say variables). This allows us to tie in syntax to pure math. #decision-point

  • Time/index notation in twisters: should twisters allow/encourage explicit \(t, t+1\) subscripts (e.g. {k[t+1]: a[t]}) even though time is otherwise implied by order? What is the canonical direction/order-of-twisting convention? (#ambiguity, #order-of-twisting)

    [!tip] Current decision Current implementation does not have time-indexing. Further implementation should. #ddsl-dev-todo #decision-point

  • Connector vs twister terminology: keep distinct names or unify under a common "adapter" concept (e.g. adapter_intra / adapter_inter) to signal structural similarity? (#ambiguity)

    [!todo] Action needed Implement adapter_inter and adapter_intra or similar. #ddsl-dev-todo

  • Twister direction + list alignment: confirm the canonical convention for which boundary a twister belongs to, whether twisters[i] maps periods[i] → periods[i+1] (forward time), and how this relates to backward accretion index \(h\).

    [!tip] Current decision (undecided) We backward accrete so Period[H], [connector], position can be just a raw index and understood as sitting between H and H-1. Periods are indexed by integers from horizon. We may suppose connectors can be indexed forward so. #decision-point/undecided

  • Nest container shape ("wide vs long"): two parallel lists (periods, twisters) vs an interleaved representation; pick the canonical shape that makes period↔twister association unambiguous. (#ambiguity near §6.4)

    [!tip] Proposed Two separate lists.

  • Connector dict schema / orientation: define succ/pred (or from/to) precisely, including rename-map direction and identity conventions. (#succ-pred)

    [!todo] Action needed

    ddsl-dev-todo

  • Continuation wiring + solver API boundary: standardize the orchestrator-level contract for terminal conditions + successor-solution wiring (including whether/when a twister is "applied" to a continuation value) and the minimal interface expected of solve_fn.

    [!todo] Action needed

    ddsl-dev-todo

  • Beyond v0.1 sequential nests (deferred): branching / non-adjacent wiring will require explicit instance identifiers and a non-positional wiring representation.

    [!todo] Action needed

    ddsl-dev-todo

A central ambiguity addressed here is reference granularity: when (if ever) the surface syntax in periods and nests should be allowed to refer to specific period instances or stage instances, as opposed to remaining purely positional (index-by-list-position). The approach we take is to not provide a syntax for connecting named stages or assigning names or indices to stages or periods to construct a nest, but only provide rules for composition with which arbitrary stages can be composed. #ambiguity #naming-stages

naming of connectors vs twisters #ambiguity ie twisters inter and twisters intra?

1. Objective and scope

Goal: Provide a minimal representation layer for periods and nest syntax that:

  • introduces no new period/model classes: instantiation artefacts are plain dicts together with existing stage objects (in this representation kit, dolo.compiler.model.SymbolicModel with functor attachments);
  • represents inter-period composition by twisters (composition adapters); #definitions/connectors
  • represents within-period composition via a common name space and when required by rename connectors; #definitions/connectors
  • attaches methodization, configuration, and calibration at the stage level (consistent with current dolo-plus semantics);
  • reuses existing dolo-plus functors (methodize, configure, calibrate) rather than defining a separate attachment representation; and
  • treats a period as a common namespace for random-variable names (economic fields), so that composition is implicit when names coincide. #definitions/economic-fields

Syntax directives vs RCRK builder conventions

The DDSL project distinguishes three layers (see ToR terminology.md and scope.md):

  1. Syntax directives: YAML shapes, BNF rules, keywords, and validity constraints that define what can be written—period templates, nest templates, connectors, twisters.
  2. Model representation / IR: "What the computer sees before numbers"—a language-specific thin and intermediate representation that mirrors the syntax. The DDSL IR for stages is given by the dolo+ symbolic model object. This document expresses the IR for the periods and nests.
  3. RCRK (Track B, Output O5): The Reference Computational Representation Kit—a set of language-specific intermediate model representations (here: Python dicts + SymbolicModel objects) that demonstrate how DDSL syntax maps to solvable structures. The RCRK includes builder conventions: how an orchestration layer instantiates templates into a runtime nest—loading stages, applying functors (calibrate, methodize, configure), computing position-dependent parameters from the accretion index, and assembling period instances. The RCRK does not include complete computational solvers.

This document specifies internal model representation of periods and nests and builder conventions at the RCRK layer expressed via examples; the upstream syntax directives are in docs/specs/dolo-plus-v0.1/syntax-semantic-rules/.

Design principle: thin composition syntax, rich builder

Following Backus's vision of an "algebra of programs" built from a small set of functional forms (combinators) rather than elaborate procedural machinery (Backus 1978), the DDSL period/nest syntax specifies only a composition structure for stages. All orchestration logic—loading templates, applying functors, computing age-dependent parameters, interpreting the model economically, assembling the runtime nest—lives in the builder layer, not in the syntax. This separation keeps the declarative layer "what the state space is" cleanly distinct from "how operators compose and instantiate".

2. Background and constraints

This specification is downstream of the semantic layer:

  • docs/specs/dolo-plus-v0.1/syntax-semantic-rules/05-periods-models.md
  • docs/theory/ddsl-foundations/periods-nests-theory.md (composition graph + twisters-as-isomorphisms)

Design constraints:

  • No new class system for periods/models.
  • Prefer backward accretion: construct a nest by incrementally adding periods and twisters, working backward from terminal.
  • Do not cache grids or large numerical objects in the representation; solvers should construct them on demand.

3. Core objects and invariants

3.1 Periods

A period is: 1. A namespace \(\mathcal{X}\): a set of declared random variables (~perches/nodes) with unique symbolic "names" in the form of strings. #definitions/perch 2. A stage list \([S_1, \ldots, S_n]\): stages as edges between nodes in \(\mathcal{X}\).

Ambiguity: should we think of the name space a mapping from fields to strings? To be resolved. #backus-functionals #definitions/economic-fields

For v0.1 (sequential examples), we deal with the stages that are ordered. More general within-period graphs (branching/merging) are deferred to later versions.

Composition rule: Within a period, stages compose by sharing a common node. Suppose \(x\) is the arvl node of \(S_1\) and \(a\) is the cntn node of \(S_1\). We can represent this stage as a directed edge in a graph: $$ a \xrightarrow{S_1} x $$ Now suppose \(x\) is the cntn node for \(S_2\) and \(b\) is the arvl node. The two stages compose when the continuation of \(S_2\) matches the arrival of \(S_1\):

\[ b \xrightarrow{S_2} x \xrightarrow{S_1} a \]

In this case, no connector is required: the shared node \(x \in \mathcal{X}\) constitutes the wiring. Composition is implicit from the common namespace. A connector is declared only when names differ and must be remapped. For example, if \(S_2\) outputs to node \(y\) but \(S_1\) arrives from node \(x\), a connector provides the rename \(y \mapsto x\).

When the stages in a period are ordered, the period boundary is straight-forward:

  • Entry: arrival node of \(S_1\) (first stage in the period's stage list)
  • Exit: continuation node of \(S_n\) (last stage in the period's stage list)

When the stages are not ordered, the period boundary can be defined by the set of arrival nodes of all nodes with in-degree 0 (entry) and terminal nodes (exit).

However, defining entry and exit nodes is necessary, because twisters between nests refer to variable names within a period after the intra-period renaming has taken place.

3.2 Nests

At runtime, an instantiated model is a nest: a finite-horizon structure of period instances (including any within-period connectors) and inter-period twisters.

\[ \mathcal{N}^{H} = \bigl( \{P_h\}_{h=0}^{H}, \{\tau_h\}_{h=1}^{H} \bigr) \]

where: - \(P_h\) is the period at distance \(h\) from terminal (\(P_0\) is terminal) - \(\tau_h: P_h \to P_{h-1}\) is the twister at the boundary, mapping continuation objects of \(P_h\) into arrival objects of \(P_{h-1}\) (forward time; note that \(h\) counts backward from terminal)

The nest is built backwards: we construct \(P_0\) first, then \(P_1\) with twister \(\tau_1\), and so on. This mirrors the structure of backward induction.

Constraint: integer/time indicators live only at the inter-period layer (twisters and the orchestration loop index). Stage templates and period templates are timeless.

Indexing convention: a period instance does not "know" its time index. The index is inferred from its position in the ordered nest, e.g. \(h = 0\) is terminal and \(h\) increases as one accretes backward from terminal.

3.3 Wiring terminology: connectors vs twisters

  • Connector: wiring within a period (stage-to-stage). A connector is an adapter that resolves naming mismatches between adjacent stages in the period sequence.
  • Connectors refer to *perch state variable names within a period
  • Identity connectors are implicit: if names line up, omit the connector entirely.
  • Twister: wiring between periods (period-to-period). A twister is an adapter that maps one period's continuation objects (terminal state/value names) into the next period's arrival objects.
  • In v0.1 sequential nests, twisters are aligned by position: twisters[i] connects periods[i] → periods[i+1] (forward time).
  • Therefore a twister needs only a rename map (possibly empty for identity). It does not carry from/to fields.

We can think of connectors and twisters as self-maps on the symbol space on which fields (the state-variables) are defined. #definitions/connectors

3.3.1 Twister as composition (foundational principle)

Following the DDSL foundations (see docs/theory/ddsl-foundations/periods-nests-theory.md), composition is determined by interface compatibility of random variables: two stages/periods compose iff the "exit" r.v. of the first is the same as the "entry" r.v. of the second (possibly up to an isomorphism / rename twister).

For a stage \(S\) and a perch label \(p \in \{\text{arvl}, \text{dcsn}, \text{cntn}\}\), define the perch interface operator

\[ X_{p}(S) := (v_1,\ldots,v_k), \]

the ordered tuple of random-variable identifiers (economic field names) comprising the stage's interface at perch \(p\). In the foundations graph, \(X_p(S)\) is the random-variable node at perch \(p\).

In the absence of an explicit adapter, two stages compose iff the continuation interface of the first equals the arrival interface of the second:

\[ \langle S_1, S_2 \rangle \text{ is valid } \iff X_{\text{cntn}}(S_1) = X_{\text{arvl}}(S_2) \]

An explicit adapter is inserted when names differ but the interfaces are isomorphic:

\[ \langle S_1, \tau, S_2 \rangle,\qquad \tau: X_{\text{cntn}}(S_1) \xrightarrow{\sim} X_{\text{arvl}}(S_2). \]

Equivalently, \(\tau\) is a bijective rename map on identifiers such that \(\tau(X_{\text{cntn}}(S_1))=X_{\text{arvl}}(S_2)\).

The composition is the twister.

Variable names are structural, not cosmetic. In the foundations view they denote random variables or economic-fields*, not placeholders; they are the primitive nodes of the composition graph (see "The Key Insight" in periods-nests-theory.md). #graphs

Time in twisters? time only appears here #ambiguity python {k[t+1]:a[t]}

Remark (MDP guarantee): under the foundational theory (see "The MDP Guarantee" in periods-nests-theory.md), if stages/periods form a well-typed composition graph, they define a valid MDP; for finite horizons, backward induction is the appropriate evaluation procedure.

Further work to define what is a valid set of connectors and twisters that define a well-typed multi-stage MDP.

Remark (implicit indexing): during backward accretion, the partially constructed nest carries its own construction depth. If \(h\) periods have been accreted from a terminal age \(T\), the current period corresponds to age \(T-h\). Position-dependent parameters (mortality, income profiles) are computed by the orchestration layer from this implicit index; the stage template remains time-invariant and is not passed \(h\) or \(T\) as an explicit argument.

4. Surface syntax (templates and abbreviated nest notation)

4.1 Period templates

This graph-first view suggests DDSL syntax should:

  1. Declare random variables explicitly before stages
  2. Define stages as edges referencing those r.v.s by identity
  3. Composition is implicit from the graph structure

Within a period, the default is a common namespace for random variables. Stage composition is therefore determined by the stage list together with random-variable names; a connector is required only when an explicit rename is necessary.

# Period template: cons_port_period (cons → port)
# Stage sequence (within a period): cons → port
#
# Intra-period wiring requires a rename:
# - cons outputs a
# - port expects k
# so we need an internal connector: k = a
# Note there is no name here 

stages:
  - "cons_stage": {!stage}
  - "port_stage": {!stage}

# Connectors are needed only when names don't match.
# If cons output matched port input, this block would be omitted entirely.
connectors:
  - from: cons_stage
    to: port_stage
    rename: {a: k}

The syntax !stage denotes a stage instance, while "cons_stage" is name for that instance. Note that the name space for stage names is within a period. !stage can be a reference to a yaml file or, in a dict, the symbolic model object or set of expressions itself.

todo #grammar #stage-instances

Return to this: in the surface language, each element of stages: denotes a stage occurrence within a period occurrence, not merely a template symbol. If a stage template can repeat within a single period, we need an explicit grammar-level term for a stage instance (e.g., a positional reference or an explicit instance id) so that connectors (from/to) and methodization targets can refer unambiguously to "the \(i\)-th occurrence of cons_stage". #syntax-directive #ambiguity

The graph structure emerges from the stage list plus the common namespace. Connectors are adapters for mismatches, not declarations of the graph itself.

4.2 Nest templates (abbreviated notation)

The purpose of "nest syntax" is to provide a declarative description of a model as a composition of period templates composed by twisters, while keeping the runtime object as an instantiated collection of PeriodInstance dicts containing bound stages.

4.2.1 Minimal nest template: list of period types

In the sequential finite-horizon case, a nest can be declared by listing period templates:

name: lifecycle_cons_example

# A list of period template names (types). Repetition is allowed.
periods:
  - !period
  - !period
  - !period
  - !period

# A list of twister renames aligning with adjacent period boundaries.
# twisters[i] connects periods[i] → periods[i+1] (forward time).
twisters:
  - {a: k}
  - {a: k}
  - {a: k}

todo #grammar #stage-instances

Return to this: in some uses the periods: list is intended to denote period instances (not merely template types). If so, the surface language needs an explicit term for stage instances (a stage occurrence within a given period occurrence), so that syntax and methodization targets can refer unambiguously to "the \(i\)-th occurrence of cons_stage" rather than to the template name alone. #syntax-directive #ambiguity

Note: in the surface syntax, an element of twisters: may be written as a bare rename map (e.g. {a: k}); the builder wraps this into a twister dict under the rename key. #syntax-directive

This is an abbreviated notation: the strings cons_period are not the model. They refer to templates in a syntax collection. The model is the instantiated nest in which each slot contains bound stages.

An optional abbreviation (not required for v0.1) is a repeat: form:

periods:
  - period: cons_period
    repeat: 4
twisters:
  - {a: k}
    repeat: 3

4.2.2 Instantiated nest (runtime): period instances + twisters

The orchestration layer expands each period template occurrence into a PeriodInstance by attaching stage objects:

  • the period template provides only stages: [...] (and optional connectors)
  • the instance provides stage objects keyed by stage name (with stage names scoped to the period instance)
  • methodization, configuration, and calibration are attachments on the stage objects and may differ by slot

Importantly: the period template name is not a time index. A list like [cons_period, cons_period, ...] becomes a list of period instances whose stage objects (and their attachments) may differ by index/age/horizon, even if they share the same template.

Stage names are scoped to a period instance: the same stage name (e.g. cons_stage) may appear in every period, and it is resolved relative to that period's stage mapping. Consequently, this layer does not introduce stage_1, stage_2, … identifiers to distinguish stages across periods.

4.2.3 Referencing rule (the key ambiguity)

There are two distinct kinds of identifiers in play:

  • Template names (timeless): cons_period, port_cons_period, cons_stage, …
  • Instance positions (runtime): in v0.1 sequential nests, period instances are identified by position in the ordered list, not by an explicit identifier field.

todo #syntax-directive #periods #nests

Formalize in the syntax directives: period instances are not named in the v0.1 sequential case. This is admissible because the model is an ordered list; the time/period index is the position in that list. Any grammar that permits references to "a period" must therefore specify whether it refers to a template name (type) or to a particular occurrence (by position). See [[0.4-periods-models]] and [[1.2-stage-structure-and-timing]].

For v0.1 sequential nests, we adopt:

  • nest = ordered lists (periods: [...], twisters: [...])
  • twisters align by position, so they do not need from/to identifiers

For later (branching graphs / non-adjacent wiring), positional alignment will be insufficient; explicit instance identifiers and a corresponding non-positional wiring representation will be required (to be specified). The same issue exists within a period if stage templates are repeated: referencing by stage name becomes ambiguous and stage-instance identifiers or positional references may be required. This is an open syntax decision (see ambiguities.md).

5. Stage attachment pipeline (stage-local functors)

This specification reuses the existing dolo-plus functors for methodization, configuration, and calibration:

from dolo.compiler.model import SymbolicModel
from dolo.compiler.calibration import calibrate as calibrate_stage
from dolo.compiler.calibration import configure as configure_stage
from dolo.compiler.methodization import methodize as methodize_stage

These functors are designed to be treated as pure:

  • each returns a new stage object (via copy.copy)
  • the input stage object is not mutated

Critical semantic point: attachments are stage-local

In dolo-plus, calibration and settings are not period-level or model-level global dictionaries. They are attachments on a stage object:

  • stage.calibration is the calibration bound for that stage instance
  • stage.settings is the settings bound for that stage instance

You may source calibration/settings from a shared YAML, but the DDSL contract is: - the orchestration layer attaches them stage-by-stage, and - once a stage object is placed into a nest, its attachments are treated as immutable.

The attachment pipeline

SymbolicModel  →  [methodize ONCE]  →  [configure ONCE]  →  [calibrate ONCE]
Important distinction: A YAML stage template is the file-based source artifact; SymbolicModel is the corresponding parsed syntax object. The SymbolicModel is already a well-formed syntactic object (declared variables, operators, transitions), but has no numerical methods/settings/parameter values until the functors attach them. The resulting stage object is then placed into the corresponding period instance. The calibration functor receives position-dependent values computed by the orchestration layer.

6. Runtime representation shapes

6.1 Period instance (in-memory dict)

A period instance is a bound, instantiated period ready for solving:

Period instance semantics

A period instance is not a new semantic object. It is simply:

  • a collection of stage objects (plus an ordering, for v0.1 sequential periods), and
  • optional connectors (rename adapters) used only when stage interface names do not match.

A stage object in turn is a syntactic representation of DDSL YAML (symbols + perches + equation blocks). At this layer we do not compile executable functions; we assemble structured syntax objects that a numerical solver backend can later interpret.

Scoping rule: stage names are scoped to a period instance. A stage name has meaning only within the period's stage mapping. To identify a particular stage instance in a nest, one uses the pair (period_index, stage_name).

Representation note: stage objects are stored in the period instance dict under their stage names. Stage names are interpreted relative to a period instance and may be repeated across periods. To avoid ambiguity, stage names must not collide with reserved keys such as connectors. Stage order within a period is determined by the associated period template's stages: [...] list (and is therefore not repeated in the period instance dict).

Within-period compatibility rule: adjacent stages must have compatible interface perches (continuation of stage \(i\) matches arrival of stage \(i+1\)) under the naming rule. If names differ, a connector supplies an explicit rename; otherwise composition is blocked.

PeriodInstance = {
  # optional: within-period connectors
  "connectors": [Connector, ...],
  # stage objects (keys are stage names; scoped to this period instance)
  "port_stage": <SymbolicModel>,
  "cons_stage": <SymbolicModel>,
}

6.2 Connector dict (within-period wiring)

Connector = {"succ": "cons_stage", "pred": "port_stage", {"a": "k"}}

**NEED TO HERE DEFINE SUCC. AND PRED. (SEE BELLMAN-DEV OR DYN-X). #succ-pred Remarks:

  • Connector identity is omitted when names match (canonical rule).

6.3 Twister dict (between-period wiring)

Twister = {
  # The twister list is aligned with period boundaries:
  # twisters[i] connects periods[i] → periods[i+1] (forward time).
  "rename": {"a": "k"},
}

Design note: twisters are wiring objects (composition adapters); they are not part of the economic problem statement. Discount factors, if present, belong in stage mathematics rather than in twister metadata.

6.4 Nest dict (the complete structure)

NestDict = {
  "periods": [PeriodInstance, ...],             # forward-time order
  "twisters": [Twister, ...],                   # twisters[i] connects periods[i] → periods[i+1]
}

The index semantics are carried by position in the periods list. Period instance dicts do not carry a time index.

NestDict[-1][]

ambiguity wide or long order of pairing periods and twisters?

7. Helper API (pure functions returning dicts)

The reference implementation exposes a small set of pure helper functions in dolo.compiler.instantiation. These functions construct dicts and introduce no new classes.

NONE OF THE BELOW ARE REQUIRED

from dolo.compiler.instantiation import (
    load_period_template,
    make_connector,
    make_twister,
    instantiate_period,
    make_model_dict,
    validate_model_dict,
)
NONE OF THE ABOVE ARE REQUIRED

TODO: conventions of how access the solution

8. Builder algorithm: backward accretion

This section presents orchestration pseudocode. It illustrates how host-language code expands templates into period instances and then accretes a nest backward from terminal.

8.1 Load a stage YAML template (construct a syntactic stage object)

from pathlib import Path
import yaml
from dolo.compiler.model import SymbolicModel
from dolo.compiler.calibration import calibrate as calibrate_stage, configure as configure_stage
from dolo.compiler.methodization import methodize as methodize_stage

# Load a YAML **template** from the library → construct a syntactic stage object (timeless)
stage_library = Path("library")
stage_path = stage_library / "cons_stage.yaml"
stage_yaml = yaml.compose(stage_path.read_text())
cons_stage = SymbolicModel(stage_yaml, filename=str(stage_path))

8.2 Attach methods/settings and calibrate a stage object

cons_stage = methodize_stage(cons_stage, "methods.yml")
cons_stage = configure_stage(cons_stage, "settings.yaml")
cons_stage = calibrate_stage(cons_stage, {"gamma": 2.0, "beta": 0.96})

8.3 Build the nest backwards

8.3.1 Templates

We can store a bunch of stage and period templates in a common library. A period template can be:

# Period template: cons_port_period (cons → port)
# Stage sequence (within a period): cons → port
#
# Intra-period wiring requires a rename:
# - cons outputs a
# - port expects k
# so we need an internal connector: k = a

name: cons_port_period

stages:
  - "cons_stage": {!stage}
  - "port_stage": {!stage}

# Connectors are needed only when names don't match.
# If cons output matched port input, this block would be omitted entirely.
connectors:
  - from: cons_stage
    to: port_stage
    rename: {a: k}

In this example, the same period template is used for each period occurrence in the nest. The stage occurrence name is taken to coincide with the stage name used to locate a YAML template in the library; this is a builder-level convention rather than a requirement of the DDSL.

Consistent with the Backus-style emphasis on composition of small transformations, we isolate the I/O boundary in load_stage_syntax(...) and keep period assembly in make_my_period(...). For clarity, the orchestration below constructs SymbolicModel objects inside the horizon loop (