COMPUTATIONAL FRAMEWORK CONCEPTS¶

*This document provides supplemental guidance for computational implementations of the mathematical framework described in unified.md.

Purpose and Scope¶

While unified.md defines the mathematical structure of the framework, this document bridges the abstract mathematical concepts with practical computational representations by outlining high-level computational objects needed for implementation.

Naming Conventions¶

Throughout this document and the framework implementation, the following naming conventions are used consistently:

Perch Naming: Perches are always referred to using the pattern perch_[type] where type is one of:
arvl: Arrival perch (abbreviated form)
dcsn: Decision perch (abbreviated form)
cntn: Continuation perch (abbreviated form)
Distribution Abbreviation: Population distributions are always abbreviated as dstn:
Variables: dstn_arvl, dstn_dcsn, dstn_cntn
Methods: get_dstn(), update_dstn()
Type names: Dstn rather than Distribution
Function Names: Function names use abbreviated forms for consistency:
dcsn_rule() rather than decision_rule()
value_function() (standard form)
transition_function() (standard form)
Parameter Names:
Perch parameters: params_perch
Mover parameters: params_mover

These conventions ensure consistency throughout the codebase and documentation, making the framework easier to understand and maintain.

Package Overview¶

The MBDP package provides a unified interface to multiple backend tools (referred to as "horses") for economic decision-making problems. The package:

Feeds configuration files to appropriate whisperers, which reformat them into the structure expected by horses for implementation
Manages the creation and interconnection of computational stages and periods
Provides consistent APIs for solving and simulating models
Facilitates interoperability between different backend solvers

Version Notes¶

The current implementation focuses on the monolithic solution approach, where the whisperer orchestrates the entire solution process. The multilithic approach, which allows individual movers to perform backward operations, is preserved as a future feature and may be reimplemented in later versions for compatible backends.

Computational Object Organization¶

The framework organizes computational elements into perchInstance, mover_spec, stageInstance, periodInstance, and modelInstance objects that efficiently implement the mathematical structures defined in unified.md.

perchInstance and mover_spec: Core Computational Objects¶

perchInstance structure¶

A perchInstance is the computational object that stores information about a perch:

Content:
State space definition (variables, domains)
- As represented by the horse in consultation with the whisperer
- e.g., a dolo yaml config file might define a domain: as w: [0.01, 4.0]
  - (for the approximated state space of the dcsn perch of a stage)
- The whisperer would be able to retrieve this from the config file
- Once retrieved, it would be represented as-is in the perchInstance
  - perhaps in perch.domain_aprx
  - because the true problem would probably have a domain like R++
  - which could be represented in perch.domain_true
- boundaries are not part of the perch
- they are used by the mover in constructing the perch
Metadata about functions (which functions are defined for the perch type)
- Each function and its signature should be obtained from the .comp attribute
- and represented as-is on the perch
Population measure information (in simulation mode)
- Again, just a copy of the horse's population measure
Implementation details (representation methods, parameters, etc)
- Again, as retrieved from the .comp object(s) created by the whisperers
Properties:
params_perch: Economic model parameters 𝔭 (risk aversion, etc) that define behavior
numeric_methods: Methods for representing/approximating functions numerically
- like interpolation method (spline, neural, polynomial)
Computational Objects (.comp): Actual numerical implementation when solved
- Contains backend-specific numerical functions (TensorFlow/PyTorch/JAX/Dolo):
- Value functions (𝒜, 𝒱, or ℰ) depending on the perch type
- Policy functions π (for decision perches)
- Other backend-specific computational implementations
- Direct exposure of backend-specific functions and methods
- Requires properly formatted state inputs for evaluation

To evaluate functions (e.g., policy or value functions) stored on a perch:

# Example: Evaluating a decision rule at a specific state
# 1. Construct a properly formatted state representation for the specific backend
formatted_state = # a representation of the state in dolo format
# 2. Directly access backend-specific function from .comp
decision = perch.comp.dcsn_rule(formatted_state)

So, we can speak of the arvl perchInstance containing all the information required to computationally instantiate the arrival perch.

mover_spec structure¶

A mover_spec is a computational object that contains the mathematical specification for transitions between perches. The mathematical content that a mover_spec holds—the transition functions and pushforward formulas defined in the Recursive Problem Definition and Population Dynamics sections—is what we call the mover-spec for that transition.

Content:
Metadata about transition operators (which transition operators are defined)
Information about the connected perches
Parameter references relevant to transitions
Required Properties:
params_mover: Model params_mover relevant to the transition
numeric_methods: Numerical methods to be used for transition functions
Computational Objects (.comp): Actual numerical implementation when solved
- Contains backend-specific numerical implementations of:
- Forward transition operators (such as gₐᵥ, gᵥₑ, etc.) used in simulation
- Population transition operators (Γ)
- Backward operators (optional, for solution in multilithic mode)
Key Characteristics:
Backward operators are optional since the whisperer can also generate the functions stored on perchInstance directly from the model configuration
When provided, backward operators offer an alternative computational pathway for generating perchInstance functions during the solution process
Forward transitions are always required for simulation

The .comp attribute contains backend-specific implementations of transition functions:

# Example: Using a transition function
# 1. Format state appropriately for the specific backend
formatted_state = mover.comp.format_state(current_state)
# 2. Directly access the backend-specific transition function
next_state = mover.comp.transition_function(formatted_state, action)

Stage and Period Organization¶

Stage: Each stage consists of:
Three perchInstance in arvl, dcsn, cntn order
In multilithic mode, also includes two mover_spec objects:
- arvl-to-dcsn mover_spec (connecting arvl and dcsn perches)
- dcsn-to-cntn mover_spec (connecting dcsn and cntn perches)
In a sequential model there are exactly two mover_spec objects: arvl-dcsn and dcsn-cntn
In a branching model there will need to be a mover_spec connecting the dcsn perchInstance to each possible successor
Period:
In a sequential model, an ordered collection of stages with corresponding inter-stage mover_spec objects
In a branching model, there is no predefined order in which periods will be executed during simulation

This organizational structure maintains separation between mathematical concepts and computational details while enabling efficient implementation of both solution and simulation algorithms.

State Tracking with Boolean Flags¶

Throughout the implementation, each perchInstance, mover_spec, and stageInstance maintains boolean flags to track its state:

perchInstance Flags - has_params: Whether params_mover have been added - has_numeric_methods: Whether numeric choices (interpolation method, e.g.) have been configured - has_comp: Whether computational/numerical instantiation is attached - This flag indicates whether backend-specific functions are available for direct access

mover_spec Flags - has_params: Whether params_mover have been added - has_numeric_methods: Whether computational numeric_methods have been configured - has_comp: Whether computational objects are attached - This flag indicates whether backend-specific transition functions are available for direct access

stageInstance Flags - has_empty_perches: Whether the stageInstance has just been created with empty perches - has_portable_model: Whether all perches and movers have been created and populated with math - movers_backward_exist: Whether movers have backward operators (multilithic mode only) - has_whisperer: Whether a whisperer has been associated with the stage - is_portable: Whether the stageInstance can be saved (serialized) without computational objects - is_solvable: Whether the stageInstance has perch_cntn.comp (terminal condition) - is_solved: Whether the stageInstance has been solved (all perches have .comp objects) - is_simulated: Whether the stageInstance has been simulated (perches have .sim objects)

These flags help track readiness for different operations and validate operations before they are performed.

periodInstance Flags - has_whisperer: Whether a whisperer has been associated with the period - has_stages: Whether any stage(s) have been added to the period - is_connected: Whether all inter-stage connections have been established - is_solvable: Whether all constituent stageInstance instances are solvable - is_solved: Whether all constituent stageInstance instances have been solved - is_simulated: Whether all constituent stageInstance instances have been simulated

These flags help track the state of period construction and readiness for operations.

connectorInstance Flags (proposed) - contract_satisfied: Whether the connector's pre/postconditions are satisfied

Backward and Forward Movers as Processes (Mover-Comps)¶

The mover_spec objects defined above hold the mathematical specification for a transition. The actual computational process that evaluates a mover_spec and creates new objects on perches is a mover-comp (𝙼← for backward, 𝙼→ for forward).

Period-level backward mover (𝙼←_prd): The whisperer's solve_stage() method acts as the period-level mover-comp. It takes the mover_spec objects for a stage, uses the connector mapping, and constructs/populates the arrival value function and policy function on the stage's perches—working backward in time.
Stage-level mover-spec (𝙼←_stg): Each within-stage mover_spec object (arvl-to-dcsn, dcsn-to-cntn) provides the mathematical information for a single within-stage transition. The whisperer uses these to implement the stage-level backward operations.
Forward movers (𝙼→): During simulation, forward movers use the same mover_spec transition functions (gₐᵥ, gᵥₑ) and the pushforward operator (Γ) to propagate population distributions forward in time.

Connectors as Computational Objects¶

A connector links stages or periods by mapping the namespace of one side to the other.

Mathematical connector (𝒞): A pure namespace mapping (e.g., 𝒞(k_{t+1} ↔ a_t)) defined at model-specification time. This is part of the model's mathematical description.
Computational connector (𝙲): The runtime object carrying the mapping plus bridging metadata. Once created, a connector is passive—it does not hold value functions or distributions.

In the proposed design, a computational connector 𝙲 may hold a copy of the backward and/or forward mover code (from its original source) as a convenience for inspection. This is not yet implemented.

Stage Lifecycle: Construction, Solution, and Simulation¶

Instance Creation Overview¶

The framework uses a specific pattern for creating the variousInstance types:

stageInstance Creation: A stage is created by providing configuration information...
mover_spec Creation:
```
mover = Mover(<required_args>[, optional_args])
```
Required arguments:
- source and destination perch info (from paths to configs)
  - path to the config/movers/[stagename] directory
- config/movers/[movername] directory
perchInstance Creation:
Empty perchInstance objects can be created directly:
```
perch_arvl = Perch(<required_args>[, optional_args])
```
Required arguments:
- path to the math description of the stage of which the perch is a part
- type of perch (arvl, dcsn, cntn)

Additional info: - The computational content (.comp attribute) of perches is populated either by: - The whisperer (in monolithic mode) when it constructs the complete solution at once; or - The backward movers (in multilithic mode) as they solve the problem step by step - Whisperers and movers don't create perches from scratch: - they add or modify attributes of existing perches

This creation pattern ensures that all components are properly mathematized and connected according to the mathematical structure of the model.

Stage Construction Process¶

A stageInstance progresses through the following phases:

stageInstance Creation with Empty Perches:

stage_cons = Stage(stage_path="configs/stages/cons") # for example
# At this point, stage_cons has empty perches

Backend Selection:

stage_cons.set_backend(horse='dolo', whisperer='whisperer_dolo')

Perch Configuration:

stage_cons.configure_perch_params(
    perch_type="cntn", 
    params_file="stage_cons_perch_cntn"
)
stage_cons.configure_perch_numeric_methods(
    perch_type="cntn",
    numeric_methods_file="numeric_methods"
)

Whisperer Invocation (in one of two modes): Either:
Monolithic Mode: python # Solve all perches in the stage using the defined terminal conditions # This is where the actual computational work is done # After solution, all perches have .comp attributes with # backend-specific functions (value functions, policy functions) stage_cons.solve(mode="monolithic") # creates # - perch_dcsn, perch_arvl # - mover.dcsn_to_cntn, mover.arvl_to_dcsn # - in monolithic mode, these do not contain methods for backward solution

# WARNING: While only monolithic mode is currently implemented, the mode parameter # and structure should be preserved to allow for future multilithic implementation. # Removing this parameter or hardcoding monolithic-specific logic would make it # more difficult to reintroduce multilithic capabilities in the future. ```

Model Solution Iteration Starting Point (REQUIRED):

# Every model MUST have a specified starting point for iterations
# For a finite horizon model:
# - this requires definition of the cntn perch for the final period T
# - it may also incorporate a dcsn perch decision rule and value
# - it may also incorporate an arvl perch value function
# For an infinite horizon model:
# - this is an 'initial guess' for the solution of the model
perch_cntn = whisperer.populate_cntn_terminal(perch=perch_cntn)
# The step of adding a .comp attribute makes the stage instance no longer portable
# The .comp attribute now contains backend-specific terminal condition functions

Backward Solution (Required for solving the model):

# Solve the stage instance by working backward from the continuation perch
# This populates .comp attributes on all perches
whisperer.solve_stage(stage)

# After this, all perches have their .comp attribute populated:
# - perch_cntn.comp contains the continuation value function
# - perch_dcsn.comp contains both the decision rule and value function
# - perch_arvl.comp contains the arrival value function

Forward Simulation (Optional):

# Run a simulation with params_mover from a configuration file
...

Whisperer and Horse Architecture¶

The framework uses a flexible backend architecture:

Backend Registry: Backends can be dynamically registered with:

BackendRegistry.register("backend_name", WhispererClass, "Description")

Whisperer Creation: Whisperers are created via the factory:
```
whisperer = WhispererFactory.create(backend_name)
```
Backend Configuration: Each backend may have specific configuration requirements
```
backend_config_path = config.backend_path(backend_name, "model_name")
```

Any computational backend can be supported by registering a whisperer that implements the standard interface.

Backend Registration and Usage¶

To register a new backend implementation:

from bellman.whisperers.registry import BackendRegistry

# Register your backend
BackendRegistry.register(
    "my_backend",
    MyBackendWhisperer,
    "Description of my specialized backend"
)

The framework ships with several pre-registered backends:

# List available backends
available_backends = WhispererFactory.available_backends()
print(f"Available backends: {available_backends}")

Users can select any registered backend:

# Select a backend
backend_name = "dolo"  # or any other registered backend
whisperer = WhispererFactory.create(backend_name)

This architecture allows the framework to remain flexible and extensible, accommodating new computational engines as they are developed.

Canonical Ordering of Components¶

The framework uses a consistent canonical ordering for stage instance components to facilitate algorithmic processing:

Perches:
perch_arvl: Arrival perch (index 0)
perch_dcsn: Decision perch (index 1)
perch_cntn: Continuation perch (index 2)
Movers:
mover_arvl_to_dcsn: Mover from arrival to decision (index 3)
mover_dcsn_to_cntn: Mover from decision to continuation (index 4)
Additional movers for branching stages (indices 5+)

This canonical ordering enables efficient algorithmic processing while maintaining the clarity of named access. The canonical ordering follows the natural flow of information through a stage instance, from arrival to decision to continuation.

Branching Stage Support¶

The framework supports branching stages where agents may transition to different subsequent stages based on decisions or stochastic events:

Period Structure with Branching¶

For models with branching, stages are organized in a Directed Acyclic Graph (DAG) structure:

Definition: Each period maintains a DAG where:
Nodes are stages
Edges represent possible transitions between stages
Requirements:
The graph must be acyclic (no loops)
Each stage must have at least one continuation path
Exit stages have no outgoing transitions

Branching Implementation¶

Branching is implemented entirely through whisperer/horse mechanisms:

Period Construction:

# Create period and stages
period = Period("quarterly")
period.add_stage("consumption", consumption_stage)
period.add_stage("portfolio", portfolio_stage)
period.add_stage("work", work_stage)

# Define stage transitions (branching)
period.connect_stages("consumption", "portfolio")
period.connect_stages("consumption", "work")

Solution Process:
Stages are solved in reverse topological order (exit points first)
The whisperer handles the details of creating branching-aware functions
Each whisperer uses horse-specific methods to implement branching
Simulation Process:
The whisperer distributes agents across branches according to internal decision rules
Branch selection can be deterministic (based on state/action) or stochastic
All branching logic is encapsulated within the horse implementation

Accessing Branching Results¶

Results from branching stages are accessed through standard interfaces:

# Get distribution at each post-branching stage
portfolio_distribution = portfolio_stage.perch_arvl.sim.dstn
work_distribution = work_stage.perch_arvl.sim.dstn

# Aggregate across branches if needed
total_agents = portfolio_distribution.total_weight + work_distribution.total_weight

Branching models are always solved in monolithic mode to ensure consistency across branches.

Hybrid Component Access¶

The framework uses a Stage class that creates stageInstance objects allowing both named attribute access and canonical ordered access...

DAG Structure for Periods with Branching¶

periodInstance objects containing multiple stageInstance objects are organized in a Directed Acyclic Graph (DAG) structure...

The whisperer is responsible for creating computational objects and attaching them to perchInstance and mover_spec objects...

The whisperer uses stageInstance configuration to create computational objects... The whisperer uses stageInstance configuration to create computational objects...

Simulation Result Structure¶

Simulation results are stored in the .sim attribute of perches, with a standardized structure:

class SimObject:
    """Container for simulation results created by a backend"""

    def __init__(self, backend):
        self.backend = backend
        self._backend_type = backend.__class__.__name__
        self._branch_sims = {}  # {label: SimObject} for branching nodes

    # Population distribution access
    @property
    def dstn(self):
        """Get the population distribution at this perch.
        For branching nodes with branch-indexed sub-distributions,
        returns the combined distribution (backward-compatible)."""
        if self._branch_sims:
            return self.backend.combine_distributions(
                list(self._branch_sims.values()))
        return self.backend.get_distribution()

    @property
    def dstn_by_branch(self):
        """Get branch-indexed sub-distributions (branching nodes only)."""
        return {label: sim.dstn for label, sim in self._branch_sims.items()}

    @property
    def branch_masses(self):
        """Get total mass in each branch."""
        return {label: sim.dstn.total_weight
                for label, sim in self._branch_sims.items()}

    def add_branch_sim(self, label, sim_object):
        """Add a branch-specific simulation result."""
        self._branch_sims[label] = sim_object

    def combine_with(self, other_sim_objects):
        """Combine sub-population distributions from multiple branches.
        Used at reconvergence points where branch-specific populations merge."""
        return self.backend.combine_distributions(
            [self] + list(other_sim_objects))

    # Aggregate statistics
    def get_statistics(self):
        """Get summary statistics for the population at this perch"""
        return self.backend.calculate_statistics()

    # Individual state access
    def get_states(self):
        """Get all individual states in the population"""
        return self.backend.get_all_states()

    # Filtering and selection
    def filter_population(self, condition):
        """Get subset of population meeting condition"""
        return self.backend.filter_states(condition)

This standardized structure ensures consistent access to simulation results across different backends.

Branch-indexed simulation: When a branching stage produces divergent branches (tagged union exit states), the SimObject at the branching node holds branch-indexed sub-distributions via _branch_sims. The .dstn property returns the combined distribution (backward-compatible with non-branching code). The .dstn_by_branch property returns branch-specific distributions for analysis. The .combine_with() method is used at reconvergence points where branch-specific sub-populations merge into a single distribution (see population-dynamics.md § Reconvergent Branch Combination).

Whisperer Method Naming Conventions¶

In the whisperer interface, methods follow these naming conventions:

Methods beginning with "populate_":
These methods populate existing structural elements with computational content (.comp attribute)
They do NOT create new structural elements (perches, movers)
Example: populate_cntn_terminal() populates an existing continuation perch with terminal condition computational content
Methods beginning with "solve_":
These methods solve mathematical problems using the horse
They populate .comp attributes of multiple perches in a coordinated way
Example: solve_stage() solves the entire stage instance and populates all perches with appropriate computational content
Methods beginning with "simulate_":
These methods perform forward simulation using the horse
They populate .sim attributes on perches with simulation results
Example: simulate_stage() simulates the entire stage instance and populates all perches with simulation results

This naming convention accurately reflects the method's purpose of populating existing objects with computational content rather than creating new structural elements.

class Mover: """Mover connects perches and represents transitions between states"""

def __init__(self, source_perch, dest_perch):
    self.source_perch = source_perch
    self.dest_perch = dest_perch
    self.comp = None  # Will be populated by whisperer

    # Flags
    self.has_comp = False
    self.is_solved = False

    # NOTE: Future implementation may include:
    # self.has_backward_operator = False  # Whether this mover can perform backward solution

    # WARNING: The source_perch and dest_perch references must be maintained
    # for future multilithic implementation. Any redesign of the Mover class
    # must preserve these direct perch references to support backward operators.

```

Future Features (In Development)¶

This section describes features that are planned for future implementation. While not currently active in the framework, these concepts are preserved for reference and potential future development.

WARNING: The multilithic approach described below represents a significant¶

architectural variation that may be implemented in the future. Current¶

development should avoid architectural decisions that would prevent this¶

implementation path. Specifically:¶

¶

1. Maintain clear boundaries between perches and movers¶

2. Preserve direct access to source and destination perches within movers¶

3. Keep mathematical operations organized in ways that could be distributed¶

4. Avoid tightly coupling the whisperer and horse in ways that would prevent¶

delegating backward operations to individual movers¶

Multilithic Solution Process¶

The multilithic approach would allow movers to independently solve parts of a model, enabling more efficient distributed computation for compatible horses: ```python

Example multilithic approach (not currently implemented)¶

if mover.dcsn_to_cntn.has_backward_operator(): # Creates backend-specific functions in a comp object for dcsn perch comp_object = mover.dcsn_to_cntn.backward_solve() # Sets the entire comp object with backend-specific functions mover.source_perch.comp = comp_object # Modified to use current architecture ```