Hierarchised Models: Architecture Overview

This section explains the core architectural pattern of the project: Hierarchiseation. This mechanism allows models (both Encoders and Dynamical Systems) to learn shared structures across a dataset ("Group Level") while adapting to specific variations ("Subject Level").

The "Director-Builder" Pattern

The implementation follows a pattern similar to the Builder or Factory pattern, but adapted for PyTorch nn.Modules.

1. The Director (Wrapper Classes):

   • Classes: Encoder (in src/models/encoder/encoder.py) and DSRModel (in src/models/dsr_model/dsr_model.py).
   • Responsibility: These are the actual nn.Modules instantiated in your training loop. They hold the State (the parameters). They verify configuration, manage the data flow (batching), and decide which subject parameters to use for which input.

2. The Logic Library (Implementation Classes):

   • Classes: The specific model implementations (e.g., StackedConvolutions, ALRNN).
   • Responsibility: These classes are stateless "libraries" of logic. They tell the Director how to initialize parameters and how to combine them during a forward pass. They do not store self.weight directly; instead, they receive weights as arguments.

The HierarchisedModel Interface

To enforce this pattern, all implementation classes must inherit from HierarchisedModel (defined in src/models/hierarchised_model.py). This abstract base class mandates four specific @classmethods that define the lifecycle of a model's parameters.

1. Configuration Phase

• default_hyperparameters(): Returns the default configuration dict. The Director merges this with the user's config file (via resolve_config) to ensure a complete hyperparameter set.

2. Initialization Phase

• initialize_group_level_params(scheme, hyperparams): Defines the shared parameters.
  • Example: In a standard neural net, this returns the weights and biases. In a Linear Projection scheme, this might return a large "Projection Matrix" basis.
• initialize_subject_level_params(scheme, hyperparams, subject_hyperparams): Defines the subject-specific parameters.
  • Example: In a standard neural net, this might return nothing (if fully shared). In a Linear Projection scheme, this returns a low-dimensional "Subject Vector".

3. Execution Phase (Forward Pass)

• construct_params(scheme, hyperparams, group_params, subject_params): The "Linker". It combines the shared Group params and the specific Subject params to create the effective parameters for this forward pass.
  • Example: Effective Weight = Projection Matrix @ Subject Vector.
• forward(x, params, hyperparams, ...): The actual computation. It takes the constructed params (effective weights) and the input data x to produce output.

Why this complexity?

This separation allows us to swap Hierarchisation Schemes (how parameters are shared) without changing the Model Logic (how the model computes).

• You can run an ALRNN where every subject has their own independent weights.
• You can run the same ALRNN where all subjects share weights but have unique biases.
• You can run it where weights are generated via a low-rank projection.

All of this is controlled simply by changing the scheme in the config, while the Wrapper class handles the plumbing.