Skip to main content

External inputs

External inputs changes in the system's environment impacting the system's dynamics. These include stimuli, rewards or anything else which cannot be traced back to the observed system's change. A counterexample is be the movement of the eyes of a subject, which is an additional observed modality and not an external input. Incorporating external inputs into the training data is important as the model needs to know when outside effects may have influenced the data, so it can distinguish between the internal dynamics and the external effects.

The latent models incorporate external inputs via the CC matrix, which may be learned during training. C⋅stC\cdot s_t is added to the state at each time step.

warning

It is important that if external inputs are present, each timestep has to be assigned one, even the ones where no external inputs happened. Latter steps should be assigned the same placeholder value.

External input engineering​

In many cases external inputs are categorical and only happen at specific times, where it is not immediately obvious how to put them into a model-readable form. In this case it is advised to one-hot encode the categories. If multiple of these categories are present (eg. multiple kinds of rewards/stimuli), sts_t should be composed of multiple one-hot-encoded vectors.

This method should be used as one doesn't want to make assumptions of how the external inputs are processed by the dynamical system, thus giving it as simply as possible allows the DSR model to incorporate them in the least biased manner possible.

note

Using one-hot encoding is the standard practice in machine learning, but it still may be infeasible for many distinct categories (eg. >20), in which case one may opt for a single channel input with distinct values for the different categories.

An example of the external input vector over time could be

Time:               1 2 3       ...           T
Stimulus (yes/no):  x       x           x     x
                    ---------------------------
s_t:                1 0 0 0 1 0 0 0 0 0 1 0 0 1
                    0 1 1 1 0 1 1 1 1 1 0 1 1 0

Where st=(1,0)s_t = (1,0) means a stimulus happened at time tt and st=(0,1)s_t = (0,1) means nothing happened.

tip

Depending on the setup, one may want to extend the number of timesteps the external input is given over. This usually aids training and incorporates the unknown factor of when the input was processed by the system.