Temporal Pooler Overview

temporal_pooler.py pseudocode

The temporal pooler attempts to form stable and unique representations of input sequences of active cells from Temporal Memory, but only if those cells were correctly predicted by Temporal Memory. A competition, similar to that of the spatial pooler's, is used to select active cells. In fact the first implementation inherits from the spatial pooler.

If a column is sufficiently activated by inputs that were predicted in the Temporal Memory, it enters a "pooling mode." A pooling column can, for a limited period, maintain active column status even without receiving any bottom-up input activity. This is implemented using a kind of "activation" for each column. When the activation drops to 0 it loses its pooling status. Whenever significant predicted input returns to a column's synapses, its activation is reset. The Temporal Pooler can be thought of as a particular kind of Spatial Pooler. One terminology note is that instead of referring to active columns we speak of active cells in the Temporal Pooler.

Pseudocode:

1) Calculate overlap:
    Between correctly predicted input cells and pooling columns’ synapses
  		if learning
    			Use pooling columns' potential pool of synapses
  		else 
    			Use pooling columns' connected synapses
  		(This type of overlap is heavily weighted compared to the others)

    Between active input cells and all columns
        Based on connected synapses (as in SP)
    Between correctly predicted input cells and all columns
  		Based on connected synapses (as in SP)
    # Unused, experimental idea
	Add (large) parametric amount of overlap from bursting columns (false negatives) to all columns

2) Perform Spatial Pooler’s inhibition process returning a set of active columns

3) Perform Spatial-Pooler-like learning based on active columns
    Stronger permanence change for synapses receiving correctly predicted (true positive) inputs. 
    (No change for false positive connections)

4) Update columns’ pooling activity and compute current pooling columns
	if unpredicted inputs exceed threshold
            Set all pooling activations to 0.0
	else
  		for all pooling columns
    	        Decay pooling activation
 		for all active columns receiving any true positive inputs
   			Set pooling activation to parameter value

	return columns having non-zero pooling activation