Abstract: A new algorithm for learning invariance manifolds is introduced that allows a neuron to learn a non-linear input-output function to extract invariant or rather slowly varying features from a vectorial input sequence. This is demonstrated by a simple model of learning complex cell responses. The algorithm is generalized to a group of neurons, referred to as a Gibson-clique, to learn slowly varying features that are uncorrelated. Since the input-output functions are non-linear, this technique can be applied iteratively. This is demonstrated by a hierarchical network of Gibson-cliques learning translation invariance.
Keywords: feed-forward network, higher-order units, invariances, manifolds, unsupervised learning