Abstract: A new algorithm for learning invariance manifolds is introduced that allows a neuron to learn a non-linear transfer function to extract invariant or rather slowly varying features from a vectorial input sequence. This is generalized to a group of neurons, referred to as a Gibson-clique, to learn slowly varying features that are uncorrelated. Since the transfer functions are non-linear, this technique can be applied iteratively. Four examples demonstrating the properties of the learning algorithm include learning complex cell response with one Gibson-clique and learning translation invariance in a hierarchical network of Gibson-cliques.