We trained a feed forward neural network to learn 1024 binary input/output pairs in a 12 dimensional input/output space (i.e., one quarter of the total binary space). Input vectors to learn were chosen at random. Each input unit was associated with its own output unit (one-to-one). For 95% of the the unit pairs, the input was copied to the output (i.e., the identity mapping). For the remaining 5% (chosen at random), the output bit was flipped. Thus, the identity mapping was the rule, and flipped bits were the exceptions. The network was trained to correctly map all 1024 input/output pairs. During training, input gain was held constant at 1. During testing, we varied input gain between 0.2 and 5. The results for three levels of gain are shown below.

The first plot shows half of the output dimensions plotted on the total input space at a gain of 0.2. At this level, the network was completely driven by the identity "rule". All exception bits were NOT flipped. The identity mapping was generalized to all untrained inputs. Without going into details, the identity mapping is represented as the grid pattern in the plot below. In some sense, low levels of input gain caused a selective impairment in computing exceptions to the identity mapping.

The next plot shows the input/output space at a gain of 0.8. At this level, there was a balance between correctly flipping the exception bits, and correctly generalizing the identity mapping to untrained inputs. Flipped bits occur inside the "pockets" of distortion in the plot below. They also occur in the indentations of the grid. A small number of bits were incorrectly flipped for untrained items. No bits were incorrectly flipped for the trained inputs.

The last plot shows the input/output space at a gain of 5.0. At this level, many bits were flipped, causing a more severe distortion of the identity mapping. What cannot be seen in this visualization is that incorrect bit flipping occurred much more often for the untrained inputs than for the trained inputs. In some sense, high levels of input gain caused a selective impairment in the generalization of learning to novel inputs.