Feature request: Skiping neurons in training

[phschafft]

I would like to ask for a feature to disable some output neurons while training.
As from how I understand the code this could easily be done by setting neuron_diff to zero in fann_compute_MSE() for the given neuron.

This is in full acknowledgement that such training in general will also have effects on the disabled neurons. When implemented this fact should be outlined in the documentation.

Consider two scenarios/usecases this seems to be helpful in (both very simplified):

The first one is a network with two binary outputs. The first provides information on whether or not a feature is present in the input and the second on the state of that feature if it is present. In this case I do not care for the value of the state output when the detection output is not active. My training data could look like this:

[
  black cat -> [cat, black],
  white cat -> [cat, white],
  dog -> [not cat, don't care]
]

In such a case pinning the network to a specific answer of the state seems like it could degrade the quality of the network and/or slow down training. (e.g. if I would pin it to a constant value I would prevent the network from learning colour detection independent of the species. Which is a valid solution to the given problem.)

The second one is when you work with real world data of mixed quality:
While you might have some data on what the complete result for a given set of inputs looks like you might also have high quality data for some of the outputs. This could for example happen if you try to learn on data from sources that include different features. Consider the first example. Let's say you have two data sets:

[
  black cat -> [cat, black],
  white cat -> [cat, white]
]

and:

[
  cat -> [cat, not given],
  dog -> [not cat, not given]
]

[phschafft]

Please note that a more complete solution to this would be to add a weight to each output on each set of data that is multiplied into neuron_diff. This would also allow to train on data with a "medium quality" in a different way than data with a "high quality" while still training the network.

[phschafft]

Please note that this is a in itself valid subset of #139. This feature more comes with less penalties and is easier to use. Therefore having some value on it's own.