Committee machine
Encyclopedia
A committee machine is a type of neural network
using a divide and conquer
strategy in which the responses of multiple neural networks (experts) are combined into a single response. The combined response of the committee machine is supposed to be superior to those of its constituent experts. Compare with ensembles of classifiers.
In ensemble averaging, outputs of different predictors are linearly combined to produce an overall output.
In boosting, a weak algorithm is converted into one that achieves arbitrarily high accuracy.
In mixture of experts, the individual responses of the experts are non-linearly combined by means of a single gating network.
In hierarchical mixture of experts, the individual responses of the individual experts are non-linearly combined by means of several gating networks arranged in a hierarchical fashion.
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
using a divide and conquer
Divide and conquer
Divide and conquer may refer to:* Divide and rule, in politics, sociology and economics, a strategy to gain or maintain power...
strategy in which the responses of multiple neural networks (experts) are combined into a single response. The combined response of the committee machine is supposed to be superior to those of its constituent experts. Compare with ensembles of classifiers.
Static structures
In this class of committee machines , the responses of several predictors (experts) are combined by means of a mechanism that does not involve the input signal, hence the designation static. This category includes the following methods:- Ensemble averagingEnsemble AveragingIn machine learning, particularly in the creation of artificial neural networks, ensemble averaging is the process of creating multiple models and combining them to produce a desired output, as opposed to creating just one model...
In ensemble averaging, outputs of different predictors are linearly combined to produce an overall output.
- BoostingBoostingBoosting is a machine learning meta-algorithm for performing supervised learning. Boosting is based on the question posed by Kearns: can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true classification...
In boosting, a weak algorithm is converted into one that achieves arbitrarily high accuracy.
Dynamic structures
In this second class of committee machines, the input signal is directly involved in actuating the mechanism that integrates the outputs of the individual experts into an overall output, hence the designation dynamic. There are two kinds of dynamic structures:- Mixture of experts
In mixture of experts, the individual responses of the experts are non-linearly combined by means of a single gating network.
- Hierarchical mixture of experts
In hierarchical mixture of experts, the individual responses of the individual experts are non-linearly combined by means of several gating networks arranged in a hierarchical fashion.