 |
| Picture Source: https://blogs.sas.com |
Aim of work: To understand how ensemble stacking helps in improving accuracy.
Requirements: 3 to 5 Hyper parameter tuned models.
Procedure:
1. First let us take 3 to 5 models such as Decision Tree, Random Forest, Gradient Boosting, Support Vector, Multi-Layer Perceptron etc.
2. Perform Hyper Parameter Tuning on these base models. This can be done by either randomized search or grid search.
3. Now perform ensemble stacking of these models. This step combines all the outputs of different models and gives the majority as our prediction.
4. Next comes the Layer 1 stacking. Here we use the best performing model lets say random forest on this ensembled model. So, here our combined outputs become f(x) in f(x) = y and then becomes our new training set.
5. This gives a significant improvement in terms of accuracy.
6. This trend was observed in many kaggle problems.
Result:
Improved accuracy
The main concept behind the ensemble methods is simple probability. Its always better to take 10 suggestions and then weighing it and deciding rather taking a single suggestion. This way error percentage reduces and thus improves the accuracy.