Boosting (machine learning)

In <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> (ML), boosting is an <a href="/facts/Ensemble_learning/GvFsd3Qc">ensemble</a> <a href="/facts/Metaheuristic/ECF71Rez">metaheuristic</a> for primarily reducing <a href="/facts/Bias%E2%80%93variance_tradeoff/xqDnRpeQ">bias (as opposed to variance)</a>. It can also improve the <a href="/facts/Stability_(learning_theory)/mreLQdwQ">stability</a> and accuracy of ML <a href="/facts/Statistical_classification/jXXHRkXR">classification</a> and <a href="/facts/Regression_analysis/n6z5Tf7K">regression</a> algorithms. Hence, it is prevalent in <a href="/facts/Supervised_learning/JlNFNPFt">supervised learning</a> for converting weak learners to strong learners.
The concept of boosting is based on the question posed by <a href="/facts/Michael_Kearns_(computer_scientist)/GM9jvLPs">Kearns</a> and <a href="/facts/Leslie_Valiant/hA5okAxL">Valiant</a> (1988, 1989): "Can a set of weak learners create a single strong learner?" A weak learner is defined as a <a href="/facts/Statistical_classification/jXXHRkXR">classifier</a> that is only slightly correlated with the true classification. A strong learner is a classifier that is arbitrarily well-correlated with the true classification. <a href="/facts/Robert_Schapire/vpmHkRTN">Robert Schapire</a> answered the question in the affirmative in a paper published in 1990. This has had significant ramifications in machine learning and <a href="/facts/Statistics/DNPsKGYU">statistics</a>, most notably leading to the development of boosting.
Initially, the hypothesis boosting problem simply referred to the process of turning a weak learner into a strong learner. Algorithms that achieve this quickly became known as "boosting". <a href="/facts/Yoav_Freund/Ry0euHMX">Freund</a> and Schapire's arcing (Adapt[at]ive Resampling and Combining), as a general technique, is more or less synonymous with boosting.

Boosting (machine learning) open-in-new

Boosting (machine learning)