Decision tree pruning

Pruning is a <a href="/facts/Data_compression/gMFuY4FT">data compression</a> technique in <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> and <a href="/facts/Search_algorithm/eJuw0nyz">search algorithms</a> that reduces the size of <a href="/facts/Decision_tree_learning/LbPMAwdI">decision trees</a> by removing sections of the tree that are non-critical and redundant to classify instances. Pruning reduces the complexity of the final <a href="/facts/Statistical_classification/jXXHRkXR">classifier</a>, and hence improves predictive accuracy by the reduction of <a href="/facts/Overfitting/5xnFLcMg">overfitting</a>.
One of the questions that arises in a decision tree algorithm is the optimal size of the final tree. A tree that is too large risks <a href="/facts/Overfitting/5xnFLcMg">overfitting</a> the training data and poorly generalizing to new samples. A small tree might not capture important structural information about the sample space. However, it is hard to tell when a tree algorithm should stop because it is impossible to tell if the addition of a single extra node will dramatically decrease error. This problem is known as the <a href="/facts/Horizon_effect/S8WemGG3">horizon effect</a>. A common strategy is to grow the tree until each node contains a small number of instances then use pruning to remove nodes that do not provide additional information.
Pruning should reduce the size of a learning tree without reducing predictive accuracy as measured by a <a href="/facts/Cross-validation_(statistics)/pDWg8s0e">cross-validation</a> set. There are many techniques for tree pruning that differ in the measurement that is used to optimize performance.

Decision tree pruning open-in-new

Decision tree pruning