Working Progress 8: Random Forest

Random Forest:

It technically is an ensemble method (based on the divide-and-conquer approach) of decision trees generated on a randomly split dataset. The collection of decision tree classifiers is also known as the forest. The individual decision trees are generated using an attribute selection indicator such as information gain, gain ratio, or Gini index for each attribute. Each tree depends on an independent random sample. In a classification problem, each tree votes and the most popular class is chosen as the final result. In the case of regression, the average of all the tree outputs is considered as the final result. It does not suffer from the overfitting problem. The main reason is that it takes the average of all the predictions, which cancels out the biases.

It works in four steps:

1. Select random samples from a given dataset.

2. Construct a decision tree for each sample and get a prediction result from each decision tree.

3. Perform a vote for each predicted result.

4. Select the prediction result with the most votes as the final prediction.

Tunable Parameters:

max_depth=8
random_state=0
n_estimators=100