When handwriting the Adaboost algorithm, the basic classifier
G
m
(
x
)
G_{m}(x)
The coefficients of Gm?(x) are:
alpha
m
=
1
2
log
?
1
?
e
m
e
m
\alpha_m=\frac{1}{2}\log\frac{1-e_m}{e_m}
αm?=21?logem?1?em
We use code to implement as: (only show part of the code)
estimator.alpha=0.5*np.log((1-min_error)/(min_error)) preds=np.ones(np.shape(y)) negative_idx=(estimator.lable*X[:,estimator.feature_index]<estimator.lable*estimator.threshold) preds[negative_idx]=-1 w*=np.exp(-estimator.alpha*y*preds) w/=np.sum(w) self.estimators.append(estimator)
But after running it gives an error:
F:\anaconda\envs\sklearn-env\python.exe F:/PycharmProject/Adaboost_numpy_demo.py
Traceback (most recent call last):
File "F:\PycharmProject\Adaboost_numpy_demo.py", line 116, in <module>
clf.fit(X_train,y_train)
File "F:\PycharmProject\Adaboost_numpy_demo.py", line 80, in fit
estimator.alpha=0.5*np.log((1-min_error)/(min_error))
ZeroDivisionError: division by zero
Process finished with exit code 1
The error here is obvious, that is, in our formula calculation,
e
m
e_m
em? may be a very small number, and it will become 0 when the calculation is performed, and an error will occur, so the correction method is also very clear:
estimator.alpha=0.5*np.log((1-min_error)/(min_error + 1e-9))
The result of this operation will not report an error!
problem solved!