[Python] 交差検証法によるテスト誤差の推定

K重交差検証法によってテスト誤差を推定する際のメモです．

K重交差検証法の計算プロセスは以下のようになります．

• 学習方法と損失：チューニングパラメータ$\lambda$をもつアルゴリズム$\mathcal{A}$，損失$l ({\bf{z}}, h)$
• 入力：データ${\bf{z}}_{1}, {\bf{z}}_{2}, \cdots, {\bf{z}}_{n}$

1. 入力データを要素数がほぼ等しいK個のグループ$D_{1}, D_{2}, \cdots, D_{K}$に分割する．
2. $k = 1, 2, \cdots, K$として，以下を反復する．
1. $h_{\lambda, k} = \mathcal{A} (D^{(k)}, \lambda)$を計算する．ここで，$D^{(k)} = D \ D_{k}$.
2. $\widehat{Err} ( h_{\lambda, k}) = \cfrac{1}{m}\sum_{x \in D_{k}} l ({\bf{z}},  h_{\lambda, k})$を計算する．
3. $\widehat{Err} ( \mathcal{A}; \lambda) = \cfrac{1}{K}\sum_{k = 1}^{K} \widehat{Err} ( h_{\lambda, k} )$

回帰関数は決定木によって推定することにします．よって，推定には，sklearn.tree の DecisionTreeRegressor を使用します．

パッケージの読み込み

>>> import numpy as np

>>> import scipy as sp

>>> import matplotlib.pyplot as plt

>>> from sklearn.tree import DecisionTreeRegressor

設定は，データ数100個，10重CV(Cross Validation)とします．

>>> n = 100; K = 10 

データの生成を行います．データは区間[-2, 2]上の一様分布とします．

>>> x = np.random.uniform(-2, 2, n)

>>> y = np.sin(2 * np.pi * x) / x + np.random.normal(scale = 0.5, size = n)

データのグループ分けと描画を行います．

>>> cv_idx = np.tile(np.arange(K), int(np.ceil(n/K)))[: n]

>>> maxdepths = np.arange(2, 10)

>>> cverr = np.array([])

>>> for mp in maxdepths:

...     cverr_lambda = np.array([])

...     for k in range(K):

...             tr_idx = (cv_idx!=k) 

...             te_idx = (cv_idx==k)

...             cvx = x[tr_idx]; cvy = y[tr_idx]   

...             dtreg = DecisionTreeRegressor(max_depth=mp)

...             dtreg.fit(np.array([cvx]).T, cvy)               

...             ypred = dtreg.predict(np.array([x[te_idx]]).T)  

...             cl = np.append(cverr_lambda, np.mean((y[te_idx]-ypred)**2/2))

...     cverr = np.append(cverr, np.mean(cl))

... 

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

.

. *1 （省略）

.

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

>>> plt.scatter(maxdepths, cverr,c='k')  # cv誤差のプロット

<matplotlib.collections.PathCollection object at 0x122d2cc18>

>>> plt.xlabel("max depth"); plt.ylabel('cv error')

Text(0.5, 0, 'max depth')

Text(0, 0.5, 'cv error')

>>> plt.show()

上記のコードの解説

• 決定木の深さの候補は maxdepths = np.arange(2, 10) で設定
• CVのためのデータ分割は cvx = x[tr_idx]; cvy = y[tr_idx] で行っている
• 決定木で推定するのは dtreg.fit(np.array([cvx]).T, cvy)で行っている．
• 予測は ypred = dtreg.predict(np.array([x[te_idx] - ypred)**2/2) で行っている

上記の例のJupyter Notebookファイルは，GitHubのStatistics.ipynbの"Estimation of test error by cross validation method"で見ることができます．

----------
REPLの実行で省略した *1 部分は，以下の様に表示されます．

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=2, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=3, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=4, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=5, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=6, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=7, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=8, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')

DecisionTreeRegressor(criterion='mse', max_depth=9, max_features=None,

           max_leaf_nodes=None, min_impurity_decrease=0.0,

           min_impurity_split=None, min_samples_leaf=1,

           min_samples_split=2, min_weight_fraction_leaf=0.0,

           presort=False, random_state=None, splitter='best')