{ "cells": [ { "cell_type": "markdown", "metadata": { "_cell_guid": "6241f8fb-5ba4-6e70-2099-c98bfeef3720" }, "source": [ "As some of you probably aware, a pretty interesting paper was released last month on Deep Forest by Prof Zhihua Zhou. (https://arxiv.org/abs/1702.08835). It proposed an alternative to deep learning by using an iterative meta ensemble forest model. Honestly, this idea is not very surprising, I believe anyone who have done ensemble must have thought of \"something\" like this. But this is the first paper that implements it in a very reasonable way.\n", "\n", "This approach is better than deep learning in two ways:\n", "1. Not relying on GPU. Well...for poor people like myself who cannot afford a TitanX (actually...I cannot even afford running an AWS p2.large on a regular basis lol) This model can be trained simply with an i7 on a laptop. (You do need some more RAM....as a trade off)\n", "2. Small sample size. This is the best part, you need very little sample to train an effective model. OK this thread's topic is very misleading, bite me~ So I sampled 100 samples from each category (except for LAG, where I added in additional images from Imagenet, which are posted in the forum). Surprisingly, after proper parameter tunning, I was able to achieve >75% local validation accuracy, and ~1.4 public LB score. This is pretty amazing. The best score I got with a Vanilla CNN trained from scratch with the full data set and augmentation was only ~1.2. This model really works on small data set.\n", "\n", "I do observe that this model is still not comparable with pre-trained VGG on the full data set, but I guess it is not fair to even make this comparison.\n", "The code here are largely developed by Pierre-Yves Lablanche (https://github.com/pylablanche/gcForest). I twick it a little bit to fit this competition.\n", "\n", "If anyone thought about using Deep Forest, please let me know of your performance, especially if you used it on cropped image with localization network (which I still do not understand....)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "0dc9f66b-4d6b-9d8a-8412-c6540d5d0554" }, "outputs": [], "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "0511d0be-18e7-f7d6-3fa8-84795ebd7ff4" }, "source": [ "

gcForest Algorithm

\n", "\n", "

The gcForest algorithm was suggested in Zhou and Feng 2017 (refer for this paper for technical details) and I provide here a python implementation of this algorithm.
\n", "I chose to adopt the scikit-learn syntax for ease of use and hereafter I present how it can be used.

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "abe9e648-faa1-53e3-bf03-023fa9219a48" }, "outputs": [], "source": [ "#from GCForest import gcForest\n", "from skimage import color, io\n", "from scipy.misc import imresize\n", "import os\n", "import glob\n", "import time\n", "import numpy as np # linear algebra\n", "from keras.utils import np_utils\n", "import pandas as pd\n", "\n", "import datetime\n", "\n", "from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "0ce4c726-b6cd-07f1-1b2c-5d32f066ee9a", "collapsed": true }, "outputs": [], "source": [ "#!usr/bin/env python\n", "\"\"\"\n", "Version : 0.1.0\n", "Date : 16th March 2017\n", "Author : Pierre-Yves Lablanche\n", "Email : plablanche@aims.ac.za\n", "Affiliation : African Institute for Mathematical Sciences - South Africa\n", " Stellenbosch University - South Africa\n", "License : MIT\n", "Status : Under Development\n", "Description :\n", "Python3 implementation of the gcForest algorithm preesented in Zhou and Feng 2017\n", "(paper can be found here : https://arxiv.org/abs/1702.08835 ).\n", "It uses the typical scikit-learn syntax with a .fit() function for training\n", "and a .predict() function for predictions.\n", "\"\"\"\n", "import itertools\n", "import numpy as np\n", "from sklearn.ensemble import RandomForestClassifier\n", "from sklearn.model_selection import train_test_split\n", "from sklearn.metrics import accuracy_score\n", "\n", "__author__ = \"Pierre-Yves Lablanche\"\n", "__email__ = \"plablanche@aims.ac.za\"\n", "__license__ = \"MIT\"\n", "__version__ = \"0.1.0\"\n", "__status__ = \"Development\"\n", "\n", "\n", "# noinspection PyUnboundLocalVariable\n", "class gcForest(object):\n", "\n", " def __init__(self, shape_1X, n_mgsRFtree=30, window=None, cascade_test_size=0.2, n_cascadeRF=2,\n", " n_cascadeRFtree=101, cascade_layer=np.inf, min_samples=0.05, tolerance=0.0):\n", " \"\"\" gcForest Classifier.\n", " :param shape_1X: tuple list or np.array\n", " Shape of a single sample element.\n", " :param n_mgsRFtree: int (default=30)\n", " Number of trees in a Random Forest during Multi Grain Scanning.\n", " :param window: int (default=None)\n", " List of window sizes to use during Multi Grain Scanning.\n", " If 'None' no slicing will be done.\n", " :param cascade_test_size: float or int (default=0.2)\n", " Split fraction or absolute number for cascade training set splitting.\n", " :param n_cascadeRF: int (default=2)\n", " Number of Random Forests in a cascade layer.\n", " For each pseudo Random Forest a complete Random Forest is created, hence\n", " the total numbe of Random Forests in a layer will be 2*n_cascadeRF.\n", " :param n_cascadeRFtree: int (default=101)\n", " Number of trees in a single Random Forest in a cascade layer.\n", " :param min_samples: float or int (default=0.1)\n", " Minimum number of samples in a node to perform a split\n", " during the training of any Random Forest.\n", " If int number_of_samples = int.\n", " If float, min_samples represents the fraction of the initial n_samples to consider.\n", " :param cascade_layer: int (default=np.inf)\n", " mMximum number of cascade layers allowed.\n", " Useful to limit the contruction of the cascade.\n", " :param tolerance: float (default=0.0)\n", " Accuracy tolerance for the casacade growth.\n", " If the improvement in accuracy is not better than the tolerance the construction is\n", " stopped.\n", " \"\"\"\n", "\n", " setattr(self, 'shape_1X', shape_1X)\n", " setattr(self, 'n_layer', 0)\n", " setattr(self, '_n_samples', 0)\n", " setattr(self, 'n_cascadeRF', int(n_cascadeRF))\n", " if isinstance(window, int):\n", " setattr(self, 'window', [window])\n", " elif isinstance(window, list):\n", " setattr(self, 'window', window)\n", " setattr(self, 'cascade_test_size', cascade_test_size)\n", " setattr(self, 'n_mgsRFtree', int(n_mgsRFtree))\n", " setattr(self, 'n_cascadeRFtree', int(n_cascadeRFtree))\n", " setattr(self, 'cascade_layer', cascade_layer)\n", " setattr(self, 'min_samples', min_samples)\n", " setattr(self, 'tolerance', tolerance)\n", "\n", " def fit(self, X, y):\n", " \"\"\" Training the gcForest on input data X and associated target y.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param y: np.array\n", " 1D array containing the target values.\n", " Must be of shape [n_samples]\n", " \"\"\"\n", "\n", " if np.shape(X)[0] != len(y):\n", " raise ValueError('Sizes of y and X do not match.')\n", " setattr(self, 'n_layer', 0)\n", " if not getattr(self, 'window'):\n", " shape_1X = getattr(self, 'shape_1X')\n", " setattr(self, 'window', [shape_1X[0]])\n", " mgs_X = self.mg_scanning(X, y)\n", " _ = self.cascade_forest(mgs_X, y)\n", "\n", " def predict(self, X):\n", " \"\"\" Predict the class of unknown samples X.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of the same shape [n_samples, data] as the training inputs.\n", " :return: np.array\n", " 1D array containing the predicted class for each input sample.\n", " \"\"\"\n", "\n", " mgs_X = self.mg_scanning(X)\n", " cascade_all_pred_prob = self.cascade_forest(mgs_X)\n", " cascade_pred_prob = np.mean(cascade_all_pred_prob, axis=0)\n", " #predictions = np.argmax(cascade_pred_prob, axis=1)\n", "\n", " return cascade_pred_prob\n", "\n", " def mg_scanning(self, X, y=None):\n", " \"\"\" Performs a Multi Grain Scanning on input data.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param y: np.array (default=None)\n", " :return: np.array\n", " Array of shape [n_samples, .. ] containing Multi Grain Scanning sliced data.\n", " \"\"\"\n", "\n", " setattr(self, '_n_samples', np.shape(X)[0])\n", " shape_1X = getattr(self, 'shape_1X')\n", " if len(shape_1X) < 2:\n", " raise ValueError('shape parameter must be a tuple')\n", "\n", " mgs_pred_prob = []\n", "\n", " for wdw_size in getattr(self, 'window'):\n", " wdw_pred_prob = self.window_slicing_pred_prob(X, wdw_size, shape_1X, y=y)\n", " mgs_pred_prob.append(wdw_pred_prob)\n", "\n", " return np.concatenate(mgs_pred_prob, axis=1)\n", "\n", " def window_slicing_pred_prob(self, X, window, shape_1X, y=None):\n", " \"\"\" Performs a window slicing of the input data and send them through Random Forests.\n", " If target values 'y' are provided sliced data are then used to train the Random Forests.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param window: int\n", " Size of the window to use for slicing.\n", " :param shape_1X: list or np.array\n", " Shape of a single sample.\n", " :param y: np.array (default=None)\n", " Target values. If 'None' no training is done.\n", " :return: np.array\n", " Array of size [n_samples, ..] containing the Random Forest.\n", " prediction probability for each input sample.\n", " \"\"\"\n", "\n", " n_tree = getattr(self, 'n_mgsRFtree')\n", " min_samples = getattr(self, 'min_samples')\n", "\n", " if shape_1X[1] > 1:\n", " print('Slicing Images...')\n", " sliced_X, sliced_y = self._window_slicing_img(X, window, shape_1X, y=y)\n", " else:\n", " print('Slicing Sequence...')\n", " sliced_X, sliced_y = self._window_slicing_sequence(X, window, shape_1X, y=y)\n", "\n", " if y is not None:\n", " prf = RandomForestClassifier(n_estimators=n_tree, max_features='sqrt',\n", " min_samples_split=min_samples, oob_score=True)\n", " crf = RandomForestClassifier(n_estimators=n_tree, max_features=None,\n", " min_samples_split=min_samples, oob_score=True)\n", " print('Training MGS Random Forests...')\n", " prf.fit(sliced_X, sliced_y)\n", " crf.fit(sliced_X, sliced_y)\n", " setattr(self, '_mgsprf_{}'.format(window), prf)\n", " setattr(self, '_mgscrf_{}'.format(window), crf)\n", " pred_prob_prf = prf.oob_decision_function_\n", " pred_prob_crf = crf.oob_decision_function_\n", "\n", " if hasattr(self, '_mgsprf_{}'.format(window)) and y is None:\n", " prf = getattr(self, '_mgsprf_{}'.format(window))\n", " crf = getattr(self, '_mgscrf_{}'.format(window))\n", " pred_prob_prf = prf.predict_proba(sliced_X)\n", " pred_prob_crf = crf.predict_proba(sliced_X)\n", "\n", " pred_prob = np.c_[pred_prob_prf, pred_prob_crf]\n", "\n", " return pred_prob.reshape([getattr(self, '_n_samples'), -1])\n", "\n", " def _window_slicing_img(self, X, window, shape_1X, y=None):\n", " \"\"\" Slicing procedure for images\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param window: int\n", " Size of the window to use for slicing.\n", " :param shape_1X: list or np.array\n", " Shape of a single sample.\n", " :param y: np.array (default=None)\n", " Target values.\n", " :return: np.array and np.array\n", " Arrays containing the sliced images and target values (empty if 'y' is None).\n", " \"\"\"\n", "\n", " if any(s < window for s in shape_1X):\n", " raise ValueError('window must be smaller than both dimensions for an image')\n", "\n", " sliced_imgs = []\n", " sliced_target = []\n", " refs = np.arange(0, window * shape_1X[1], shape_1X[0])\n", "\n", " iterx = list(range(shape_1X[0] - window + 1))\n", " itery = list(range(shape_1X[1] - window + 1))\n", "\n", " for img, ix, iy in itertools.product(enumerate(X), iterx, itery):\n", " rind = refs + ix + shape_1X[0] * iy\n", " sliced_imgs.append(np.ravel([img[1][i:i + window] for i in rind]))\n", " if y is not None:\n", " sliced_target.append(y[img[0]])\n", "\n", " return np.asarray(sliced_imgs), np.asarray(sliced_target)\n", "\n", " def _window_slicing_sequence(self, X, window, shape_1X, y=None):\n", " \"\"\" Slicing procedure for sequences (aka shape_1X = [.., 1]).\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param window: int\n", " Size of the window to use for slicing.\n", " :param shape_1X: list or np.array\n", " Shape of a single sample.\n", " :param y: np.array (default=None)\n", " Target values.\n", " :return: np.array and np.array\n", " Arrays containing the sliced sequences and target values (empty if 'y' is None).\n", " \"\"\"\n", " if shape_1X[0] < window:\n", " raise ValueError('window must be smaller than the sequence dimension')\n", "\n", " sliced_sqce = []\n", " sliced_target = []\n", "\n", " for sqce in enumerate(X):\n", " slice_sqce = [sqce[1][i:i + window] for i in np.arange(shape_1X[0] - window + 1)]\n", " sliced_sqce.append(slice_sqce)\n", " if y is not None:\n", " sliced_target.append(np.repeat(y[sqce[0]], shape_1X[0] - window + 1))\n", "\n", " return np.reshape(sliced_sqce, [-1, window]), np.ravel(sliced_target)\n", "\n", " def cascade_forest(self, X, y=None):\n", " \"\"\" Perform (or train if 'y' is not None) a cascade forest estimator.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param y: np.array (default=None)\n", " Target values. If 'None' perform training.\n", " :return: np.array\n", " 1D array containing the predicted class for each input sample.\n", " \"\"\"\n", "\n", " if y is not None:\n", " test_size = getattr(self, 'cascade_test_size')\n", " max_layers = getattr(self, 'cascade_layer')\n", " tol = getattr(self, 'tolerance')\n", "\n", " X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=test_size)\n", "\n", " self.n_layer += 1\n", " prf_crf_pred_ref = self._cascade_layer(X_train, y_train)\n", " accuracy_ref = self._cascade_evaluation(X_test, y_test)\n", " feat_arr = self._create_feat_arr(X_train, prf_crf_pred_ref)\n", "\n", " self.n_layer += 1\n", " prf_crf_pred_layer = self._cascade_layer(feat_arr, y_train)\n", " accuracy_layer = self._cascade_evaluation(X_test, y_test)\n", "\n", " while accuracy_layer > (accuracy_ref + tol) and self.n_layer <= max_layers:\n", " accuracy_ref = accuracy_layer\n", " prf_crf_pred_ref = prf_crf_pred_layer\n", " feat_arr = self._create_feat_arr(X_train, prf_crf_pred_ref)\n", " self.n_layer += 1\n", " prf_crf_pred_layer = self._cascade_layer(feat_arr, y_train)\n", " accuracy_layer = self._cascade_evaluation(X_test, y_test)\n", "\n", " elif y is None:\n", " at_layer = 1\n", " prf_crf_pred_ref = self._cascade_layer(X, layer=at_layer)\n", " while at_layer < getattr(self, 'n_layer'):\n", " at_layer += 1\n", " feat_arr = self._create_feat_arr(X, prf_crf_pred_ref)\n", " prf_crf_pred_ref = self._cascade_layer(feat_arr, layer=at_layer)\n", "\n", " return prf_crf_pred_ref\n", "\n", " def _cascade_layer(self, X, y=None, cv=3, layer=0):\n", " \"\"\" Cascade layer containing Random Forest estimators.\n", " If y is not None the layer is trained.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param y: np.array (default=None)\n", " Target values. If 'None' perform training.\n", " :param cv: int (default=3)\n", " Number of split for k-fold cross-validation.\n", " :param layer: int (default=0)\n", " Layer indice. Used to call the previously trained layer.\n", " :return: list\n", " List containing the prediction probabilities for all samples.\n", " \"\"\"\n", "\n", " n_tree = getattr(self, 'n_cascadeRFtree')\n", " n_cascadeRF = getattr(self, 'n_cascadeRF')\n", " min_samples = getattr(self, 'min_samples')\n", "\n", " prf = RandomForestClassifier(n_estimators=n_tree, max_features='sqrt',\n", " min_samples_split=min_samples, oob_score=True)\n", " crf = RandomForestClassifier(n_estimators=n_tree, max_features=None,\n", " min_samples_split=min_samples, oob_score=True)\n", "\n", " prf_crf_pred = []\n", " if y is not None:\n", " print('Adding/Training Layer, n_layer={}'.format(self.n_layer))\n", " for irf in range(n_cascadeRF):\n", " prf.fit(X, y)\n", " crf.fit(X, y)\n", " setattr(self, '_casprf{}_{}'.format(self.n_layer, irf), prf)\n", " setattr(self, '_cascrf{}_{}'.format(self.n_layer, irf), crf)\n", " prf_crf_pred.append(prf.oob_decision_function_)\n", " prf_crf_pred.append(crf.oob_decision_function_)\n", " elif y is None:\n", " for irf in range(n_cascadeRF):\n", " prf = getattr(self, '_casprf{}_{}'.format(layer, irf))\n", " crf = getattr(self, '_cascrf{}_{}'.format(layer, irf))\n", " prf_crf_pred.append(prf.predict_proba(X))\n", " prf_crf_pred.append(crf.predict_proba(X))\n", "\n", " return prf_crf_pred\n", "\n", " def _cascade_evaluation(self, X_test, y_test):\n", " \"\"\" Evaluate the accuracy of the cascade using X and y.\n", " :param X_test: np.array\n", " Array containing the test input samples.\n", " Must be of the same shape as training data.\n", " :param y_test: np.array\n", " Test target values.\n", " :return: float\n", " the cascade accuracy.\n", " \"\"\"\n", "\n", " casc_pred_prob = np.mean(self.cascade_forest(X_test), axis=0)\n", " casc_pred = np.argmax(casc_pred_prob, axis=1)\n", " casc_accuracy = accuracy_score(y_true=y_test, y_pred=casc_pred)\n", " print('Layer validation accuracy = {}'.format(casc_accuracy))\n", "\n", " return casc_accuracy\n", "\n", " def _create_feat_arr(self, X, prf_crf_pred):\n", " \"\"\" Concatenate the original feature vector with the predicition probabilities\n", " of a cascade layer.\n", " :param X: np.array\n", " Array containing the input samples.\n", " Must be of shape [n_samples, data] where data is a 1D array.\n", " :param prf_crf_pred: list\n", " Prediction probabilities by a cascade layer for X.\n", " :return: np.array\n", " Concatenation of X and the predicted probabilities.\n", " To be used for the next layer in a cascade forest.\n", " \"\"\"\n", "\n", " swap_pred = np.swapaxes(prf_crf_pred, 0, 1)\n", " add_feat = swap_pred.reshape([np.shape(X)[0], -1])\n", " feat_arr = np.concatenate([add_feat, X], axis=1)\n", "\n", " return feat_arr" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "10631d92-833f-90d0-c8ec-c7a27f05274d", "collapsed": true }, "outputs": [], "source": [ "img_size_glob = 128\n", "\n", "def get_im_cv2(path):\n", " img = io.imread(path)\n", " resized = imresize(img, (img_size_glob, img_size_glob, 3))\n", " return resized\n", "\n", "def load_train():\n", " X_train = []\n", " X_train_id = []\n", " y_train = []\n", " start_time = time.time()\n", "\n", " print('Read train images')\n", " folders = ['ALB', 'BET', 'DOL', 'LAG', 'NoF', 'OTHER', 'SHARK', 'YFT']\n", " for fld in folders:\n", " index = folders.index(fld)\n", " print('Load folder {} (Index: {})'.format(fld, index))\n", " path = os.path.join('..', 'Fishery','input', 'train_sample', fld, '*.jpg')\n", " files = glob.glob(path) \n", " for fl in files:\n", " flbase = os.path.basename(fl)\n", " x = get_im_cv2(fl)\n", " X_train.append(x)\n", " X_train_id.append(flbase)\n", " y_train.append(index)\n", "\n", " print('Read train data time: {} seconds'.format(round(time.time() - start_time, 2)))\n", " return X_train, y_train, X_train_id\n", "\n", "\n", "def load_test():\n", " path = os.path.join('..', 'Fishery','input', 'test', 'test_stg1', '*.jpg')\n", " files = sorted(glob.glob(path))\n", "\n", " X_test = []\n", " X_test_id = []\n", " for fl in files:\n", " flbase = os.path.basename(fl)\n", " img = get_im_cv2(fl)\n", " X_test.append(img)\n", " X_test_id.append(flbase)\n", "\n", " return X_test, X_test_id\n", "\n", "def read_and_normalize_train_data():\n", " train_data, train_target, train_id = load_train()\n", "\n", " print('Convert to numpy...')\n", " train_data = np.array(train_data, dtype=np.uint8)\n", " train_target = np.array(train_target, dtype=np.uint8)\n", " #print('Current Shape: ', train_data.shape)\n", " print('Reshape...')\n", " #train_data = train_data.flatten('F')\n", " train_data = train_data.reshape(train_data.shape[0], -1)\n", " #print('Updated Shape: ', train_data.shape)\n", " print('Convert to float...')\n", " train_data = train_data.astype('float32')\n", " train_X = train_data / 255\n", " #train_target = np_utils.to_categorical(train_target, 8)\n", "\n", " return train_X, train_target, train_id\n", "\n", "\n", "def read_and_normalize_test_data():\n", " start_time = time.time()\n", " test_data, test_id = load_test()\n", "\n", " test_data = np.array(test_data, dtype=np.uint8)\n", " test_data = test_data.reshape(test_data.shape[0], -1)\n", "\n", " test_data = test_data.astype('float32')\n", " test_data = test_data / 255\n", "\n", " print('Test shape:', test_data.shape)\n", " print(test_data.shape[0], 'test samples')\n", " print('test shape', test_data.shape)\n", " \n", " print('Read and process test data time: {} seconds'.format(round(time.time() - start_time, 2)))\n", " return test_data, test_id\n", "\n", "def create_submission(predictions, test_id, info):\n", " result1 = pd.DataFrame(predictions, columns=['ALB', 'BET', 'DOL', 'LAG', 'NoF', 'OTHER', 'SHARK', 'YFT'])\n", " result1.loc[:, 'image'] = pd.Series(test_id, index=result1.index)\n", " now = datetime.datetime.now()\n", " sub_file = 'submission_' + info + '_' + str(now.strftime(\"%Y-%m-%d-%H-%M\")) + '.csv'\n", " result1.to_csv(sub_file, index=False)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "5cd17907-f902-4012-5d7d-cf114801e027" }, "outputs": [], "source": [ "train_data, train_target, train_id = read_and_normalize_train_data()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "43fb4638-0d07-5a47-18cb-e3eaf64692bd" }, "source": [ "

... taining gcForest ... (can take some time...)

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "6ff9fb5e-a96c-7b7b-e16b-63006e1d8c06" }, "outputs": [], "source": [ "#change windows size depending on your ram and core/thread \n", "gcf = gcForest(shape_1X=[128,128], window=[128,128], tolerance=0.0, min_samples=7)\n", "gcf.fit(train_data, train_target)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "e7520b7c-53fc-6360-7a68-16566db37edf" }, "outputs": [], "source": [ "test_data, test_id = read_and_normalize_test_data()" ] }, { "cell_type": "markdown", "metadata": { "_cell_guid": "869ab32c-197a-9bd8-5f5f-93ebba287d0b" }, "source": [ "

... and predicting classes ...

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "c0e1e483-e0a1-c780-12ac-99a49dfbfdc3" }, "outputs": [], "source": [ "pred_X = gcf.predict(test_data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "ec72ca17-e7c7-58b4-8ef5-b151eace0783", "collapsed": true }, "outputs": [], "source": [ "clip = 0.02\n", "preds = np.clip(pred_X, clip, 1-clip)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "_cell_guid": "1e9a7252-05c1-8698-6785-b49318f3b007" }, "outputs": [], "source": [ "create_submission(preds, test_id, 'gc_forest')" ] } ], "metadata": { "_change_revision": 0, "_is_fork": false, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.0" } }, "nbformat": 4, "nbformat_minor": 0 }