{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Handling Imbalance dataset\n\nThis notebook explore different technique on handling imbalance dataset\n\n**CREDIT:** Infrared and Gaju Ahmed\n\n**Suggestions by Infrared** [here](https://www.kaggle.com/code/infrarosso/tps-oct-2022-eda-lgbm-model-ensemble#Targets-Distribution-(imbalanced))\n* Use class weight option in ML model (if supported)\n* Imbalanced-learn tool which handle imbalanced classes [here](https://imbalanced-learn.org/stable/)\n* Use ensemble/bagging techniques to reduce effect of imbalanced\n* Validate the model with Stratified cross-validation or train/test split (Implemented in Infrared's notebook)\n\n**Suggestions by Gaju Ahmed** [here](https://www.kaggle.com/code/gazu468/tps-oct-22-simple-eda-and-xgboost#Class-Imbalance)\n* Random under-sampling or over_sampling\n* SMOTE (Synthetic Minority Oversampling Technique)\n* Change performance metric (Confusion Matrix, Precision, Recall, F1 Score, AUROC)\n\n**Code example in this notebook**\n1. [Stratified cross validation](#section-one)\n1. [Class weight in ML model](#section-two)\n1. [Bagging with subsample](#section-three)\n1. [Undersampling](#section-four)","metadata":{}},{"cell_type":"markdown","source":"# Load Libraries and dataset","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom catboost import CatBoostClassifier\nfrom sklearn.model_selection import cross_validate, train_test_split\nfrom sklearn.metrics import log_loss\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n# for dirname, _, filenames in os.walk('/kaggle/input'):\n#     for filename in filenames:\n#         print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-10-06T04:35:21.995139Z","iopub.execute_input":"2022-10-06T04:35:21.995547Z","iopub.status.idle":"2022-10-06T04:35:22.003129Z","shell.execute_reply.started":"2022-10-06T04:35:21.995511Z","shell.execute_reply":"2022-10-06T04:35:22.001748Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I have create this dataset in parquet (float16) format to speed up the data loading time [here](https://www.kaggle.com/datasets/alvinleenh/tps-rocket-league-data-float16-parquet-format)","metadata":{}},{"cell_type":"code","source":"train0_df = pd.read_parquet('/kaggle/input/tps-rocket-league-data-float16-parquet-format/train_0.parquet.gzip')\ntest_df = pd.read_parquet('/kaggle/input/tps-rocket-league-data-float16-parquet-format/test.parquet.gzip')","metadata":{"execution":{"iopub.status.busy":"2022-10-06T03:45:18.782011Z","iopub.execute_input":"2022-10-06T03:45:18.782425Z","iopub.status.idle":"2022-10-06T03:45:21.244677Z","shell.execute_reply.started":"2022-10-06T03:45:18.782393Z","shell.execute_reply":"2022-10-06T03:45:21.242991Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explore dataset for imbalanced target\n\nTeam A has the ratio for no_score:score of 16:1\n\nTeam B has the ratio for no_score:score of 17:1","metadata":{}},{"cell_type":"code","source":"print(train0_df['team_A_scoring_within_10sec'].value_counts())\nprint(\"Team A ratio for no_score:score \", \n      np.round(train0_df['team_A_scoring_within_10sec'].value_counts()[0]/\n      train0_df['team_A_scoring_within_10sec'].value_counts()[1],2), \":1\\n\")\n\nprint(train0_df['team_B_scoring_within_10sec'].value_counts())\nprint(\"Team B ratio for no_score:score \", \n      np.round(train0_df['team_B_scoring_within_10sec'].value_counts()[0]/\n      train0_df['team_B_scoring_within_10sec'].value_counts()[1],2), \":1\\n\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T03:54:55.696168Z","iopub.execute_input":"2022-10-06T03:54:55.696718Z","iopub.status.idle":"2022-10-06T03:54:55.801909Z","shell.execute_reply.started":"2022-10-06T03:54:55.696674Z","shell.execute_reply":"2022-10-06T03:54:55.800601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preparation","metadata":{}},{"cell_type":"code","source":"def generateXY(train_df,test_df):\n    def preprocessing(data):    \n        if 'game_num' in data.columns:\n            data = data.drop(columns=['game_num', 'event_id', 'event_time',\n                                      'player_scoring_next','team_scoring_next'])\n        if 'id' in data.columns:\n            data = data.drop(columns='id')\n        return data\n\n    train = preprocessing(train_df)\n    testx = preprocessing(test_df)\n    TARGET = ['team_A_scoring_within_10sec','team_B_scoring_within_10sec']\n    x = train.drop(columns=TARGET)\n    y = train[TARGET]\n    return x,y","metadata":{"execution":{"iopub.status.busy":"2022-10-06T05:08:28.102986Z","iopub.execute_input":"2022-10-06T05:08:28.103513Z","iopub.status.idle":"2022-10-06T05:08:28.113484Z","shell.execute_reply.started":"2022-10-06T05:08:28.103474Z","shell.execute_reply":"2022-10-06T05:08:28.111951Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clfA = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,learning_rate=0.1)\nclfB = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,learning_rate=0.1)\nbase_model = [clfA, clfB]","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:29:09.075510Z","iopub.execute_input":"2022-10-06T04:29:09.076016Z","iopub.status.idle":"2022-10-06T04:29:09.082987Z","shell.execute_reply.started":"2022-10-06T04:29:09.075979Z","shell.execute_reply":"2022-10-06T04:29:09.081727Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Basic train/test split - CV 1.938","metadata":{}},{"cell_type":"code","source":"# Train test split\nx,y = generateXY(train0_df,test_df)\nX_train,X_val,y_train,y_val=train_test_split(x,y,test_size=0.1, random_state=0)\n\nloss = []\nfor i, feature in enumerate(TARGET):\n    base_model[i].fit(X_train,y_train[feature])\n    y_pred = base_model[i].predict(X_val)\n    feat_loss = log_loss(y_val[feature],y_pred)\n    loss.append(feat_loss)\n    print(f\"CV for {feature} = {feat_loss}\")\nprint(f\"Final CV = {np.mean(loss)}\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:36:27.299934Z","iopub.execute_input":"2022-10-06T04:36:27.300362Z","iopub.status.idle":"2022-10-06T04:36:57.353213Z","shell.execute_reply.started":"2022-10-06T04:36:27.300330Z","shell.execute_reply":"2022-10-06T04:36:57.351864Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-one\"></a>\n# Stratified Cross Validation - CV 0.2073","metadata":{}},{"cell_type":"code","source":"x,y = generateXY(train0_df,test_df)\nFOLDS = 2\nloss = []\nfor i, feature in enumerate(TARGET):\n    # Cross validation\n    cv = cross_validate(base_model[i], x,y[feature], scoring=\"neg_log_loss\",\n                        cv=FOLDS,return_train_score=True, return_estimator=True)\n    ave_loss = -cv['test_score'].mean()\n    loss.append(ave_loss)\n    print(f\"CV for {feature} = {ave_loss}\")\n      \nprint(f\"Final CV = {np.mean(loss)}\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:37:08.880507Z","iopub.execute_input":"2022-10-06T04:37:08.880951Z","iopub.status.idle":"2022-10-06T04:37:46.569713Z","shell.execute_reply.started":"2022-10-06T04:37:08.880913Z","shell.execute_reply":"2022-10-06T04:37:46.568258Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-two\"></a>\n# Class weight - CV 0.2059","metadata":{"execution":{"iopub.status.busy":"2022-10-06T03:47:38.266707Z","iopub.execute_input":"2022-10-06T03:47:38.267224Z","iopub.status.idle":"2022-10-06T03:47:38.273295Z","shell.execute_reply.started":"2022-10-06T03:47:38.267188Z","shell.execute_reply":"2022-10-06T03:47:38.272270Z"}}},{"cell_type":"code","source":"clfA1 = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,\n                           learning_rate=0.1,scale_pos_weight=0.9)\nclfB1 = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,\n                           learning_rate=0.1,scale_pos_weight=0.9)\nbase_model = [clfA1, clfB1]","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:45:05.682740Z","iopub.execute_input":"2022-10-06T04:45:05.683832Z","iopub.status.idle":"2022-10-06T04:45:05.690658Z","shell.execute_reply.started":"2022-10-06T04:45:05.683787Z","shell.execute_reply":"2022-10-06T04:45:05.689466Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x,y = generateXY(train0_df,test_df)\nFOLDS = 2\nloss = []\nfor i, feature in enumerate(TARGET):\n    # Cross validation\n    cv = cross_validate(base_model[i], x,y[feature], scoring=\"neg_log_loss\",\n                        cv=FOLDS,return_train_score=True, return_estimator=True)\n    ave_loss = -cv['test_score'].mean()\n    loss.append(ave_loss)\n    print(f\"CV for {feature} = {ave_loss}\")\n      \nprint(f\"Final CV = {np.mean(loss)}\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:45:10.393994Z","iopub.execute_input":"2022-10-06T04:45:10.394548Z","iopub.status.idle":"2022-10-06T04:45:48.164583Z","shell.execute_reply.started":"2022-10-06T04:45:10.394498Z","shell.execute_reply":"2022-10-06T04:45:48.163054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-three\"></a>\n# Bagging with subsample - CV 0.2071","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:43:04.528376Z","iopub.execute_input":"2022-10-06T04:43:04.529378Z","iopub.status.idle":"2022-10-06T04:43:04.534715Z","shell.execute_reply.started":"2022-10-06T04:43:04.529324Z","shell.execute_reply":"2022-10-06T04:43:04.533329Z"}}},{"cell_type":"code","source":"clfA2 = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,\n                           learning_rate=0.1,subsample=0.5)\nclfB2 = CatBoostClassifier(metric_period = 100,verbose=0,n_estimators=20,\n                           learning_rate=0.1,subsample=0.5)\nbase_model = [clfA2, clfB2]","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:51:25.174644Z","iopub.execute_input":"2022-10-06T04:51:25.175146Z","iopub.status.idle":"2022-10-06T04:51:25.182988Z","shell.execute_reply.started":"2022-10-06T04:51:25.175107Z","shell.execute_reply":"2022-10-06T04:51:25.181272Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"x,y = generateXY(train0_df,test_df)\nFOLDS = 2\nloss = []\nfor i, feature in enumerate(TARGET):\n    # Cross validation\n    cv = cross_validate(base_model[i], x,y[feature], scoring=\"neg_log_loss\",\n                        cv=FOLDS,return_train_score=True, return_estimator=True)\n    ave_loss = -cv['test_score'].mean()\n    loss.append(ave_loss)\n    print(f\"CV for {feature} = {ave_loss}\")\n      \nprint(f\"Final CV = {np.mean(loss)}\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:51:32.359742Z","iopub.execute_input":"2022-10-06T04:51:32.360185Z","iopub.status.idle":"2022-10-06T04:52:04.076868Z","shell.execute_reply.started":"2022-10-06T04:51:32.360149Z","shell.execute_reply":"2022-10-06T04:52:04.075382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"section-four\"></a>\n# Under sampling (not suitable for this dataset) - CV 0.2144","metadata":{"execution":{"iopub.status.busy":"2022-10-06T04:47:31.236149Z","iopub.execute_input":"2022-10-06T04:47:31.236606Z","iopub.status.idle":"2022-10-06T04:47:31.241894Z","shell.execute_reply.started":"2022-10-06T04:47:31.236565Z","shell.execute_reply":"2022-10-06T04:47:31.240543Z"}}},{"cell_type":"code","source":"train0_undersamp = train0_df.copy()\ntrain0_grouped = train0_undersamp.groupby(['game_num','event_id']).head(1)\nimport matplotlib.pyplot as plt\nplt.hist(train0_grouped['event_time'],bins=60)","metadata":{"execution":{"iopub.status.busy":"2022-10-06T05:14:49.873235Z","iopub.execute_input":"2022-10-06T05:14:49.873644Z","iopub.status.idle":"2022-10-06T05:14:50.644420Z","shell.execute_reply.started":"2022-10-06T05:14:49.873603Z","shell.execute_reply":"2022-10-06T05:14:50.643146Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Remove event_time that is longer that 200 minutes\ntrain0_undersamp = train0_undersamp[train0_undersamp['event_time']>-200]\nori_row_count = train0_df.shape[0]\nnew_row_count = train0_undersamp.shape[0]\nprint(ori_row_count,\" rows reduced to \",new_row_count, \"rows\")\nprint(\"Reduced \", np.round((ori_row_count-new_row_count)/ori_row_count,2)*100,\"% of dataset\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T05:15:02.559064Z","iopub.execute_input":"2022-10-06T05:15:02.559478Z","iopub.status.idle":"2022-10-06T05:15:03.134203Z","shell.execute_reply.started":"2022-10-06T05:15:02.559443Z","shell.execute_reply":"2022-10-06T05:15:03.132866Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"base_model = [clfA, clfB]\nx,y = generateXY(train0_undersamp,test_df)\nFOLDS = 2\nloss = []\nfor i, feature in enumerate(TARGET):\n    # Cross validation\n    cv = cross_validate(base_model[i], x,y[feature], scoring=\"neg_log_loss\",\n                        cv=FOLDS,return_train_score=True, return_estimator=True)\n    ave_loss = -cv['test_score'].mean()\n    loss.append(ave_loss)\n    print(f\"CV for {feature} = {ave_loss}\")\n      \nprint(f\"Final CV = {np.mean(loss)}\")","metadata":{"execution":{"iopub.status.busy":"2022-10-06T05:16:44.949387Z","iopub.execute_input":"2022-10-06T05:16:44.950035Z","iopub.status.idle":"2022-10-06T05:17:20.730025Z","shell.execute_reply.started":"2022-10-06T05:16:44.949962Z","shell.execute_reply":"2022-10-06T05:17:20.728657Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![Thanks](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAT4AAACfCAMAAABX0UX9AAABIFBMVEX////+/v4AAAAdHR3DiUAaGhr///0ZGRkXFxcREREQEBANDQ0UFBT7+/u5ubn///wnJyd4eHihoaHx8fGqqqrChzvq6urFxcXj4+OPj4/Y2NghISHQ0NCzs7Oenp5DQ0NcXFxsbGw5OTm/gi8tLS2JiYlQUFBzc3NCQkJVVVWVlZVkZGSBgYE0NDRtbW3U1NTy6+DNpXbo0LTIl1rMoGjcwZrYtofEjkrs4M/69u6+eyTbw6TFjk7m1sHbvJb59ujTsX3PuIq6gijGjj3Hll7QpHXClVDDhEHeu57o3dLm18XYto++eB/HuaPXvIzKmmXgxq8gEQC4iE6Ddm82KROpiWR0VixiTS5OOiJyZ1q3k2aieD2YajO9fTUCEBnTrIcQUcucAAAcSElEQVR4nO1dCXviSJKVUkgCJIE4xX0KMJfNjSkwru6yy13lnnZt7+zszM7W+v//i41MnYCwAaW7qc8dXx02RyrzZRwvIlNKhjldWJb88dHCjyF4jCz1YbIEu3eAHsu8AXoYuPeBHksdPNY03HeAnqEmNFtkTZf3DtCzfBTVBt+H2TJmdPwLvhPlTUyMpT4hZyqG6lFv9T3gZ7KyN4HvHdBlk9W+ySjfQqf9CmV+8R5UxC004WPfKlM7X6GoLqZxvSPwzCHTGLGtc+8JPoYaQ2PP07e/qVA1XeYNyPJZC0uxfvEueNmmUKw8vp+U1BLKRaU3KPCds7xDa6MoPotKrEC1Nz+asD7rIsIXhV5nfjjxX1SaXVPqylsLdTMxsfMFnzBe0+/Ym0g0SrlBGkWl69WKdrfeRATmeiFQpQQU0iuBWY7rA0r9eWMZXrNRqkUl37kGWO1sXl/S6tNbihCdzJjoIepyCCJWhcWv8t1+TKXGvtr4A4T45uuPAfxTlPXta5y84ET0hGgU/IgAPRkBfKlbRhBwmwI946Aqi+n0ZvTT8GG6XP5yI/gPdKzP1IpdRIkownCYegDrjVq/n2MQFtjb4WS4Hs/nqZ+XAg2e73PfjvL8aTbF8tMyEEgFAuPVHCSQGp0jekSWD4HAKjAe0EhOWf9lkZt6PfAQwJhhAQMOBD7c3Z5r5ixEmWkgNU/RYfj+94xFmdvJh1TAJan6mj1L0yUSFe7Go/UH8DJUukijxLKs2/g9BOrzc87dWOZ6/O2WxN4zsQ+IuoNVyla958WZ9MtTWOZxDaaxeLo+F/hwAhQdp0DxQL6fdd6Le4ptg2UW1+fUz4WtfcM/uysvCGuRNIzcOZUnR/VAoH7/GUCsL/7svniLVQs2UTsj7KBvk1Qg9dtsPamn6qM/uzPeYu5mOxOHtym3gYdP09niejgMPEyo7pChJuwZL1yNfp78soQYsni8n9cXlAgVxdFaW23OET7gAZP7xy/4J0ZYjgM3NBqluqeS3radN5Hb1McFLoADkOyX+R2VNlm/K1eulhhzczWFxuiLwCynWPHwjwIwqns6NWdq+x38FpXeWAR2AH+d36M0qAvNnUrU78ehKgCdu0JApVpAcW+bdd8WlcZ+DKHm9yi39WMIbXU5Z9f3BvIOFObt3LF1H83btH4mQlM9Npp6FzuiWWr3hrOb6dm5pvWUhR6rZd0/+t+p9EMISy2ld22/Z888OaUmFHd9u+4qpdFgWtO0DIV2vEW50rSC712gFA3M5nhWyPDVsNaKdWtBVHsbAAutGsISS/hpxbeJse75s57CYt5E80KziUIx3slmsx38T7ZUjGi5xIYmJGIRrSyJHBfi3mCfcPqy1JK4IJI4pPtpx6/lJlTUdquHhdlLk5IoNnrxSEGLRIqdbC8Wi7UqzXKzmc+XK72klkng7+UamS4Kchz8QcXTu7dHtC60zqGa1gD8Cns+lM7l0t7fblRqtWa1E2E23NUp0tf5MNrV/xc0L9KLF7RktSahLZE5MUxe1JuXjYiGwhyoXpvnUPLk3u3tQi4PuDVglgBEzftDMdyV+O7riUqkUA4F4c284peaacG2uKseL7DIZFkrxDiELJxi2XgyEkmWsr1Wt6zaUGpxxAN6Yb2A9qvHqRLpFZDIk14noXlv35pFTZ73UAylic1CQt1YK+07KajF86Aeke2X9yl0Ts5m2kiSAbleJOfxESWdK2jFltZBAB6HLpSSxIuUfd/V5RXiRWNSasFg3/NDBVS5EL2Ur53WQXGrBq4+0buqgXZwu3O0B744yiWRLKFa3NupmJIA3SPoNRgFtCPrp4cerbczGL0c/llDezxrBukRJHpElXikK3FS1V4s9dWVSqEZ5KTY9st72u2E0lWw2svczjuJZKek2V9JRgz0wOfFJC7ki1nsSjctg+VekX7qwaBn4L0C91ENeXgN5QL6JhJ78M+V03oOK9+BxKyEMmWEuh6f7uhaoYzCZohI5mz0CihPKe7aM1q6aoYsd1Pa41gjSOoWULiy+04xphuxjEYxpKdVZBLADpECapRRyCPMKRfZTAVICm+E2EKuH8TogU4rKo/thIZYITJX6tj+AIzYs/MleD0DhN1jopuVECeqinWHri9RsPJJlwd+Wq81Uc3D5yV0LYlCHGYCWNGUJAkbPAedq0qyt2M/WlgrCYpdIS7UNF5shsLl3Y8ql0iSekXkhWwB8aoqldy5/emSLLbCPH9gXNRQPyR7uDFFv+phD6BH0sSOOqbpApQdFPT6xkliLtQnry6CPDImsYTE4G7zGcjlSigdEiWPS3fDXF4V03TS3HIG7MAjtnvKZZjfJTggNa2F0euZv+Yi3RBmfEgBD+RpPyeJWVRSGhDTTRoO7AXtBrEkQmo62el4MmYYL8DXplMb1ToNSTxU+ZgQH7zweLmajWH0Stbvl5gnA3AXYChBiRJ69p75Rg7xsmGwaSTu5huZJkJVlhHhTd0DoUsJ4OOSdJSvm5P4wxMqb0WN6ET3bDcTibTlcJVFYi2HZJma7pnopUs9ySQKCVXeMYYc5GkheDGZjElelkKUTwejplHfS8cgsVIP/TROLndNheGBhHJSxfqVrWAim1YknpflPLVKlcVaYrgXxE0k+uFtgBohCF5ZbEwXORSseTRTlXBMa9JZeW1c6d7ezFMKnsmlhkSOC6q2A4hrTRmzCuBDvJqm9wgag6KlsxDqSIqUUXfUK470bpFEi0iyJXnVERw6SqFfShvGfjityCCvygnpkaOVSlkDf67gsYTkIFfa+fxJYt/fHMNz2GHwrO3ioygWKE3wG15uui0T+F7MNw+WYrEi76v1eEk7LOZ3woyGHZ/jEkvFZhD/WkLoso08yh3Hi8vQErFYmNQfOkh6wTFo4CC9BqYR5Qt6MMVTpLnHQeyTBBLl/nangcM7jo9hVVA+nlTaDqVDr4prFaxTRDjEpyG6Nl/gC+U9yqcHifLR6VmmlfWco/0CvQpuVU/AmHgXO40nQfmSCiTGRzW8V9jN4kW3J/GqUgLD7bzwpUKs43l9owh0cIL/isQ0UT5G+RjC50NoY+GiLG/UCrki4vtaCJmWldhnvLliNhaLvVz0Al+m2NugsSSrDZETm32gxTkm0eAR6nlGgPJV3mtgCVK+5Ta4qwLMtxF5hfgWW53dqgQrR9DhYdeSSLvmrp+AN5G6zq/JRlvmdbAsDJtSyiNU8Wgjna2Vm22dlxDaDC2FYsRhRoUKQlLORdCSKF8F1y9KqKfAhTEYnrWc3IX3wAhp4TiX4rKdMJIlCZKUl4asITAmfvsjxUY77Dub79vpJ0ghooJvUk1eVtAxpdkNHoleJd6riaSgL26MP4JfMsedqOKFEnBTzlJEC1VqPBdEzRy2Q6xJobyXdleKzZAHmTXihpsl5NSLbo3DS3Wt7T66lgxZqaDzfHj7ShdF5LsQV3QnIqAQlYqeV41WS8jw09uXLVW1BmhlNZJJZJKyW/3S5QvRGlyOkyA7yBOGZmmfUu7KnFzD+JKKTtCz+gOpnLat1eTbeVHEHXKYWpHvNUWkx7NIbm58NKuHpXzVumy2FPdgbBk1Jqk+lyAUVXStMzRQuBdU1T42OFAVfpcksEyuUkiGkVQyQU2756+k9XlzIBrKq3lVVwFMJ7XP6WWRl/FPGD0R1bw9T6MK+dquVrZQsKxCjyQru0yWI3qIBEJxMxw2OJF3qohpCOOcvMN1sl3VY46Ok/hGqTeNapW82iRLf8AswhbBd0mnkbkw0yqGOLOYKwu8xPkYgaSIwkjV9by+wT70ct4oTMBleaTucdssKkrStjHiCYH8ijB8s8NJSC6DxuWy5W7bCdTFSp93OchqrhbkjIUBt6jVkF9Sq4R4dzW8iaqqKuHeKTUE7AU89eYamFLRkgiVLd5QAoaIkP1uohdHYh7rWhLHnh4on+xWzp6kYtIR07DHkPbWOYoi0LGd0SoYvSTAx5vEXwOl4k12cyWFwnY/Mr2m7CofFRolr3pyQW2infWhI6WzkQT3kNpVdUwYEn0E7LqI0xE3+8qUM1Xk2EmJ6IKjQ8UCxBM86YBeh1F0Pq+72YeGAD25WkM1KYhD7z4pVy486tsVCUYLuFiLYplQRhQt02i4V3tiSeSULhmmlvFch+w1PcsnxwjwKNcKXRHJ7TKpfiVqOCUAv8htuORcM33hYrOEw4bazvuXZkkiSaJvSVT1sMs+FYlUYLMIiIv+wqJ7BrWk3XXRJApdMAnchNmBC60r2epDVmpNhe3kZF5yehVvZCVul6Mrqu57DSKLXMn3FQr3myqupAF62GVheNwTVCinVckp/yXRVgKQzjYkuY2nAaOXkPK66vbXFUzaxBq4r1D1pYDXyVeCO3oBdhpKE+bCh8iXk90IctY3M472ZbIdxDvLC2lUwIW3HXJeQJzfwJFwK3kCMgAgLflLoBcIszGF47mwy2Xk+mku5IyrSNBzLy/HCyounkWMZDmLNgwb/MQFVp343kVxS2ptVdwmfYoexMrVgBkIEY6voJwqOp0B+Pi88WM1g9zXLde2SEs6Tr6FS7Y+yzbuJtgLVMnk8zqYBsRcPJHxzdXjNEpfhJx+abvZZwXHRswdMaRKmOfcKGjAdSTsg2JhLlR5oVMZ1AyGtxdGwU4x5BdBa0pK1QhyReeIve6ZKxkmYEoWXZZFMe+0lFCJ8StBtMO0jxRQPslWvhgqM1kwN8R0zbwBez6XY+sXIGrYHy8YpFCuWC+wTK8TC0uNHJKIu0xuVsESgHwbKHPXoB4vGG+Sz+9UVLKIOK80slk8uqoFXdrTDVmKoOJ1YztsF5Bc3fR8lTCxFyXW8sv5GpKjPGnUVxiVy0vZnhlaI5thN3uRdRVmC0gMqVthF2kcJmayse7YFzdQKqMWzBYXBtUIch7bmRyp4u1Om5GlaM4b9hfGjF6hHArZmbqStZWvUoEe2JOekMV8U9wkFzXl9UL1IYVsCLuWkbBMupNgNKBlwQoyo1Yl5B5/EYHnt2dVg1Q3zm8kdBrSk4hX+0Fj6DgUOuMDjb4w0rS8jPO2HeN0Cd7dsFmQAndguOCWZLH4TizuUABg+MamA4xOs+Zau2tKohrkXBXROC4kvQ7OIfDhvSUWHsYaTh7naCJPIMG64pgmqCLP2Vs1ACdUwiV+J6ErIKkLuID+GTaBnbwTIS6RmsDOAL+fKXnSMEsSaLsWD+gZF2bzvPVWpdh1UgbNtpMGCkG2Jlnf7CEpFnKHtySSC7Q2kUO2a47evCPHcGem4WDbdZJL8palE8b+FBxZ7I7BEGuXeBXCKlvrokuHKkjMEH0kq6GEo+2NvZlt+Gz0GNeEqYWgk8TC6yJWMKWK1FbIYQugoPGOrbAMWX6I0LpHSDPie6GbM9dfzQ1pppvNuq7bQfgti0G3DEdDPmC6eODJxYsKWJ1o7qJUCAhGRxNlXPTDu9xg9HjdG+ZiN4G35AptxvOkjR6xfqkRaTRrF6FW2HENJEoVFU1HElZti86Bx+zimGJ78J4bvbRP1tINAcsAVegrZjUdOztn0YP4GcPDXyISNQ1VSzTNDrYc82xAfpwLqq5KXMaBL5MnPEgxdn0UrG/uy5eMZT/be8WQk5iWyZ4vIlKND1moXiEZ44c3y2ZICcToVAlJZYVkeYY9KRWyJG+i1/O3lx+TAOhABklpc/89cTpBex2uGzaVK13DRUWYYRlPtyaGRQNVAkLJABQ+0CP5lGUmV7YJFpEUwv8bhIV4A20zrOz0y3n3qo9s0pHJkhZqJXwTDb6fwazl5CSpViSY4sYLZgxOtJGEFYPAhycjo2LSaha+E7EM5y9jAzzCsbTqNEmKGY5SYFsToQNxczUWK10mU0WiFDQ+g41X1BPYRvAHSOHdJqJEh8B1poFEXhAXj3XbWr7g+f2rPQrPWy7kqous8JpI9okDsXwmiRYFo9vYftLFZMSIJE0gCMkriCGEoZBJBiKjdFBQKlibURPlq+CLK1WvC15uDhHmbcIH8+Ter0CyiiAP5Ny4oUipILx/QkItUz9JKAjKYJoXGXM8rvICrvJzSMc3DJHWsVLJVkTAerRVHnbEWM9A+bYMF6sQTDL49oRuHk+X9aky5ipaOlLeXu8ETxuGT4dRl7xMUstwXkJBXGUzLbdfDiGvVdDDRTFWGvqKhV4GBTdXZypIlmW4rE3OI21VrfUcn9U1PiDGjUGHUcU1DoiXoRDgXTO9WBx8U976btpWpKvdol+GIM/xcgjppD+5CkL5Tjq9sQdMaaMwhgmFdpKHSLev6l3Le5aNXubjrsXSmIwu3JhHWeHIx20owJBEvD5l7UwvoX5pk4yV2s1ma2MFkN3MteLOBxSIj+4CJ8tkOt1KpWGHgAtULjqMC5sUz0XKLd3DgWshIzy0DfDAgvtFxWSSrh5Gqs1mu+W9Qul+sdRsVmIRYxu5/X7CzV5YgR0xRz5xOtnt662E83Rz5fRFE8wZybqWQMT43+kbFpY0b79IAjOP5LDn/Q6KlowXNYO8g9nmDX8H5rJ798AxvdzL91j29vPJz131+2R9gTyW29iIYaHGGi9agh/jLQhRF6g9wjFFpL88acmajjpG/yJH3D2wJa/uHxduAsZzmv60ZxAK+NCJBciACPwQffEQihY2z35nL3okQLYrNXtZpQyZ8j6y86Ic8pCD6Wp6StOHCX6OFn4UvMu9CtHF4Pr662i5fnwc3t+N56l6/cOHD7/++mFD6iCfPn2af767u5tMhsPh4+N6vVwuR6Prm7/97QsBeXvKTbUVmEj50mEXV+gFqv2CmLf67gfvy+NytPwtMF4u1+vb49s/QjBkN6PpbPi0CtQBKgxWvR4IrMbjp/vhM0AznWJoduXbEsvj42w2HE4m49UqkAqkUh+MFlKr8RMAO11+G91cm4oLV2t0Wq66VTW8n2nvFdbWu33wCaxwMxnOhoFU/dfVgJbxbsRxQA2DNhljDfsOw53fDWfrJYwVBAwV/v16Mxot8fwBQDOM0bY8P8PL6/WUaN4NFvgSfPEGvohVeDh5Gs8DRIPhAuOnybAVKf/H366vAUvcB6BVZP3DeMwsec7sIS779buOBEZYDL/jx+z+tKDm+qJG7Fx8uVkP7+bYBOupVGA+nszW09ENgLVcz+6f7kCL6o6kDpK6W6Db8zEGa/ibYdQwBbPZx8nk9//UuL8bjQdWk/vH//rHf//zXyaUZNQQjg6gauzrbg8DGKjPH8bHUpeXGx2MZmMTkwAZ5ASsbvwZbK7+c/17ynrDn+Am3NDC3/l4NZ78T0dD/xxO7u/hkmP85O8UPvwAkLy7f15/+3o9WBw01oPOTRGYr7/+tLyvD2joHmlj8e0+9euvqTsww4lrrB7DN54KvwXIYcg97L5E/p3PU7+XNPR38th0cKvgVwHE+YMNNlFbsITHJbhK57GuRsjZMMCDTl8QmOfHm8Vy/o0CenCt258mk8fR9SB6uwR3NH9VfTxssv6hvke+f3d0zbPB+epzAKPH/W5fIWWAOp+vVq7eWNdMzSez5bevg9uoMfnC9nhe85FCNDUA1G/mNKrQguECFtfrMViMF3g2WNgbQhDBIXOJA8kXg/DhYEKiw/XAkS9fSIDBEWYNAWY4vH8aYzQ2gcctzgMPjQKS/v3atLmmwGzi83D59fYECxzcsgoMeuDrEZ0CdscYOuXLcjgmQzHk4SHgaBiMbzUePi6J//H/PGQBWMotxhuAJcxmeD95+lcOoX/Uv+OrGeGIsJzUp4DjJz7PgD+uh093c9wr80O4hx8Ckxm2HNyzKMl8/sBEYnFtxAuXhZHugcsGC8Gs7Nb22Tjw+T2gA5oQhO1iRzajS80FhtSkNHe///t//303J7Nnh3hgi8T3DW5B15ePw/H8E5lcQt/rgfFwPfqyOCzA+BUjXNysJ4R2fa+DTd5/XD+OV0/D2fTmC4S5jQGSLFcwGdjJAJrkTTAeDyywxl+WiUR6aDO3vUT/hzSiqOAZwAMQ0jQO1H/GKU59jpG6gXldYHq6XD8DmPfjwM94KKnP94DxwACRqKKPDhuy6x8FZjAargLzJzBKrGFA+vE1bv+YubO6Zd6zpVQKaHNrZATt7o/CSeTiFvRzOcO8FLLFD6kVNpAvC5J4sUD1v2KdTOHEJjCZAr7Wd2mcRrX5y+2tCZnZPPGCLK6D/XGPTTSfHNXB99O5lS+NeOTO1nDpJsq6ObOgAFaj6eP9CmeS9dU9CQLGB6K3g5vl81Mg9SE1nvwGmnhKbNkSYfDVdxtYaCJrPgqaVXNocz9rORSuHXwhMO5vUwh8gY+/fB24ixGgrNej9WyCjexx+XWQMKbgNOX49njKtzaE7vkB9o3mkVgJOduWGFwYlE/Y/R6FeLIe3n/8CZDa9EHgIJePQGzhjVMCH34IPnMfYFifD9O2ntJ28GX3X411FZViye7GnQ5JFFITh1/HjkHm76CNo+lshmP0rUGzjLREwCgCWb0+cmZIw7fPP/s+euvIk04G+8PQxvkB5Y65O9oQDUk6hVtPwXJB52azNVZGdy3p9sjoGF0+LZfT376PJ/fjqS/jO077HvfPF+s83gXfuVDjnbpeBKGy3/thBSf+4crbt/Xz8y/fro3a/AnU4ub31R3kYvXvSz/Ge9xTyAVh/lIx3LUW0eTwHRn2cjC6pOFft60EWy6klV8JoQV4owf7MWhoMYQM59PE14mSr5chNz892H/kCLsxFVm8/CxjlWMjfUT/mYGGCFEgZuD/vt4AitfHnIIHPb0Bpv596udYsKPjxnJc37eOsOlDE2SNXEbNJkKoSeeO+53uGNmOqfMsXrTBawSHHq8+GY+mk88+Uopjj+tg8THXo/103z0VBQnJoihK4PXo3I19mAjRxGKhHDKmxXy0iE5Tpx9Ac/z5YgPwF5MDHXUi3tR1vdnxecPPkYKJ1Xah1VtGa6y/o5OPbjSz01fhM3cbQIrFfHtOBeqLqPnrH5lKnygvOPYFfivKLE6kzQcfXcE6rH42BPic477O6jxBT3lhhMRpnpxyHB4yhMVwhVfPltPlFJc4V/eTyd3d3efV6EdQv7dok315zXlHph8C89VqNX56sArsgXrgnM8atuRtCk2vbXjY+rTADH4nS4wPn6wlivpsZ1PGGYrj2ikXlY6dluisvrEsd/NDeD5ziZPq+TNHs2UsAjua2yuS9cninI7w3SusvSmL2tFsB5wf4CUQom4/W8r36I7GZyu2ktA7lJKxGjqhsUXAge/8wbMKt5bxUkLv9GiEj7lOjbEJP5w/Y3EBdsgGj8ObPfFUAoEZpub12eP07pObOZ+rbJx4RPNQytOmQmAXgdT36fNg9HwPrIVKb95KtoMjtbjLHlff2+jSqD5ePkIEGXy8T6XO2Xq3U/nzOJRyMlkbR9NHH8epmzfJhujItnc6i5MBbwND46xhCLo3n4Y0u0QrLlrNMZsadxaHUn57jDJk4RADN7ijcU6zJTRPPzzdPb2tuFcoo+yCInyv3nJwTEv0GqMqGzVcXNKlGDxOJlO7DZ2Jqf6BQjWlOk/LfUs5oX6xtymrxvJ+hOZxzTTOi/qhhGVo+T3S2p96dPafcGHqA35fpntK4fsvMeXYXQ5/iVvoHaP97uS0FZe/xBD2XWYI1OS92S1tOnrcrtYfXugWld4bXTnkZvjDWzp1qfQk+X/c8qJgiqs/sgAAAABJRU5ErkJggg==)","metadata":{}},{"cell_type":"markdown","source":"\nPlease leave a comment, if you have any suggestion on other techniques for handling imbalance dataset. I will continue to improve this notebook for knowledge sharing.","metadata":{}}]}