{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Objective\n\n### Disclaimer: This notebook is not intended to give a high LB or show an awesome blend but rather to show how the blending can be done.\n\nHere, I will blend multiple public submissions with 3 personal submissions (which you can replace with yours) to improve the LB score. Score ranges of my submissions were from 0.87 and 0.955. If you have better LB scores to begin with, you can definitely get better results than mine. Note that this is the final step in your classification, develope your model first and then polish results using this technique.\n\nWhy this approach works? Imagine one of your models rightly thinks that an image is a hit and gives it a very high score (e.g. 0.89) while other models fail to identify it as a melanoma and score it low (e.g. 0.3). Then, averaging scores can lead (depending on the weights used) to a score above the threshold and correctly classifying it. However, this is a very hand-waving argument!","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"!pip install xgboost\n\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.datasets import load_iris\nimport xgboost as xgb\nfrom sklearn.metrics import accuracy_score","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/siim-isic-melanoma-classification/train.csv')\ntest = pd.read_csv('../input/siim-isic-melanoma-classification/test.csv')\ntrain.head()\n\ntrain.target.value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train['sex'] = train['sex'].fillna('na')\ntrain['anatom_site_general_challenge'] = train['anatom_site_general_challenge'].fillna('na')\ntrain['age_approx'] = train['age_approx'].fillna(0)\n\ntest['sex'] = test['sex'].fillna('na')\ntest['anatom_site_general_challenge'] = test['anatom_site_general_challenge'].fillna('na')\ntest['age_approx'] = test['age_approx'].fillna(0)\n\ntrain['sex'] = train['sex'].astype(\"category\").cat.codes +1\ntrain['anatom_site_general_challenge'] = train['anatom_site_general_challenge'].astype(\"category\").cat.codes +1\n\ntest['sex'] = test['sex'].astype(\"category\").cat.codes +1\ntest['anatom_site_general_challenge'] = test['anatom_site_general_challenge'].astype(\"category\").cat.codes +1\n\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"x_train = train[['sex', 'age_approx','anatom_site_general_challenge']]\ny_train = train['target']\n\nx_test = test[['sex', 'age_approx','anatom_site_general_challenge']]\n\ntrain_DMatrix = xgb.DMatrix(x_train, label= y_train)\ntest_DMatrix = xgb.DMatrix(x_test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"clf = xgb.XGBClassifier(n_estimators=3000, \n                        max_depth=18, \n                        learning_rate=0.15, \n                        num_class = 2, \n                        objective='multi:softprob',\n                        seed=0,  \n                        nthread=-1, \n                        scale_pos_weight = (32542./584.))\n\nclf.fit(x_train, y_train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"clf.predict_proba(x_test)[:,1]\n\nsub_xgb = pd.read_csv('../input/siim-isic-melanoma-classification/sample_submission.csv')\nsub_xgb.target = clf.predict_proba(x_test)[:,1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_new = pd.read_csv('../input/siimisicmysubmissions/sub-new.csv')\nsub_mean = pd.read_csv('../input/siimisicmysubmissions/sub-mean.csv')\n\nsubmission = pd.read_csv('../input/siim-isic-melanoma-classification/sample_submission.csv')\nsubmission.target = sub_mean.target *0.71 + sub_new.target *0.15 + sub_xgb.target *0.14\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv('submission.csv', index = False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Just blend (or rank then blend, see https://www.kaggle.com/ragnar123/rank-then-blend) your submissions and enjoy!","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}