{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Foreword & Remarks\n\n- This work was adapted from the 1st place solution of the Jigsaw 2020 competition: https://www.kaggle.com/rafiko1/1st-place-jigsaw-post-processing-example\n- First place post-processing details can be found at: https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/160862. Main idea was to track the delta of predictions for each sample using successful submissions, then averaging them and 'pushing' the predictions in the same direction.\n- I only configured it a little bit so that it can be used in this competition\n- This notebook uses **only public notebook submission files**!! For me, using my ensemble as the \"best sub\" with minor tweaks to give a small boost in LB\n- Since we are allowed to make 3 final submissions, if you are willing to risk it, feel free to have a submission that uses this pipeline","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Update Log\n\n- **Version 6**: As per Chris' feedback, submisison dataframes are now sorted before their predictions are concatenated to the test dataframe. Also changed a public sub to another one as the submission could no longer be extracted. ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Imports","execution_count":null},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","trusted":true},"cell_type":"code","source":"# General imports\nimport numpy as np\nimport pandas as pd\nimport os\nfrom matplotlib import pyplot as plt","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"WEIGHT = 1 # best to keep between 1 and 2 from the orignal authors","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission = pd.read_csv('../input/siim-isic-melanoma-classification/sample_submission.csv')\ntest = pd.read_csv('../input/siim-isic-melanoma-classification/test.csv')\nsub_best = pd.read_csv('../input/eda-modelling-of-the-external-data-inc-ensemble/external_meta_ensembled.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"files_sub = [\n    '../input/minmax-ensemble-0-9526-lb/submission.csv',\n    '../input/new-basline-np-log2-ensemble-top-10/submission.csv',\n    '../input/stacking-ensemble-on-my-submissions/submission_mean.csv',\n    '../input/analysis-of-melanoma-metadata-and-effnet-ensemble/ensembled.csv',\n    '../input/eda-modelling-of-the-external-data-inc-ensemble/external_meta_ensembled.csv',\n    '../input/submission-exploration/submission.csv',\n    '../input/rc-fork-siim-isic-melanoma-384x384/sub_EfficientNetB2_384.csv',\n    '../input/train-cv/submission.csv',\n    '../input/triple-stratified-kfold-with-tfrecords/submission.csv',\n    '../input/rank-then-blend/blend_sub.csv',\n    '../input/siim-isic-melanoma-classification-ensemble/submission.csv'\n]\nfiles_sub = sorted(files_sub)\nprint(len(files_sub))\nfiles_sub","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for file in files_sub:\n    test[file.replace(\".csv\", \"\")] = pd.read_csv(file).sort_values('image_name')[\"target\"]\ntest['id'] = test.index","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Derive the given sub increases or decreases in score\ntest[\"diff_good1\"] =  test['../input/rank-then-blend/blend_sub'] - test['../input/triple-stratified-kfold-with-tfrecords/submission']\ntest[\"diff_good1\"] =  test['../input/train-cv/submission'] - test['../input/siim-isic-melanoma-classification-ensemble/submission']\ntest[\"diff_good2\"] = test['../input/rc-fork-siim-isic-melanoma-384x384/sub_EfficientNetB2_384'] - test['../input/submission-exploration/submission']\ntest[\"diff_good3\"] = test['../input/analysis-of-melanoma-metadata-and-effnet-ensemble/ensembled'] - test['../input/new-basline-np-log2-ensemble-top-10/submission']\n\ntest[\"diff_bad1\"] = test['../input/stacking-ensemble-on-my-submissions/submission_mean'] - test['../input/minmax-ensemble-0-9526-lb/submission']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test[\"sub_best\"] = test['../input/eda-modelling-of-the-external-data-inc-ensemble/external_meta_ensembled']\ncol_comment = [\"id\", \"image_name\", \"patient_id\", \"sub_best\"]\ncol_diff = [column for column in test.columns if \"diff\" in column]\ntest_diff = test[col_comment + col_diff].reset_index(drop=True)\n\ntest_diff[\"diff_avg\"] = test_diff[col_diff].mean(axis=1) # the mean trend","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Apply the post-processing technique in one line (as explained in the pseudo-code of my post.\ntest_diff[\"sub_new\"] = test_diff.apply(lambda x: (1+WEIGHT*x[\"diff_avg\"])*x[\"sub_best\"] if x[\"diff_avg\"]<0 else (1-WEIGHT*x[\"diff_avg\"])*x[\"sub_best\"] + WEIGHT*x[\"diff_avg\"] , axis=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission[\"target\"] = sub_best[\"target\"]\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_diff.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.loc[test[\"id\"], \"target\"] = test_diff[\"sub_new\"].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index=False)\nsubmission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.hist(submission.target,bins=100)\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}