{"cells":[{"metadata":{},"cell_type":"markdown","source":"The evaluation metric of this competition,\nlabel-weighted label-ranking average precision,\nis only affected by ranking of prediction scores.\n\nI tried to transform raw prediction scores according to ranking of predictions before averaging them.\nThis blending strategy greatly improved the evaluation score. (〜 0.881 -> 0.895)"},{"metadata":{},"cell_type":"markdown","source":"My best two submissions (public score = 0.881) were obtained by ensembling and blending many convnet predictions.\nThe file submission_879.csv is from [[AutoML] [Inference] Audio Detection - SoliSet](https://www.kaggle.com/hypnotu/automl-inference-audio-detection-soliset?scriptVersionId=52918021)."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nCLASS_N = 24\nfeatures = [f's{i}' for i in range(CLASS_N)]\n\nfilename0 = '/kaggle/input/rfcx-my-best-submissions/mybest_881_1.csv'\nfilename1 = '/kaggle/input/rfcx-my-best-submissions/mybest_881_2.csv'\nfilename2 = '/kaggle/input/ensembling-0-880-audio-detection-101/submission.csv'\nfilename3 = '/kaggle/input/rfcx-my-best-submissions/submission_879.csv'\nfilename4 = '/kaggle/input/rfcx-bagging-with-different-weights-0-879-score/submission.csv'\nfilename5 = '/kaggle/input/bagging-rainforest/submission.csv'\n\ndf0 = pd.read_csv(filename0).sort_values('recording_id')\ndf1 = pd.read_csv(filename1).sort_values('recording_id')\ndf2 = pd.read_csv(filename2).sort_values('recording_id')\ndf3 = pd.read_csv(filename3).sort_values('recording_id')\ndf4 = pd.read_csv(filename4).sort_values('recording_id')\ndf5 = pd.read_csv(filename5).sort_values('recording_id')\n\nX0 = df0[features].values\nX1 = df1[features].values\nX2 = df2[features].values\nX3 = df3[features].values\nX4 = df4[features].values\nX5 = df5[features].values\n\nXs  = [X0, X1, X2, X3, X4, X5]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Because my two submissions are highly correlated, I assigned relatively small weights for my submissions."},{"metadata":{"trusted":true},"cell_type":"code","source":"W = [0.14, 0.14, 0.215, 0.17, 0.17, 0.165]\nprint(sum(W))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Raw prediction scores are transformed by ranking of prediction.\nAdding nonlinear term improved the public evaluation score."},{"metadata":{"trusted":true},"cell_type":"code","source":"POINT = 30 - np.arange(CLASS_N) + np.cumsum(np.cumsum(0.03 * np.ones((CLASS_N, ))))\nprint(POINT)\ndef score_to_score(x):\n    i = np.argsort(x)[::-1]\n    r = np.argsort(i)\n    return POINT[r]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"plt.figure()\nplt.plot(range(1, CLASS_N+1), POINT)\nplt.scatter(range(1, CLASS_N+1), POINT)\nplt.xlabel('rank')\nplt.ylabel('score')\nplt.xlim([0, CLASS_N+1])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"N = X0.shape[0]\npred_list = []\nfor row in range(N):\n    pred_list.append(np.sum(np.stack([w * score_to_score(X[row, :]) for X, w in zip(Xs, W)], axis=0), axis=0))\npred = np.stack(pred_list, axis=0)\n\ndata = {'recording_id' : df0['recording_id'].values }\nfor i in range(CLASS_N):\n    data[f's{i}'] = pred[:, i]\nsub = pd.DataFrame(data)\nsub.to_csv(\"submission.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}