{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"pixyz  \nlast update 2022 03 23  \nゆっくりしていってね！  ","metadata":{}},{"cell_type":"markdown","source":"version6: Items that costomer has bought are no longer included in the submission","metadata":{}},{"cell_type":"markdown","source":"**EDA書いたのでよかったら見てね。**\n\nhttps://www.kaggle.com/lunapandachan/h-m-eda-english","metadata":{}},{"cell_type":"markdown","source":"**datasetをつくるnotebookも観てね！**  \nhttps://www.kaggle.com/code/pixyz0130/eng-h-m-items-of-other-customers-buy","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://4.bp.blogspot.com/-6jLig_Zuhyk/UUhH8z560_I/AAAAAAAAO6A/lFCDFT8S1FM/s400/shopping_fasion.png\" width=200>","metadata":{}},{"cell_type":"markdown","source":"**霊夢:今日は前のnotebookで作ったdatasetでsubmissionを作っていくよ‼**\n\n**魔理沙:Byfone氏のcodeを参考にしたぜ。**\n\n<br>\n\n**Reimu: Today I'm going to make a submission with the dataset I made in the previous notebook!**\n\n**Marisa: I referred to Mr. Byfone's code.**\n\nhttps://www.kaggle.com/byfone/h-m-trending-products-weekly","metadata":{}},{"cell_type":"markdown","source":"<img src = \"https://4.bp.blogspot.com/-uoVuBWIbdiA/WvQHqpx_YCI/AAAAAAABL8g/NiFZ6K71VBc_0_dcKb3_4nhnvFJ_JMNuACLcBGAs/s450/network_dennou_sekai_figure.png\" width = 200>","metadata":{}},{"cell_type":"markdown","source":"**霊夢:データが多いのでtest=Trueの時は1000個のデータを使うよ。**  \n\n**Reimu: Since there is a lot of data, 1000 data will be used when test = True.**","metadata":{}},{"cell_type":"code","source":"test=False","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:57:24.928746Z","iopub.execute_input":"2022-03-26T13:57:24.9291Z","iopub.status.idle":"2022-03-26T13:57:24.953058Z","shell.execute_reply.started":"2022-03-26T13:57:24.928989Z","shell.execute_reply":"2022-03-26T13:57:24.952376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport json\nfrom math import sqrt\nfrom pathlib import Path\nfrom tqdm import tqdm\ntqdm.pandas()","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-26T13:57:24.96788Z","iopub.execute_input":"2022-03-26T13:57:24.96837Z","iopub.status.idle":"2022-03-26T13:57:24.974114Z","shell.execute_reply.started":"2022-03-26T13:57:24.968322Z","shell.execute_reply":"2022-03-26T13:57:24.973256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_path = Path('../input/h-and-m-personalized-fashion-recommendations/')\nN = 12\nM = 100","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:57:25.008235Z","iopub.execute_input":"2022-03-26T13:57:25.008515Z","iopub.status.idle":"2022-03-26T13:57:25.012724Z","shell.execute_reply.started":"2022-03-26T13:57:25.008488Z","shell.execute_reply":"2022-03-26T13:57:25.011973Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**魔理沙:N=12は予測アイテムの最大数だぜ。**\n\n**Marisa: N = 12 is the maximum number of predictable items.**","metadata":{}},{"cell_type":"markdown","source":"### Read the train data\ntrainデータの読み込み","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv(data_path / 'transactions_train.csv',\n                 usecols = ['t_dat', 'customer_id', 'article_id'],\n                 dtype={'article_id': str})\n\nif test :\n    df=df[:1000]","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:57:25.067821Z","iopub.execute_input":"2022-03-26T13:57:25.068382Z","iopub.status.idle":"2022-03-26T13:58:32.532423Z","shell.execute_reply.started":"2022-03-26T13:57:25.06834Z","shell.execute_reply":"2022-03-26T13:58:32.531815Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Take supposedly popular products\n\nおそらく人気のある製品を取る","metadata":{}},{"cell_type":"markdown","source":"**魔理沙:predictionが12個に満たなかった時や、trainデータに無いcustomer_idがあったときのために、総購入数が多い商品のランキングを作るよ。**\n\n**霊夢:実際にtraindataに無いcustomer_idが存在するとは思えないけどなあ...**\n\n<br>\n\n**Marisa: I'll make a ranking of the products with the highest total purchases in case the prediction is less than 12 or there is a customer_id that is not in the train data.**\n\n**Reimu: I don't think there is actually a customer_id that isn't in traindata ...**","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://1.bp.blogspot.com/-_s9y-vGJIvM/Xhwql3w5nLI/AAAAAAABXBs/J1_f2TIYxukb1VxSlzn1xSysPsGd4lf3ACNcBGAsYHQ/s1600/pose_syanikamaeru_woman.png\" width=200>","metadata":{}},{"cell_type":"code","source":"general_pred = df[\"article_id\"].astype(str).value_counts().sort_values(ascending=True)[-N:].index.to_list()","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:58:32.533609Z","iopub.execute_input":"2022-03-26T13:58:32.534278Z","iopub.status.idle":"2022-03-26T13:58:39.355737Z","shell.execute_reply.started":"2022-03-26T13:58:32.534242Z","shell.execute_reply":"2022-03-26T13:58:39.354862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Load json\njsonファイルを読み込む","metadata":{}},{"cell_type":"markdown","source":"**魔理沙:次は、前回つくったdatasetを読み込んでくよ！**\n\n**霊夢:「ある商品を買った人が他に買った商品ランキング」のデータだったよね。実際に使えるかなあ**\n\n＜br＞\n\n**Marisa: Next, let's load the dataset we created last time!**\n\n**Reimu: It was the data of \"the ranking of products that the person who bought one product bought another\". I wonder if it can actually be used**","metadata":{}},{"cell_type":"code","source":"with open(\"../input/hm-dictionary/items_of_other_costomers.json\", mode=\"r\") as f:\n        ds_dict = json.load(f)\nwith open(\"../input/hm-dictionary/dict_c_a.json\", mode=\"r\") as f:\n        c_a_dict = json.load(f)        ","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:58:39.356979Z","iopub.execute_input":"2022-03-26T13:58:39.357467Z","iopub.status.idle":"2022-03-26T13:58:59.644809Z","shell.execute_reply.started":"2022-03-26T13:58:39.357421Z","shell.execute_reply":"2022-03-26T13:58:59.643937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for x in ds_dict:\n#     ds_dict[x] = ds_dict[x][:12]","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:58:59.646504Z","iopub.execute_input":"2022-03-26T13:58:59.646722Z","iopub.status.idle":"2022-03-26T13:58:59.651055Z","shell.execute_reply.started":"2022-03-26T13:58:59.646689Z","shell.execute_reply":"2022-03-26T13:58:59.649839Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Fill in purchase dictionary\n\n購入辞書に記入する","metadata":{}},{"cell_type":"markdown","source":"**魔理沙:それじゃあ次は、それぞれの客が購入した商品から、次に買いそうな商品を予測していくよ**  \n\n**霊夢:どうやって作るんだ？**  \n\n**魔理沙:さっき読み込んだdatasetを使って、客が買ったそれぞれの商品について、その商品を買った時、他に買う商品ランキング1位はM点、2位はM-1点、3位はM-2点、4位はM-3点、という感じに配点をしていくぜ。その後、配点が高かった12個の商品をsubmissionにするぜ。これをすべての客について行うぜ。もしも、点数が1点以上の商品が12個よりも少なかったら、general_predから補充するぜ。**\n\n**Marisa: Then, from the products purchased by each customer, we will predict the products that are likely to be bought next**\n\n**Reimu: How do you make it?**\n\n**Marisa: For each product that the customer bought using the dataset that was just read, when the customer bought that product, the other product rankings they bought were M points, 2nd place was M-1 points, and 3rd place was M-2 points. I will give points to the point, 4th place is M-3 point. After that, I will submit the 12 products with the highest points. I'll do this for every guest. If there are less than 12 items with 1 or more points, I will replenish them from general_pred.**","metadata":{}},{"cell_type":"markdown","source":"<img src=\"https://2.bp.blogspot.com/-GuJM5cIi3K8/VHbNOnnoEzI/AAAAAAAApU8/VAa2CK1C360/s400/hyousyou_sports_man.png\" width=200>","metadata":{}},{"cell_type":"code","source":"sub = pd.read_csv(data_path / 'sample_submission.csv')\n\npred_list = []\nfor cust_id in tqdm(sub['customer_id']):\n    if cust_id in c_a_dict:\n        purchase_dict = {}\n        past_list = c_a_dict[cust_id]        \n        for art_id in past_list:\n            rank_list = ds_dict[str(art_id)]\n            for j in range(M):\n                item = str(rank_list[j]).zfill(10)\n                if item not in purchase_dict:\n                    purchase_dict[item] = M-j\n                else:\n                    purchase_dict[item] += M-j\n        for art_id in past_list:\n            if str(art_id).zfill(10) in purchase_dict:\n                purchase_dict[str(art_id).zfill(10)] = 0\n        series = pd.Series(purchase_dict)\n        series = series[series > 0]\n        \n#         print(len(series))\n        \n        sub_list = series.nlargest(N).index.tolist()\n    else:\n        sub_list = general_pred\n    pred_list.append(' '.join(sub_list))\nsub['prediction'] = pred_list\nsub.to_csv('submission.csv', index=None)","metadata":{"execution":{"iopub.status.busy":"2022-03-26T13:58:59.652576Z","iopub.execute_input":"2022-03-26T13:58:59.653079Z","iopub.status.idle":"2022-03-26T15:00:16.732882Z","shell.execute_reply.started":"2022-03-26T13:58:59.652943Z","shell.execute_reply":"2022-03-26T15:00:16.730399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Make a submission","metadata":{}},{"cell_type":"code","source":"sub.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-26T15:00:16.738796Z","iopub.execute_input":"2022-03-26T15:00:16.739269Z","iopub.status.idle":"2022-03-26T15:00:16.778762Z","shell.execute_reply.started":"2022-03-26T15:00:16.739233Z","shell.execute_reply":"2022-03-26T15:00:16.777789Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**霊夢:0706016001ってのがたくさんあるね。大丈夫かな？**\n\n**魔理沙:確か最も多く買われてた商品だったな。一番上に来てもおかしくはないぞ。ほかの商品は散っているみたいだし、悪くないんじゃないか？**\n\n<br>\n\n**Reimu: There are a lot of 076016001. Is it OK?**\n\n**Marisa: It was certainly the most bought item. It wouldn't be strange to come to the top. It seems that other products are scattered, isn't it bad?**","metadata":{}},{"cell_type":"markdown","source":"![](https://1.bp.blogspot.com/-PNcKwFw1PpM/U1T3oDIr9CI/AAAAAAAAfT4/gEn86X8Ppx0/s400/figure_goodjob.png)","metadata":{}}]}