{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Is there a difference between sales of products of the requested week and the prior week for last year?\nYesterday, I shared my finding of an anomaly in sales data for the last days of September. You can find my observation here: [!!! WARNING!!! Sales spike at end of September](https://www.kaggle.com/amanabdullayev/warning-sales-spike-at-end-of-september)\n\n## Brief summary of the previous observation\n- I observed an abnormally high amount of sales for the end of September of last years. Maybe, opening of winter sales? or discount on products for the summer/fall season? \n- Basically, we can say that the last days of September are vibrant! And h&M requested us to predict articles a customer will buy on the last days of September (7 days after 22-09-2020). Therefore, these findings are critical for model building.\n- When we analyze public notebooks with having a high score, we see that their logic is: \"find hot products for the period just before 22-09-2020 (I will call this period \"prior week\" hereafter) and make recommendations from these hot products\". However, as I said, we have an anomaly in sales data at the end of September (I will call this period \"interested week\" hereafter). Based on this, it is not a good idea simply to identify hot products of recent weeks and make recommendations out of them. We will understand the reason for this by analyzing data from last year for the same period.\n\n\n## Goal of this notebook\n- Show briefly/clearly that there are much more sales at the last days of September;\n- Find of top 12 hot products from the prior week of 2019, i.e. 15-09-2019 to 22-09-2019, and check if they are completely different than the interested week of 2019, i.e. 23-09-2019 to 30-09-2019.\n\nAcknowledgments:\nThanks for your [comments](https://www.kaggle.com/amanabdullayev/warning-sales-spike-at-end-of-september/comments) and pushing me to make more findings on this data:\n[ActulVerma](https://www.kaggle.com/atulverma).","metadata":{"jupyter":{"source_hidden":true}}},{"cell_type":"code","source":"# necessary libraries\nimport pandas as pd\nimport matplotlib.pyplot as plt\nfrom tqdm.auto import tqdm\nimport functools","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-03-05T17:20:19.375908Z","iopub.execute_input":"2022-03-05T17:20:19.376218Z","iopub.status.idle":"2022-03-05T17:20:19.389282Z","shell.execute_reply.started":"2022-03-05T17:20:19.376157Z","shell.execute_reply":"2022-03-05T17:20:19.387669Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# load data with only required columns to save computation power\ntransactions = pd.read_csv(\n    \"../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv\",\n    index_col=\"t_dat\",\n    usecols=[\"t_dat\", \"price\", \"article_id\"],\n    parse_dates=[\"t_dat\"],\n    infer_datetime_format=True, dtype={'article_id':str}\n)\ntransactions.head()","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:20:19.391209Z","iopub.execute_input":"2022-03-05T17:20:19.39174Z","iopub.status.idle":"2022-03-05T17:21:04.014067Z","shell.execute_reply.started":"2022-03-05T17:20:19.391688Z","shell.execute_reply":"2022-03-05T17:21:04.013208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# pandas resample() function is quite usefull to deal with timeseries data\n# we will get daily sum of price column and plot it for two year\ntransactions[\"price\"].resample(\"1D\").sum().plot(figsize=(20, 6), alpha=0.6)\nplt.title(\"Daily sales data for 2 years\", fontsize=18, color=\"r\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:21:04.016473Z","iopub.execute_input":"2022-03-05T17:21:04.01679Z","iopub.status.idle":"2022-03-05T17:21:05.134318Z","shell.execute_reply.started":"2022-03-05T17:21:04.016742Z","shell.execute_reply":"2022-03-05T17:21:05.133489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Here we see that actually, we have peaks in sales data time to time, which most likely to the start of sales or new seasonal produts. Since, we are interested for the period from 23-09-2020 to 30-09-2020, let's deep dive in that periods data.","metadata":{}},{"cell_type":"code","source":"# apply similar technique as above but for the data before 01-10-2018\nplt.figure(figsize=(14, 10))\nplt.subplot(311)\ntransactions[\"price\"].resample(\"1D\").sum().loc[:\"2018-09-30\"].plot(\n    alpha=0.8, ax=plt.gca()\n)\nplt.title(\"September 2018\", fontsize=16, color=\"r\")\nplt.ylim(500, 6500)\nplt.xlim(\"2018-09-15\", \"2018-09-30\")\nplt.grid(True, linestyle='--')\n\n\n# apply similar technique as above but for the data between 15-09-2019 to 01-10-2019\nplt.subplot(312)\ntransactions[\"price\"].resample(\"1D\").sum().loc[\"2019-09-15\":\"2019-09-30\"].plot(\n    alpha=0.8, ax=plt.gca()\n)\nplt.title(\"September 2019\", fontsize=16, color=\"r\")\nplt.ylim(500, 6500)\nplt.grid(True, linestyle='--')\n\n# apply similar technique as above but for the data after 15-06-2020\nplt.subplot(313)\ntransactions[\"price\"].resample(\"1D\").sum().loc[\"2020-09-15\":].plot(\n    alpha=0.8, ax=plt.gca()\n)\nplt.title(\"September 2020\", fontsize=16, color=\"r\")\nplt.ylim(500, 6500)\nplt.xlim(\"2020-09-15\", \"2020-09-30\")\nplt.text(\n    \"2020-09-23\",\n    1000,\n    \"We have to predict sales this vibrant period\",\n    color=\"blue\",\n    fontsize=14,\n)\nplt.grid(True, linestyle='--')\n\nplt.tight_layout()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:21:05.135637Z","iopub.execute_input":"2022-03-05T17:21:05.135843Z","iopub.status.idle":"2022-03-05T17:21:07.258288Z","shell.execute_reply.started":"2022-03-05T17:21:05.135819Z","shell.execute_reply":"2022-03-05T17:21:07.257403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### As you can see from the above graphs, sharp peaks are there for the end of September. And, surprisingly, H&M chose that particular vibrant period for us to predict the products. I have the following hypothesis here: top-selling products that are sold before this vibrant period would not be much relevant for the requested period. To see if this is true, let's analyze the same period of last year, i.e. 2019.","metadata":{}},{"cell_type":"code","source":"# take top-selling products for the same periods of previous year\nhot_12_prior_2019 = transactions.loc[\"2019-09-15\":\"2019-09-22\"]['article_id'].value_counts()[0:12].index.tolist()\nhot_12_interested_2019 = transactions.loc[\"2019-09-23\":\"2019-09-30\"]['article_id'].value_counts()[0:12].index.tolist()\n\n\n# quickly compare to list if they are same\nif functools.reduce(lambda x, y : x and y, map(lambda p, q: p == q,hot_12_prior_2019,hot_12_interested_2019), True): \n    print (\"Hot products of prior and interested weeks are the same.\") \nelse: \n    print (\"Hot products are not the same!\")","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:21:07.259505Z","iopub.execute_input":"2022-03-05T17:21:07.259716Z","iopub.status.idle":"2022-03-05T17:21:07.645227Z","shell.execute_reply.started":"2022-03-05T17:21:07.259693Z","shell.execute_reply":"2022-03-05T17:21:07.644376Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nfig = plt.figure(figsize=(24,18))\ncount = 1\nfor i in hot_12_prior_2019:\n    root_path = '../input/h-and-m-personalized-fashion-recommendations/images/'\n    img_path = root_path + i[:3] + '/' + i + '.jpg'\n    img = plt.imread(img_path)\n    fig.add_subplot(3, 4, count)\n    plt.imshow(img)\n    plt.title(f'Article {i}')\n\t# remove axes and place the images closer to one another for a more compact output\n    plt.xticks([])\n    plt.yticks([])\n    plt.suptitle('Top-selling products of the prior week',  y=1.01,fontsize=20, color='b')\n    plt.tight_layout()\n    count += 1","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:21:07.646521Z","iopub.execute_input":"2022-03-05T17:21:07.647014Z","iopub.status.idle":"2022-03-05T17:21:13.096827Z","shell.execute_reply.started":"2022-03-05T17:21:07.646968Z","shell.execute_reply":"2022-03-05T17:21:13.096264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = plt.figure(figsize=(24,18))\ncount = 1\nfor i in hot_12_interested_2019:\n    root_path = '../input/h-and-m-personalized-fashion-recommendations/images/'\n    img_path = root_path + i[:3] + '/' + i + '.jpg'\n    img = plt.imread(img_path)\n    fig.add_subplot(3, 4, count)\n    plt.imshow(img)\n    plt.title(f'Article {i}')\n\t# remove axes and place the images closer to one another for a more compact output\n    plt.xticks([])\n    plt.yticks([])\n    plt.suptitle('Top-selling products of the interested week',  y=1.01,fontsize=20, color='b')\n    plt.tight_layout()\n    count += 1","metadata":{"execution":{"iopub.status.busy":"2022-03-05T17:21:13.097902Z","iopub.execute_input":"2022-03-05T17:21:13.098377Z","iopub.status.idle":"2022-03-05T17:21:18.788674Z","shell.execute_reply.started":"2022-03-05T17:21:13.098343Z","shell.execute_reply":"2022-03-05T17:21:18.788128Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Obviosuly, while there are more thinner and bright colored clothes in the prior week, more darker and warm clothes are in the interested week. Even though this data is for 2019, I am pretty sure that the similar trend will exist in the data for 2020.","metadata":{}},{"cell_type":"markdown","source":"Stay safe and healthy! If you like my findings, please upvote it and leave your comments/improvement suggestions below!","metadata":{}},{"cell_type":"markdown","source":"### ","metadata":{}}]}