{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"**AIM of the notebook**\n\nThe Aim of this notebook is to implement LightFM that's why I didn't include that much EDAs because there already are many great public notebooks! ","metadata":{}},{"cell_type":"markdown","source":"IF YOU FIND THIS NOTEBOOK USEFUL THEN PLEASE UPVOTE!","metadata":{}},{"cell_type":"markdown","source":"**Background**\n\nWe can divide recommendation models into two categories:\n\n     1)Content based model,\n \n     2)Collaborative filtering model.\n\nThe Content-Based Model recommends based on similarity of the items and/or users using their description/profile. On the other hand, Collaborative Filtering Model computes the latent factors of the users and items. It works based on the assumption that if a group of people expressed similar opinions on an item, these people would tend to have similar opinions on other items.","metadata":{}},{"cell_type":"markdown","source":"**LightFM**\n\nLightFM is a Python implementation of a hybrid recommendation algorithms for both implicit and explicit feedbacks.\n\nIt is a hybrid content-collaborative model which represents users and items as linear combinations of their content features’ latent factors. The model learns embeddings or latent representations of the users and items in such a way that it encodes user preferences over items. These representations produce scores for every item for a given user; items scored highly are more likely to be interesting to the user.\n\nThe user and item embeddings are estimated for every feature, and these features are then added together to be the final representations for users and items.\n\nFor example, for user i, the model retrieves the i-th row of the feature matrix to find the features with non-zero weights. The embeddings for these features will then be added together to become the user representation e.g. if user 10 has weight 1 in the 5th column of the user feature matrix, and weight 3 in the 20th column, the user 10’s representation is the sum of embedding for the 5th and the 20th features multiplying their corresponding weights. The representation for each items is computed in the same approach.","metadata":{}},{"cell_type":"markdown","source":"**Notebook Logs:-**\n\n**update 1:**\n\nAdded Submission Pipeline and added a trained model\n\nFor now the submission is taking 13 hours to run thats why I am not comitting the notebook until I found a solution to squeeze the runtime","metadata":{}},{"cell_type":"markdown","source":"**Importing Libraries**","metadata":{}},{"cell_type":"code","source":"import os\nimport cv2\nimport tqdm\nfrom PIL import Image\nimport seaborn as sns\nfrom matplotlib import pyplot as plt\n\nimport pandas as pd\nimport numpy as np\nfrom lightfm import LightFM\nfrom lightfm.data import Dataset\n\n# Import LightFM's evaluation metrics\nfrom lightfm.evaluation import precision_at_k\n\n%matplotlib inline\nSEED = 42\nnp.random.seed(SEED)\nos.environ['PYTHONHASHSEED'] = str(SEED)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:32:41.835705Z","iopub.execute_input":"2022-02-28T04:32:41.836252Z","iopub.status.idle":"2022-02-28T04:32:42.836759Z","shell.execute_reply.started":"2022-02-28T04:32:41.836128Z","shell.execute_reply":"2022-02-28T04:32:42.835704Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Configs**","metadata":{}},{"cell_type":"code","source":"# default number of recommendations\nK = 12\nEPOCHS = 1\n\n# model learning rate\nLEARNING_RATE = 0.25\n# no of latent factors\nNO_COMPONENTS = 20\n\n# no of threads to fit model\nNO_THREADS = 32\n# regularisation for both user and item features\nITEM_ALPHA=1e-6\nUSER_ALPHA=1e-6","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:32:43.623918Z","iopub.execute_input":"2022-02-28T04:32:43.624223Z","iopub.status.idle":"2022-02-28T04:32:43.628853Z","shell.execute_reply.started":"2022-02-28T04:32:43.624187Z","shell.execute_reply":"2022-02-28T04:32:43.627882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Load the data**","metadata":{}},{"cell_type":"code","source":"main_dir = \"../input/h-and-m-personalized-fashion-recommendations\"\nimages_dir = main_dir+\"/images/\" \ncustomers = pd.read_csv(main_dir+\"/customers.csv\")\narticles = pd.read_csv(main_dir+\"/articles.csv\", dtype={'article_id': str})\nsample_submission = pd.read_csv(main_dir+\"/sample_submission.csv\", dtype={'article_id': str})\n\ntrain = pd.read_csv('../input/h-and-m-personalized-fashion-recommendations/transactions_train.csv',  dtype={'article_id': str}, parse_dates=['t_dat'])","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:32:44.096424Z","iopub.execute_input":"2022-02-28T04:32:44.096716Z","iopub.status.idle":"2022-02-28T04:33:39.150584Z","shell.execute_reply.started":"2022-02-28T04:32:44.096681Z","shell.execute_reply":"2022-02-28T04:33:39.149013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**article_id** : A unique identifier of every article.\n\n**product_code, prod_name** : A unique identifier of every product and its name (not the same).\n\n**product_type, product_type_name** : The group of product_code and its name\n\n**graphical_appearance_no, graphical_appearance_name** : The group of graphics and its name\n\n**colour_group_code, colour_group_name** The group of color and its name\n\n**graphical_appearance_no, graphical_appearance_name** : The group of graphics and its name\n\n**perceived_colour_value_id, perceived_colour_value_name, perceived_colour_master_id, perceived_colour_master_name** The added color info\n\n**department_no, department_name**: A unique identifier of every dep and its name\n\n**index_code, index_name**: A unique identifier of every index and its name\n\n**index_group_no, index_group_name**: A group of indeces and its name\n\n**section_no, section_name**: A unique identifier of every section and its name\n\n**garment_group_no, garment_group_name**:  A unique identifier of every garment and its name","metadata":{}},{"cell_type":"markdown","source":"**Some Basic EDA**","metadata":{}},{"cell_type":"markdown","source":"Age Distribution","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.histplot(data=customers, x='age', bins=50, color='orange')\nax.set_xlabel('Distribution of the customers age')\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Price","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(10,5))\nax = sns.boxplot(data=train, x='price', color='orange')\nax.set_xlabel('Price')\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"  In product_group_name Lower/Upper/Full body have a huge price variance","metadata":{}},{"cell_type":"code","source":"sns.set_style(\"darkgrid\")\nf, ax = plt.subplots(figsize=(25,18))\nax = sns.boxplot(data=train[['customer_id', 'article_id', 'price', 't_dat']].merge(articles[['article_id', 'prod_name', 'product_type_name', 'product_group_name', 'index_name']], on='article_id', how='left'), x='price', y='product_group_name')\nax.set_xlabel('Price outliers', fontsize=22)\nax.set_ylabel('Index names', fontsize=22)\nax.xaxis.set_tick_params(labelsize=22)\nax.yaxis.set_tick_params(labelsize=22)\n\nplt.show()","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Some Images","metadata":{}},{"cell_type":"code","source":"total_folders = 0\ntotal_files = 0\nfolder_info = []\nimages_names = []\n\n\nfor base, dirs, files in tqdm.tqdm(os.walk(main_dir)):\n    for directories in dirs:\n        folder_info.append((directories, \n                            len(os.listdir(os.path.join(base, directories)))))\n        total_folders = total_folders + 1\n    \n    for _files in files:\n        total_files = total_files + 1\n        if (len(_files.split(\".jpg\"))==2):\n            images_names.append(_files.split(\".jpg\")[0])\n\nimage_name_df = pd.DataFrame(images_names, columns = [\"image_name\"])\nimage_name_df[\"article_id\"] = image_name_df[\"image_name\"].apply(lambda x: int(x[1:]))\nimage_name_df.head().style.set_properties(**{'background-color': 'rgba(184,230,194,.5)'})\n\narticles_df = pd.read_csv(\"../input/h-and-m-personalized-fashion-recommendations/articles.csv\")\nimage_article_df = articles_df[[\"article_id\", \n                                \"product_code\", \n                                \"product_group_name\", \n                                \"product_type_name\"]].merge(image_name_df, \n                                                            on=[\"article_id\"], \n                                                            how=\"left\")\nimage_article_df.head().style.set_properties(**{'background-color': 'rgba(184,230,194,.5)'})\n\ndef plot_image_samples(image_article_df, product_group_name, cols=1, rows=-1):\n    image_path = \"../input/h-and-m-personalized-fashion-recommendations/images/\"\n    _df = image_article_df.loc[image_article_df.product_group_name==product_group_name]\n    article_ids = _df.article_id.values[0:cols*rows]\n    plt.figure(figsize=(2 + 3 * cols, 2 + 4 * rows))\n    for i in range(cols * rows):\n        article_id = (\"0\" + str(article_ids[i]))[-10:]\n        plt.subplot(rows, cols, i + 1)\n        plt.axis('off')\n        plt.title(f\"{product_group_name} {article_id[:3]}\\n{article_id}.jpg\")\n        image = Image.open(f\"{image_path}{article_id[:3]}/{article_id}.jpg\")\n        plt.imshow(image)\n        \nplot_image_samples(image_article_df, \"Garment Lower body\", 5, 1)\nplot_image_samples(image_article_df, \"Accessories\", 5, 1)\nplot_image_samples(image_article_df, \"Swimwear\", 5, 1)\nplot_image_samples(image_article_df, \"Bags\", 5, 1)","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now let's get back to the main objective ","metadata":{}},{"cell_type":"code","source":"dataset = Dataset()\ndataset.fit(users=customers['customer_id'], \n            items=articles['article_id'])\n\nnum_users, num_topics = dataset.interactions_shape()\nprint(f'Number of users: {num_users}, Number of topics: {num_topics}.')","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:33:39.152489Z","iopub.execute_input":"2022-02-28T04:33:39.152908Z","iopub.status.idle":"2022-02-28T04:33:40.697635Z","shell.execute_reply.started":"2022-02-28T04:33:39.152864Z","shell.execute_reply":"2022-02-28T04:33:40.696596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Make Train and Validation Set**\n\nWe are taking the previous week before the test week to Validate our model.\n\nAs Chris Deotte said that a more robust validation with \"folds\" will be to do this with 5 validation periods. For \"fold 1\", use the last week of train and then train model with weeks prior. For \"fold 2\", use the second to last week of train and then train model with weeks prior. For \"fold 3, fold4, fold5\", etc etc. [link](https://www.kaggle.com/c/h-and-m-personalized-fashion-recommendations/discussion/307517#1697263)\n\nTrain TimeLine : start   ->   2020-09-15\n\nValidation TimeLine : 2020-09-16   ->   2020-09-22\n\nTest TimeLine  :  2020-09-23   ->  2020-09-29","metadata":{}},{"cell_type":"code","source":"#train_set = train[(train.t_dat>='2020-8-26')&(train.t_dat<='2020-9-15')]\ntrain_set = train[train.t_dat<='2020-9-15']\nval_set = train[(train.t_dat>='2020-9-16')&(train.t_dat<='2020-9-22')]\n\n(interactions, weights) = dataset.build_interactions(train_set.iloc[:, 1:3].values)\n(val_interactions, val_weights) = dataset.build_interactions(val_set.iloc[:, 1:3].values)\nprint(interactions.shape, val_interactions.shape)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:33:40.699356Z","iopub.execute_input":"2022-02-28T04:33:40.699717Z","iopub.status.idle":"2022-02-28T04:35:37.810864Z","shell.execute_reply.started":"2022-02-28T04:33:40.699673Z","shell.execute_reply":"2022-02-28T04:35:37.80962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"LightLM works slightly differently compared to other packages as it expects the train and test sets to have same dimension so double check it ","metadata":{}},{"cell_type":"markdown","source":"**Declare and Fit the LightFM model**\n\nIn this notebook, the LightFM model will be using the weighted Approximate-Rank Pairwise (WARP) as the loss.\n\nIn general, it maximises the rank of positive examples by repeatedly sampling negative examples until a rank violation has been located. This approach is recommended when only positive interactions are present","metadata":{}},{"cell_type":"code","source":"model = LightFM(loss='warp', no_components=NO_COMPONENTS, \n                 learning_rate=LEARNING_RATE,                 \n                 random_state=np.random.RandomState(SEED))\nmodel.fit(interactions=interactions, epochs=EPOCHS, verbose=1)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:35:37.813222Z","iopub.execute_input":"2022-02-28T04:35:37.813841Z","iopub.status.idle":"2022-02-28T04:37:02.424803Z","shell.execute_reply.started":"2022-02-28T04:35:37.813792Z","shell.execute_reply":"2022-02-28T04:37:02.423687Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Weighted Approximate-Rank Pairwise loss**\n\nWARP loss was first introduced in 2011. It was used to assign to an image the correct label from a very large sample of possible labels. Originally, the motivation for developing this loss — which in particular, has a novel sampling technique — was one of memory efficiency. However, the sampling technique also has additional benefits which make it well suited to training a recommender system.\n\n**How does WARP loss work?**\n\nAt a high level, WARP loss will randomly sample output labels of a model, until it finds a pair which it knows are wrongly labelled, and will then only apply an update to these two incorrectly labelled examples.\n","metadata":{}},{"cell_type":"markdown","source":"**Train and Validation scores**","metadata":{}},{"cell_type":"markdown","source":"Loading the trained model(trained for 200 epochs)","metadata":{}},{"cell_type":"code","source":"#Load Trained Model\n\n!pip3 install pickle5\nimport pickle5 as pickle\nwith open('../input/lightfm1/lightFM1.pickle', \"rb\") as fh:\n    trained_model = pickle.load(fh)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T05:03:51.588941Z","iopub.execute_input":"2022-02-28T05:03:51.589282Z","iopub.status.idle":"2022-02-28T05:04:07.345927Z","shell.execute_reply.started":"2022-02-28T05:03:51.589247Z","shell.execute_reply":"2022-02-28T05:04:07.344857Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\n#train_precision = precision_at_k(model, interactions, k=K).mean()(IT takes too much time thats why I am not running it)\nval_precision = precision_at_k(trained_model, val_interactions, k=K).mean()\n\nprint(val_precision)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**AS FOR NOW IT'S JUST A MODEL TRAINED FOR 5 EPOCHS BUT CAN BE HEAVILY IMPROVED AND IN NEXT UPDATES I WILL BE CREATING A FULL SUBMISSION PIPELINE!**","metadata":{}},{"cell_type":"markdown","source":"**Submission**","metadata":{}},{"cell_type":"code","source":"#Get the mappings\n'''\nuid = mapping from customer_id to model equivalent user_id\niid = mapping from article_id to  model equivalent article_id\n'''\nuid_map, ufeature_map, iid_map, ifeature_map = dataset.mapping() \n'''\ncreate inverse mappings\n'''\ninv_uid_map = {v:k for k, v in uid_map.items()}\ninv_iid_map = {v:k for k, v in iid_map.items()}\n\n#convert submission user_id and article_id to model equivalent user_id and article_id\n\ntest_X = sample_submission.customer_id.values\nlfn_user = lambda x: uid_map[x]\ntest_X_m = [lfn_user(tx) for tx in test_X]\n\nprint(len(test_X_m))","metadata":{"execution":{"iopub.status.busy":"2022-02-28T04:37:02.427164Z","iopub.execute_input":"2022-02-28T04:37:02.427669Z","iopub.status.idle":"2022-02-28T04:37:02.433689Z","shell.execute_reply.started":"2022-02-28T04:37:02.427604Z","shell.execute_reply":"2022-02-28T04:37:02.43263Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"customer_ids = []\npreds = []\n\nfor usr_ in tqdm.tqdm(test_X_m, total = len(test_X_m)):\n    m_opt = trained_model.predict(np.array([usr_] * len(iid_map)), np.array(list(iid_map.values())))\n    pred = np.argsort(-m_opt)[:K]\n    customer_ids.append(inv_uid_map[usr_])\n    preds.append(' '.join([inv_iid_map[p] for p in pred]).strip())\n    #break\n    \ncustomer_ids = np.array(customer_ids).reshape(-1, 1)\npreds = np.array(preds).reshape(-1, 1)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T05:05:02.054245Z","iopub.execute_input":"2022-02-28T05:05:02.054592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Create the submission\nfinal_sub = pd.DataFrame(data=np.concatenate((customer_ids, preds), axis=1).reshape(-1, 2), columns=['customer_id', 'prediction'])\nfinal_sub.to_csv('/kaggle/working/submission.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2022-02-28T05:00:38.516392Z","iopub.execute_input":"2022-02-28T05:00:38.516864Z","iopub.status.idle":"2022-02-28T05:00:38.524961Z","shell.execute_reply.started":"2022-02-28T05:00:38.516829Z","shell.execute_reply":"2022-02-28T05:00:38.524025Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**<h1>Work in progress 🚧</h1>**\n1) Full Submission Pipeline\n\n2) Optimized Model ","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}