{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Proyecto Final: Modelos avanzados de ciencia de datos\n## H&M Personalized Product Recommendations","metadata":{}},{"cell_type":"markdown","source":"**Motivación de la elección del problema**\n\nLa construcción de un modelo de predicción basado en el historial de compra de productos (en este caso, productos de las tiendas en línea del Grupo H&M) tiene un enfoque particularmente interesante al tener como resultado recomendaciones personalizadas ya que implantar la necesidad o el deseo de un producto a un cliente, resulta sumamente conveniente y de cierta manera controlador, una idea codiciada y de gran valor para todas las empresas o personas que ofrecen sus productos con el único objetivo de maximizar las ventas.\n\nEn este mismo sentido, es de igual manera motivante el lograr adelantarse al deseo o querer de alguien, predecir el gusto del cliente y ofrecer un producto antes que el cliente mismo sepa que quiere ese producto.\n\nPor último, el proyecto contiene distintos tipos de información para analizar: datos de clientes, artículos y transacciones de clientes (datos estructurados); así como textos e imágenes (datos no estructurados), lo que lo hace también atrayente para el equipo, por la diversidad de técnicas de análisis y procesamiento que se estarán aplicando.","metadata":{}},{"cell_type":"markdown","source":"**Descripción general del problema**\n\n\nGrupo H&M es una familia de marcas y negocios con 53 mercados en línea y aproximadamente 4850 tiendas. Sus tiendas en línea ofrecen a los compradores una amplia selección de productos. Pero con demasiadas opciones, es posible que los clientes no encuentren rápidamente lo que les interesa o lo que están buscando y, al final, ellos podrían no realizar una compra. Para mejorar la experiencia de compra, las recomendaciones de productos son clave. Más importante aún, ayudar a los clientes a tomar las decisiones correctas también tiene implicaciones positivas para la sostenibilidad, ya que reduce las devoluciones y, por lo tanto, minimiza las emisiones del transporte.\n \nGrupo H&M requiere desarrollar recomendaciones de productos basadas en datos de transacciones previas, así como en metadatos de clientes y productos. \n \nEl proyecto se evaluará con el estadístico Mean Average Precision @ 12 (MAP@12):\n\n\\begin{equation*}\nMAP@12=\\frac{1}{U}\\sum_{u=1}^{U}\\frac{1}{min(m,12)}\\sum_{k=1}^{min(n,12)} P(k)*rel(k)\n\\end{equation*}\n\nU = Número de clientes  \nP(k) = Precisión del corte k  \nn = Número de predicciones por cliente  \nm = Número de elementos por cliente  \nrel(k) = Función indicadora, 1 si el artículo es correcto y 0 si no  \n\n\n> Bloque con sangría\n\n\nDescripción del set de datos\n \nArchivos:\n \n* images/ - Carpeta con imágenes (.jpg) que corresponden a cada artículo. Nota: no todos los artículos tienen una imagen.\n* articles.csv – Detalle de cada artículo disponible para compra.\n* customers.csv - Datos del cliente\n* sample_submission.csv – Ejemplo del archivo de envío (formato requerido)\n* transaction_train.csv - Datos de las compras de cada cliente.\n\n","metadata":{}},{"cell_type":"code","source":"#Librerías básicas\nimport pandas as pd\nimport numpy as np\nimport sklearn\n\n#Librerías para graficar\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport plotly.express as px\nimport plotly.graph_objects as go\n\n#Para ignorar warnings\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n\n#Para realizar los clúster\nfrom sklearn.cluster import KMeans\nfrom sklearn.preprocessing import MinMaxScaler\n\n#Realizar Market Basket Analysis\nfrom mlxtend.preprocessing import TransactionEncoder\nfrom mlxtend.frequent_patterns import apriori, association_rules\n\n#Liberías para realizar ML (Se utilizan 4 modelos)\nfrom sklearn.ensemble import GradientBoostingClassifier\nfrom sklearn.tree import DecisionTreeClassifier\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.linear_model import LogisticRegression\nimport xgboost as xgb\nfrom sklearn.metrics.pairwise import cosine_similarity\n\n#Para partir población para test y train // Para comprobar que tan bueno es el modelo\nfrom sklearn.model_selection import KFold, cross_val_score, train_test_split, GridSearchCV, cross_validate\nfrom sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score, confusion_matrix, classification_report\n\n#Para fechas\nimport time \n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:23.787297Z","iopub.execute_input":"2023-05-22T05:47:23.787691Z","iopub.status.idle":"2023-05-22T05:47:27.415498Z","shell.execute_reply.started":"2023-05-22T05:47:23.787653Z","shell.execute_reply":"2023-05-22T05:47:27.414161Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Preprocesamiento y limpieza de los datos","metadata":{}},{"cell_type":"markdown","source":"### Tabla de artículos","metadata":{}},{"cell_type":"code","source":"articulos=pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/articles.csv')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:27.417674Z","iopub.execute_input":"2023-05-22T05:47:27.418069Z","iopub.status.idle":"2023-05-22T05:47:28.844795Z","shell.execute_reply.started":"2023-05-22T05:47:27.418035Z","shell.execute_reply":"2023-05-22T05:47:28.842739Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articulos.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:28.846811Z","iopub.execute_input":"2023-05-22T05:47:28.847410Z","iopub.status.idle":"2023-05-22T05:47:28.906849Z","shell.execute_reply.started":"2023-05-22T05:47:28.847356Z","shell.execute_reply":"2023-05-22T05:47:28.904827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Revisión de las columnas que contiene la tabla, tipo de datos y descriptivo de las variables flotantes.","metadata":{}},{"cell_type":"code","source":"articulos.columns.values","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:28.911576Z","iopub.execute_input":"2023-05-22T05:47:28.912465Z","iopub.status.idle":"2023-05-22T05:47:28.922199Z","shell.execute_reply.started":"2023-05-22T05:47:28.912423Z","shell.execute_reply":"2023-05-22T05:47:28.920966Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articulos.info()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:28.924390Z","iopub.execute_input":"2023-05-22T05:47:28.925212Z","iopub.status.idle":"2023-05-22T05:47:29.450698Z","shell.execute_reply.started":"2023-05-22T05:47:28.925175Z","shell.execute_reply":"2023-05-22T05:47:29.449537Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articulos.describe()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:29.452173Z","iopub.execute_input":"2023-05-22T05:47:29.452546Z","iopub.status.idle":"2023-05-22T05:47:29.604757Z","shell.execute_reply.started":"2023-05-22T05:47:29.452514Z","shell.execute_reply":"2023-05-22T05:47:29.603899Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"La tabla de información de artículos contiene 105,542 datos, con 11 datos de tipo entero y 14 de tipo caráctes.\n\nEl descriptivo  no es de mucha utilidad debido a que los códigos son relacionados al artículo, es una llave para saber que producto es.","metadata":{}},{"cell_type":"markdown","source":"Para revisar la presencia de nulos se recurre a la siguiente función.","metadata":{}},{"cell_type":"code","source":"articulos.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:29.606378Z","iopub.execute_input":"2023-05-22T05:47:29.607535Z","iopub.status.idle":"2023-05-22T05:47:30.098547Z","shell.execute_reply.started":"2023-05-22T05:47:29.607491Z","shell.execute_reply":"2023-05-22T05:47:30.097207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"La única variable que contiene nulos es la variable detail_des, la variable contiene información descriptiva del producto y debido a que ya tenemos desglosada la información relacionada al producto para el análisis se estará omitiendo la variable detail_desc.\n\nAún así, se revisa que contiene la variable para corroborar su estructura.","metadata":{}},{"cell_type":"code","source":"def producto(x):\n    print(\"El producto es:\\n\",articulos.prod_name[x])\n    print(\"\\nTipo de producto: \\n\",articulos.product_type_name[x])\n    print(\"\\nDetalle del producto:\\n\",articulos.detail_desc[x])","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:30.101754Z","iopub.execute_input":"2023-05-22T05:47:30.102303Z","iopub.status.idle":"2023-05-22T05:47:30.110597Z","shell.execute_reply.started":"2023-05-22T05:47:30.102267Z","shell.execute_reply":"2023-05-22T05:47:30.108535Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"producto(13)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:30.112764Z","iopub.execute_input":"2023-05-22T05:47:30.114989Z","iopub.status.idle":"2023-05-22T05:47:30.135451Z","shell.execute_reply.started":"2023-05-22T05:47:30.114873Z","shell.execute_reply":"2023-05-22T05:47:30.133421Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articulos_1=articulos.drop(['detail_desc'],axis=1)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:30.142134Z","iopub.execute_input":"2023-05-22T05:47:30.143520Z","iopub.status.idle":"2023-05-22T05:47:30.179634Z","shell.execute_reply.started":"2023-05-22T05:47:30.143462Z","shell.execute_reply":"2023-05-22T05:47:30.178097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Tabla de transacciones","metadata":{}},{"cell_type":"code","source":"transactions=pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/transactions_train.csv')\ntransactions_df = transactions.copy()\n#transactions=transactions.sample(frac = 0.7)\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:47:30.181256Z","iopub.execute_input":"2023-05-22T05:47:30.181637Z","iopub.status.idle":"2023-05-22T05:48:45.885437Z","shell.execute_reply.started":"2023-05-22T05:47:30.181603Z","shell.execute_reply":"2023-05-22T05:48:45.883823Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se convierten las variables a formato datatime y se extrae el mes de la fecha de compra (t_dat).","metadata":{}},{"cell_type":"code","source":"#Transformzar a datetime\ntransactions['t_dat']=pd.to_datetime(transactions['t_dat'])\n#Nombre del mes\ntransactions['mes']=transactions['t_dat'].dt.month","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:48:45.888055Z","iopub.execute_input":"2023-05-22T05:48:45.888874Z","iopub.status.idle":"2023-05-22T05:48:56.060546Z","shell.execute_reply.started":"2023-05-22T05:48:45.888836Z","shell.execute_reply":"2023-05-22T05:48:56.058879Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Tabla de clientes","metadata":{}},{"cell_type":"code","source":"# Lectura de los archivos\n\n# Clientes\ncustomers_df = pd.read_csv('/kaggle/input/h-and-m-personalized-fashion-recommendations/customers.csv')\ntotal_rows = customers_df.shape[0]\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:48:56.062788Z","iopub.execute_input":"2023-05-22T05:48:56.063250Z","iopub.status.idle":"2023-05-22T05:49:02.768190Z","shell.execute_reply.started":"2023-05-22T05:48:56.063202Z","shell.execute_reply":"2023-05-22T05:49:02.767190Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Conteo de valores únicos\nprint(\"Conteo valores únicos FN: \\n\", customers_df[\"FN\"].value_counts(dropna = False))\nprint(\"\\n\")\nprint(\"Conteo valores únicos Active: \\n\", customers_df[\"Active\"].value_counts(dropna = False))\nprint(\"\\n\")\nprint(\"Conteo valores únicos Club member status: \\n\", customers_df[\"club_member_status\"].value_counts(dropna = False))\nprint(\"\\n\")\nprint(\"Conteo valores únicos Fashion news frequency: \\n\", customers_df[\"fashion_news_frequency\"].value_counts(dropna = False))\nprint(\"\\n\")\nprint(\"Conteo valores únicos Postal code: \\n\", customers_df[\"postal_code\"].value_counts(dropna = False))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:02.769772Z","iopub.execute_input":"2023-05-22T05:49:02.770456Z","iopub.status.idle":"2023-05-22T05:49:04.022899Z","shell.execute_reply.started":"2023-05-22T05:49:02.770420Z","shell.execute_reply":"2023-05-22T05:49:04.021343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se genera función para eliminar valores nulos","metadata":{}},{"cell_type":"code","source":"# Análisis de Missing values\ndef get_missing_values(column_name):\n    missing_val = customers_df[column_name].isnull().sum()\n    percent = round((missing_val * 100)/total_rows, 2)\n    good_val = round(100 - percent, 2)\n    return percent, good_val","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.024920Z","iopub.execute_input":"2023-05-22T05:49:04.025435Z","iopub.status.idle":"2023-05-22T05:49:04.033811Z","shell.execute_reply.started":"2023-05-22T05:49:04.025390Z","shell.execute_reply":"2023-05-22T05:49:04.032648Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Columna FN\nmissing_fn, good_fn = get_missing_values(\"FN\")\nprint(\"Missing values en FN: \" + str(missing_fn) + \"%\")\nprint(\"Valores válidos en FN: \" + str(good_fn) + \"%\")\nprint(\"\\n\")\n\n# Columna Active\nmissing_active, good_active = get_missing_values(\"Active\")\nprint(\"Missing values en Active: \" + str(missing_active) + \"%\")\nprint(\"Valores válidos en Active: \" + str(good_active) + \"%\")\nprint(\"\\n\")\n\n# Columna Club member status\nmissing_club, good_club = get_missing_values(\"club_member_status\")\nprint(\"Missing values en Club member status: \" + str(missing_club) + \"%\")\nprint(\"Valores válidos en Club member status: \" + str(good_club) + \"%\")\nprint(\"\\n\")\n\n# Columna Fashion news frequency\nmissing_news, good_news = get_missing_values(\"fashion_news_frequency\")\nprint(\"Missing values en Fashion news frequency: \" + str(missing_news) + \"%\")\nprint(\"Valores válidos en Fashion news frequency: \" + str(good_news) + \"%\")\nprint(\"\\n\")\n\n# Age\nmissing_age, good_age = get_missing_values(\"age\")\nprint(\"Missing values en Age: \" + str(missing_age) + \"%\")\nprint(\"Valores válidos en Age: \" + str(good_age) + \"%\")","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.036012Z","iopub.execute_input":"2023-05-22T05:49:04.036554Z","iopub.status.idle":"2023-05-22T05:49:04.357797Z","shell.execute_reply.started":"2023-05-22T05:49:04.036510Z","shell.execute_reply":"2023-05-22T05:49:04.356577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Tabla de contingencia FN y Active\npd.crosstab(index=customers_df['FN'], columns=customers_df['Active'], margins=True)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.359420Z","iopub.execute_input":"2023-05-22T05:49:04.359777Z","iopub.status.idle":"2023-05-22T05:49:04.645834Z","shell.execute_reply.started":"2023-05-22T05:49:04.359744Z","shell.execute_reply":"2023-05-22T05:49:04.644644Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Eliminación de columnas no requeridas para el análisis de recomendaciones\ncustomers_df = customers_df.drop(['FN', 'fashion_news_frequency', 'club_member_status'], axis=1)\ncustomers_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.647464Z","iopub.execute_input":"2023-05-22T05:49:04.647806Z","iopub.status.idle":"2023-05-22T05:49:04.777411Z","shell.execute_reply.started":"2023-05-22T05:49:04.647776Z","shell.execute_reply":"2023-05-22T05:49:04.775919Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Se eliminan clientes sin edad\ncust_with_age_df = customers_df[customers_df['age'].notna()]","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.779458Z","iopub.execute_input":"2023-05-22T05:49:04.779855Z","iopub.status.idle":"2023-05-22T05:49:04.869389Z","shell.execute_reply.started":"2023-05-22T05:49:04.779825Z","shell.execute_reply":"2023-05-22T05:49:04.867697Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Análisis exploratorio de los datos","metadata":{}},{"cell_type":"markdown","source":"### Clientes por edad","metadata":{}},{"cell_type":"code","source":"# Histogram de edad de clientes\nplt.figure(figsize=(12,6))\nplt.title(\"Customers Age\")\nsns.histplot(data=customers_df['age'])","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:04.870631Z","iopub.execute_input":"2023-05-22T05:49:04.871213Z","iopub.status.idle":"2023-05-22T05:49:07.686721Z","shell.execute_reply.started":"2023-05-22T05:49:04.871134Z","shell.execute_reply":"2023-05-22T05:49:07.682672Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Análisis outliers de Edad\nsns.boxplot(customers_df['age'])\n\n# Cálculo IQR\n# IQR\nQ1 = customers_df['age'].quantile(0.25)\nQ3 = customers_df['age'].quantile(0.75)\nIQR = Q3 - Q1\n\nprint(\"Valor Q1:\", Q1)\nprint(\"Valor Q3:\", Q3)\nprint(\"Valor calculado IQR:\", IQR)\n\n# Cálculo de límites inferior y superior\nupper_limit = Q3 + 1.5 * IQR\nlower_limit = Q1 - 1.5 * IQR\n\nprint(\"Valor límite inferior:\", lower_limit)\nprint(\"Valor límite superior:\", upper_limit)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:07.688930Z","iopub.execute_input":"2023-05-22T05:49:07.689410Z","iopub.status.idle":"2023-05-22T05:49:08.026008Z","shell.execute_reply.started":"2023-05-22T05:49:07.689374Z","shell.execute_reply":"2023-05-22T05:49:08.024763Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"La mayor parte de clientes se encuentran entre 50 y 25 años.","metadata":{}},{"cell_type":"code","source":"# Conteo de clientes outliers\nage_outliers_df = customers_df[customers_df['age'] > upper_limit]\nprint(\"Total de clientes outliers\", len(age_outliers_df))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:08.027316Z","iopub.execute_input":"2023-05-22T05:49:08.027649Z","iopub.status.idle":"2023-05-22T05:49:08.036284Z","shell.execute_reply.started":"2023-05-22T05:49:08.027619Z","shell.execute_reply":"2023-05-22T05:49:08.035147Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Conteo de transacciones de clientes outliers\ntrans_age_outliers = transactions.merge(age_outliers_df, on=['customer_id','customer_id'])\nprint(\"Total de transacciones de clientes outliers\", len(trans_age_outliers))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:08.037647Z","iopub.execute_input":"2023-05-22T05:49:08.038083Z","iopub.status.idle":"2023-05-22T05:49:34.964674Z","shell.execute_reply.started":"2023-05-22T05:49:08.038052Z","shell.execute_reply":"2023-05-22T05:49:34.962777Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Conteo de clientes con Nan en Edad\nage_nan = customers_df[customers_df['age'].isna()]\nprint(\"Total de clientes con Edad Nan\", len(age_nan))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:34.966608Z","iopub.execute_input":"2023-05-22T05:49:34.967094Z","iopub.status.idle":"2023-05-22T05:49:34.985271Z","shell.execute_reply.started":"2023-05-22T05:49:34.967050Z","shell.execute_reply":"2023-05-22T05:49:34.983580Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Conteo de transacciones de clientes con Edad Nan\ntrans_age_nan = transactions.merge(age_nan, on=['customer_id','customer_id'])\nprint(\"Total de transacciones de clientes con Edad Nan\", len(trans_age_nan))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:49:34.989136Z","iopub.execute_input":"2023-05-22T05:49:34.989579Z","iopub.status.idle":"2023-05-22T05:50:01.173381Z","shell.execute_reply.started":"2023-05-22T05:49:34.989546Z","shell.execute_reply":"2023-05-22T05:50:01.171783Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Tabla de artículos","metadata":{}},{"cell_type":"markdown","source":"Se recurre a una función definida para obtener la frecuencia de alguna de las variables de la tabla de artículos.","metadata":{}},{"cell_type":"code","source":"\ndef frecuencia(grupo,leyenda):\n    #Revisar en que productos se venden más productos\n    productos= articulos_1.groupby(grupo)['article_id'].count()\n    productos=pd.DataFrame(productos).sort_values('article_id', ascending=False)\n    productos[grupo] = productos.index\n\n    #Para graficar\n    grafica=productos.head(20)\n\n    fig = go.Figure()\n\n    fig.add_trace(\n        go.Bar(\n                y=grafica['article_id'], x=grafica[grupo]\n                ,text=grafica['article_id']\n                ,textposition='outside'\n                ,textfont=dict(color='black')\n                ,textfont_size=12\n                ,textfont_color='black'\n                ,yaxis='y1'\n    ))\n\n    fig.update_layout(xaxis=dict(domain=[0, 1]),\n        #Crear eje primario          \n        yaxis=dict(title=leyenda),\n\n                     )\n\n    fig.update_layout(title_text='<b>Frecuencia de los artículos comprados</b>', title_x=0.5\n                 #,xaxis_title=\"Año de reporte de la información\"\n                 ,showlegend=False #Quitar la legenda del grafico\n                 ,font=dict(family=\"Montserrat (Títulos)\",size=15,color=\"black\")\n                 ,template = 'simple_white' ) #Cambiar el fondo\n\n    #Para modificar la posición: textangle=90\n    fig.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:01.175792Z","iopub.execute_input":"2023-05-22T05:50:01.176346Z","iopub.status.idle":"2023-05-22T05:50:01.190707Z","shell.execute_reply.started":"2023-05-22T05:50:01.176302Z","shell.execute_reply":"2023-05-22T05:50:01.189500Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"frecuencia('prod_name','Nombre del producto')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:01.192601Z","iopub.execute_input":"2023-05-22T05:50:01.193581Z","iopub.status.idle":"2023-05-22T05:50:03.115601Z","shell.execute_reply.started":"2023-05-22T05:50:01.193543Z","shell.execute_reply":"2023-05-22T05:50:03.114367Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"En la gráfica anterior se observa que el producto Dragonfly se encuentra 98 veces en la tabla, se revisa cuál es la diferencia den producto con nombre Dragonfly.","metadata":{}},{"cell_type":"code","source":"articulos_1[articulos_1['prod_name'] == 'Dragonfly dress']","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.127356Z","iopub.execute_input":"2023-05-22T05:50:03.128218Z","iopub.status.idle":"2023-05-22T05:50:03.187621Z","shell.execute_reply.started":"2023-05-22T05:50:03.128172Z","shell.execute_reply":"2023-05-22T05:50:03.186176Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Como se observa en la tabla, aunque el nombre del producto es el mismo el id del artículo es diferente debido a que el color del producto varia.","metadata":{}},{"cell_type":"code","source":"frecuencia('colour_group_name','Nombre del color del grupo')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.189238Z","iopub.execute_input":"2023-05-22T05:50:03.190257Z","iopub.status.idle":"2023-05-22T05:50:03.261288Z","shell.execute_reply.started":"2023-05-22T05:50:03.190212Z","shell.execute_reply":"2023-05-22T05:50:03.260074Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Con la gráfica anterior podemos observar que las prendas de color negro o que predomina el negro, son las que el público más compra.","metadata":{}},{"cell_type":"code","source":"frecuencia('department_name','Nombre del departamento del artículo')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.263043Z","iopub.execute_input":"2023-05-22T05:50:03.263744Z","iopub.status.idle":"2023-05-22T05:50:03.330041Z","shell.execute_reply.started":"2023-05-22T05:50:03.263705Z","shell.execute_reply":"2023-05-22T05:50:03.328904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"El departamento que tiene más productos es el de Jersey.","metadata":{}},{"cell_type":"code","source":"def univariado(variable,leyenda):\n\n    plt.hist(articulos_1[variable], rwidth=0.6)\n    plt.title('Histograma de los productos comprados')\n    plt.xlabel(leyenda)\n    plt.ylabel('Frecuencia')\n\n    plt.show()\n\n    print(\"Descripción de la variable: \\n\",articulos_1[variable].describe())\n    \n    missing_values_count = articulos_1[variable].isnull().sum()\n\n    total_cells = np.product(articulos_1[variable].shape)\n    total_missing = missing_values_count.sum() \n    \n    print(\"\\nMissing en la variable: \\n\",articulos_1[variable].isnull().sum())\n    print(\"\\nPorcentaje de missing: \\n\",(total_missing/total_cells)*100)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.331758Z","iopub.execute_input":"2023-05-22T05:50:03.332667Z","iopub.status.idle":"2023-05-22T05:50:03.344009Z","shell.execute_reply.started":"2023-05-22T05:50:03.332624Z","shell.execute_reply":"2023-05-22T05:50:03.342601Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"univariado('product_code','Nombre del producto')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.345692Z","iopub.execute_input":"2023-05-22T05:50:03.346068Z","iopub.status.idle":"2023-05-22T05:50:03.659159Z","shell.execute_reply.started":"2023-05-22T05:50:03.346038Z","shell.execute_reply":"2023-05-22T05:50:03.657985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"La variable product_code es una variable númerica y únicamente es una llave relacionada a la variable prod_name, no contiene missing y su llave indica que los existen más productos con llaves más grandes, en la grafica se ve la información más cargada a la derecha.","metadata":{}},{"cell_type":"markdown","source":"### Correlaciones","metadata":{}},{"cell_type":"markdown","source":"Se revisa la correlación y las variables articulo_id y product_code se encuentran muy correlacionas.\n\nEl articulo_id se estará utilizando para unir la tabla de articulos con la de transacciones, ya que la variable articulo_id funge como llave.","metadata":{}},{"cell_type":"code","source":"df_sincategoricas=articulos_1.select_dtypes(exclude=['object'])\ndf_sincategoricas.info()\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.660707Z","iopub.execute_input":"2023-05-22T05:50:03.661132Z","iopub.status.idle":"2023-05-22T05:50:03.681619Z","shell.execute_reply.started":"2023-05-22T05:50:03.661098Z","shell.execute_reply":"2023-05-22T05:50:03.680350Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.set(rc = {'figure.figsize':(30,20)})\nsns.set_style(\"white\")\nsns.pairplot(df_sincategoricas,vars = [\"article_id\",\"product_code\", \"product_type_no\",\"colour_group_code\"])\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:03.683390Z","iopub.execute_input":"2023-05-22T05:50:03.684022Z","iopub.status.idle":"2023-05-22T05:50:16.391111Z","shell.execute_reply.started":"2023-05-22T05:50:03.683962Z","shell.execute_reply":"2023-05-22T05:50:16.389303Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Para las transacciones, es octubre 2019 el periodo con mayor número de transacciones.","metadata":{}},{"cell_type":"code","source":"def elementos_grafica():\n    plt.rcParams['font.family'] = 'sans-serif'\n    plt.figure(figsize=(10,5))\n    plt.xticks(rotation=90, size=8)\n    plt.grid(False)\n    #Eliminar los bordes\n    plt.gca().spines['top'].set_visible(False)\n    plt.gca().spines['right'].set_visible(False)\n    #Eje y con separadores de miles\n    plt.gca().get_yaxis().set_major_formatter(plt.FuncFormatter(lambda x, loc: \"{:,}\".format(int(x))))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:16.393230Z","iopub.execute_input":"2023-05-22T05:50:16.393837Z","iopub.status.idle":"2023-05-22T05:50:16.402570Z","shell.execute_reply.started":"2023-05-22T05:50:16.393797Z","shell.execute_reply":"2023-05-22T05:50:16.401292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"elementos_grafica()\n#gráfica de  línea de transacciones por día\ntransactions.groupby('t_dat')['t_dat'].count().plot(color='#a1d99b')\n#Média móvil de 7 días\ntransactions.groupby('t_dat')['t_dat'].count().rolling(7).mean().plot()\nplt.gcf().set_facecolor(\"none\")\nplt.legend(['Transacciones','Media móvil 7 días'], frameon=False)\nplt.title('Transacciones por día', size=14, weight='bold')\n#Título ejes\nplt.xlabel('Fecha')\nplt.ylabel('Transacciones')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:16.403909Z","iopub.execute_input":"2023-05-22T05:50:16.405144Z","iopub.status.idle":"2023-05-22T05:50:18.234679Z","shell.execute_reply.started":"2023-05-22T05:50:16.405083Z","shell.execute_reply":"2023-05-22T05:50:18.233322Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"raw","source":"Junio es el mes que presenta mayor número de compras, mientras que febrero y diciembre son los meese con un menor número de compras.","metadata":{}},{"cell_type":"code","source":"#Gráfica de transacciones por mes para ver estacionalidad\nelementos_grafica()\ntransactions.groupby(transactions['mes'])['mes'].count().plot(kind='bar')\nplt.gcf().set_facecolor(\"none\")\nplt.title('Transacciones por mes', size=14, weight='bold')\nplt.xlabel('Mes')\nplt.ylabel('Transacciones')\n#Incorporar nombre de los meses\nplt.xticks([0,1,2,3,4,5,6,7,8,9,10,11],['Ene','Feb','Mar','Abr','May','Jun','Jul','Ago','Sep','Oct','Nov','Dic'], rotation=0)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:18.236527Z","iopub.execute_input":"2023-05-22T05:50:18.236951Z","iopub.status.idle":"2023-05-22T05:50:19.282094Z","shell.execute_reply.started":"2023-05-22T05:50:18.236882Z","shell.execute_reply":"2023-05-22T05:50:19.280964Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Las transaciones se realizan principalmente por el canal 2 ().","metadata":{}},{"cell_type":"code","source":"#Transacciones por channel_id\nelementos_grafica()\ntransactions.groupby('sales_channel_id')['sales_channel_id'].count().plot(kind='bar')\nplt.gcf().set_facecolor(\"none\")\nplt.title('Transacciones por canal de ventas', size=14, weight='bold')\nplt.xlabel('Canal de ventas')\nplt.ylabel('Transacciones')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:19.283684Z","iopub.execute_input":"2023-05-22T05:50:19.284536Z","iopub.status.idle":"2023-05-22T05:50:20.140695Z","shell.execute_reply.started":"2023-05-22T05:50:19.284500Z","shell.execute_reply":"2023-05-22T05:50:20.139638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Transacciones por precio\nelementos_grafica()\ntransactions.groupby('price')['price'].count().plot(color='#a1d99b',label=\"Transacciones\")\n#Precio medio y mediano\nplt.axvline(transactions['price'].mean(), color='red', linestyle='--', label='Precio promedio')\nplt.axvline(transactions['price'].median(), color='green', linestyle='--', label='Precio mediano')\nplt.gcf().set_facecolor(\"none\")\nplt.legend(frameon=False)\n#Título y ejes\nplt.title('Transacciones por precio', size=14, weight='bold')\nplt.xlabel('Precio')\nplt.ylabel('Transacciones')\n#Anotaciones\nplt.annotate('Precio medio: ${:.2f}'.format(transactions['price'].mean()), xy=(transactions['price'].mean(), 0.7), xytext=(transactions['price'].mean()+.2,3000000)\n             )\nplt.annotate('Precio mediano: ${:.2f}'.format(transactions['price'].median()), xy=(transactions['price'].mean(), 0.7), xytext=(transactions['price'].mean()+.2,2900000)\n             )\nplt.annotate('Precio máximo: ${:.2f}'.format(transactions['price'].max()), xy=(transactions['price'].mean(), 0.7), xytext=(transactions['price'].mean()+.2,2800000)\n             )\nplt.annotate('Precio mínimo: ${:.2f}'.format(transactions['price'].min()), xy=(transactions['price'].mean(), 0.7), xytext=(transactions['price'].mean()+.2,2700000)\n             )","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:20.142041Z","iopub.execute_input":"2023-05-22T05:50:20.144293Z","iopub.status.idle":"2023-05-22T05:50:23.952011Z","shell.execute_reply.started":"2023-05-22T05:50:20.144254Z","shell.execute_reply":"2023-05-22T05:50:23.950837Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Tabla con 20 productos más vendidos\ntop_products=transactions.groupby('article_id')['article_id'].count().sort_values(ascending=False).head(20)\nbottom_products=transactions.groupby('article_id')['article_id'].count().sort_values(ascending=False).tail(20)\nprint('Los 20 productos más vendidos son:')\nprint(top_products)\nprint(\"\\n\")\nprint('Los 20 productos menos vendidos son:')\nprint(bottom_products)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:23.953518Z","iopub.execute_input":"2023-05-22T05:50:23.953936Z","iopub.status.idle":"2023-05-22T05:50:25.959141Z","shell.execute_reply.started":"2023-05-22T05:50:23.953881Z","shell.execute_reply":"2023-05-22T05:50:25.957962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Top 20 de customers\ntop_customers=transactions.groupby('customer_id')['customer_id'].count().sort_values(ascending=False).head(20)\nbottom_customers=transactions.groupby('customer_id')['customer_id'].count().sort_values(ascending=False).tail(20)\nprint('Los 20 clientes que más compran son:')\nprint(top_customers)\nprint(\"\\n\")\nprint('Los 20 clientes que menos compran son:')\nprint(bottom_customers)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:50:25.961092Z","iopub.execute_input":"2023-05-22T05:50:25.961613Z","iopub.status.idle":"2023-05-22T05:51:01.283934Z","shell.execute_reply.started":"2023-05-22T05:50:25.961576Z","shell.execute_reply":"2023-05-22T05:51:01.282206Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Selección y entrenamiento de los modelos","metadata":{}},{"cell_type":"markdown","source":"Para el presente trabajo se utilizarán dos modelos de recomendación:\n\n1. Xgboost (Extreme Gradient Boosting), el cual es de carácter no lineal y se basa en árboles de decisión. Este modelo es de tipo colaborativo, ya que utiliza la información de los usuarios y de los artículos para generar las recomendaciones. Es un modelo de tipo supervisado.\n2. Modelo de datos tipo no supervisado. El modelo tiene como base el filtrado colaborativo basado en artículos. Se construyó una representación vectorial multidimensional de los artículos a través de una matriz de co-ocurrencia y se buscó la similaridad entre ellos medida por la similitud del coseno. Los datos utilizados fueron las transacciones de los clientes; específicamente, se trabajó en analizar por transacción, qué productos se compran con otros productos.","metadata":{}},{"cell_type":"markdown","source":"Unir las tablas en un dataframe concentrado para realizar los modelos","metadata":{}},{"cell_type":"code","source":"#transacciones=transacciones.sample(frac = fracc)\n\n#Importar la tabla transacciones\ntransactions=(transactions.merge(articulos,on=['article_id'], how='left')\n        .merge(customers_df,on=['customer_id'], how='left'))\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:51:01.285815Z","iopub.execute_input":"2023-05-22T05:51:01.286769Z","iopub.status.idle":"2023-05-22T05:52:20.822211Z","shell.execute_reply.started":"2023-05-22T05:51:01.286728Z","shell.execute_reply":"2023-05-22T05:52:20.820876Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ya que tenemos una tabla que concentra las 3 tablas de los datos necesarios para el modelos, se da una revisión a la existencia de valores nulos.","metadata":{}},{"cell_type":"code","source":"#Revisión de valores nulos\nprint('Valores nulos existentes: ',(transactions['age'].isnull().sum()/(transactions['age'].count()+transactions['age'].isnull().sum()))*100)\n","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:52:20.825496Z","iopub.execute_input":"2023-05-22T05:52:20.826113Z","iopub.status.idle":"2023-05-22T05:52:21.009569Z","shell.execute_reply.started":"2023-05-22T05:52:20.826070Z","shell.execute_reply":"2023-05-22T05:52:21.008392Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Los valores nulos existentes en la tabla de datos es del 0.44%, por lo que serán eliminados para que no causen ruido en el modelo.","metadata":{}},{"cell_type":"code","source":"#Eliminar valores nulos\ntransactions=transactions.dropna()\n\n#Valores nulos despues de NA\nprint('Valores nulos existentes después de aplicar drop NA: ',(transactions['age'].isnull().sum()/(transactions['age'].count()+transactions['age'].isnull().sum()))*100)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:52:21.011168Z","iopub.execute_input":"2023-05-22T05:52:21.015739Z","iopub.status.idle":"2023-05-22T05:55:37.446883Z","shell.execute_reply.started":"2023-05-22T05:52:21.015699Z","shell.execute_reply":"2023-05-22T05:55:37.445699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Frecuencia de edad\nplt.hist(x=transactions['age'], color='#F2AB6D', rwidth=0.85)\nplt.gcf().set_facecolor(\"none\")\nplt.title('Histograma de edades de los clientes')\nplt.xlabel('Edades')\nplt.ylabel('Frecuencia')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:55:37.448463Z","iopub.execute_input":"2023-05-22T05:55:37.449250Z","iopub.status.idle":"2023-05-22T05:55:38.365419Z","shell.execute_reply.started":"2023-05-22T05:55:37.449206Z","shell.execute_reply":"2023-05-22T05:55:38.363969Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Como se observa en la grafica, la mayoría de las comrpas la realizan las personas de 30 años, a H&M recurren más personas de una edad de 30 hacia abajo.","metadata":{}},{"cell_type":"markdown","source":"Para realizar los clúster, dejamos únicamente las variables numericas.","metadata":{}},{"cell_type":"code","source":"#Extraer únicamente las variables númericas\ndf_numericas=transactions._get_numeric_data()\ndf_numericas.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:55:38.366936Z","iopub.execute_input":"2023-05-22T05:55:38.367297Z","iopub.status.idle":"2023-05-22T05:55:38.388157Z","shell.execute_reply.started":"2023-05-22T05:55:38.367266Z","shell.execute_reply":"2023-05-22T05:55:38.386862Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se obtiene el número de clúster obtimo mediante una elbow.","metadata":{}},{"cell_type":"code","source":"#Revisión del codo para saber el número de clúster ideal\nkmeans_kwargs = {\"init\": \"random\",\"n_init\": 10,\"random_state\": 1,}\n\nsse = []\nfor k in range(1, 11):\n    kmeans = KMeans(n_clusters=k, **kmeans_kwargs)\n    kmeans.fit(df_numericas)\n    sse.append(kmeans.inertia_)\n\nplt.plot(range(1, 11), sse)\nplt.xticks(range(1, 11))\nplt.xlabel(\"Clúster\")\nplt.title('Elbow Curve')\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T05:55:38.389335Z","iopub.execute_input":"2023-05-22T05:55:38.389661Z","iopub.status.idle":"2023-05-22T06:20:42.110833Z","shell.execute_reply.started":"2023-05-22T05:55:38.389634Z","shell.execute_reply":"2023-05-22T06:20:42.109448Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Preparación de datos\ndf_numericas['edad']=df_numericas['age']\n\nescalar = MinMaxScaler()\nescalar.fit(transactions[['age']])\ndf_numericas['age'] = escalar.transform(df_numericas[['age']])\n\n#Encontrar los clúster\nkm = KMeans(n_clusters=6)\ny_predicted = km.fit_predict(df_numericas[['age','price']])\n\n#Añadir clúster a la tabla de transacciones\ntransactions['cluster']=y_predicted\ntransactions.head()\n\n#Revisar la distribución de los clúser\ntransactions.groupby(by='cluster').count()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:20:42.112859Z","iopub.execute_input":"2023-05-22T06:20:42.113942Z","iopub.status.idle":"2023-05-22T06:22:39.542696Z","shell.execute_reply.started":"2023-05-22T06:20:42.113878Z","shell.execute_reply":"2023-05-22T06:22:39.541425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize=(20,8))\n\nplt.subplot(2,3,1)\nplt.hist(x=transactions[transactions['cluster'] == 0]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 0')\n\nplt.subplot(2,3,2)\nplt.hist(x=transactions[transactions['cluster'] == 1]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 1')\n\nplt.subplot(2,3,3)\nplt.hist(x=transactions[transactions['cluster'] == 2]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 2')\n\nplt.subplot(2,3,4)\nplt.hist(x=transactions[transactions['cluster'] == 3]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 3')\n\nplt.subplot(2,3,5)\nplt.hist(x=transactions[transactions['cluster'] == 4]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 4')\n\nplt.subplot(2,3,6)\nplt.hist(x=transactions[transactions['cluster'] == 5]['age'], color='#00688B', rwidth=0.85)\nplt.title('Clúster 5')","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:39.544216Z","iopub.execute_input":"2023-05-22T06:22:39.544569Z","iopub.status.idle":"2023-05-22T06:22:50.624183Z","shell.execute_reply.started":"2023-05-22T06:22:39.544538Z","shell.execute_reply":"2023-05-22T06:22:50.622369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def poblacion(df,n,prodctos):\n    cluster=df[df['cluster'] == n]\n    cluster.t_dat = pd.to_datetime(cluster.t_dat)\n\n    #Grafica del comportamiento de compras del clúster\n    cluster[['t_dat','article_id']].groupby('t_dat').count().plot(figsize=(12,6), legend=False, fontsize=10)\n    plt.title('Productos comprados por día', fontsize=18)\n    plt.xlabel('Fecha',fontsize=14)\n\n    #Crear meses y días\n    cluster['mes']=pd.to_datetime(cluster['t_dat']).dt.month_name()\n    cluster['dia']=pd.to_datetime(cluster['t_dat']).dt.day\n\n    lista_meses = ['January','February','March','April','May','June','July','August','September','October','November','December']\n    meses = pd.api.types.CategoricalDtype(categories=lista_meses)\n    cluster['mes']= cluster['mes'].astype(meses)\n\n\n    fig2 = px.density_heatmap(cluster, x=cluster['mes'].sort_values(), y=\"dia\", template=\"seaborn\")\n    fig2.show()\n    \n    color = plt.cm.rainbow(np.linspace(0, 1, prodctos))\n    cluster['prod_name'].value_counts().head(prodctos).plot.bar(color = color, figsize=(13,5))\n    plt.title('Items más comprados', fontsize = 20)\n    plt.xticks(rotation = 90 )\n    plt.grid()\n    plt.show()\n    \n    return cluster","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.626015Z","iopub.execute_input":"2023-05-22T06:22:50.626380Z","iopub.status.idle":"2023-05-22T06:22:50.640764Z","shell.execute_reply.started":"2023-05-22T06:22:50.626351Z","shell.execute_reply":"2023-05-22T06:22:50.638945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se define una función para obtener el comportamiento de compra del clúster solicitado.\n\nY otra para reducir dimensiones podenmos quedarnos con los productos más vendidos o elegir que frecuencia de productos se desean seleccionar.","metadata":{}},{"cell_type":"code","source":"def mba(df):\n\n    #Realizo una base para el mba\n    base=df.groupby(['customer_id']).agg({'prod_name': lambda x: x.ravel().tolist()}).reset_index()\n\n    #Crear una llave nueva\n    base=base.reset_index()\n    base = base.rename(columns={'index':'customer_id','customer_id':'cliente_id'})\n    base=base.drop(['cliente_id'], axis=1)\n    \n    basket=list(base['prod_name'])\n\n    encoder = TransactionEncoder()\n    transactions = encoder.fit(basket).transform(basket)\n    transactions = pd.DataFrame(transactions, columns=encoder.columns_)\n\n    frequent_itemsets = apriori(transactions, min_support= 6/len(basket), use_colnames=True, max_len = 2)\n    rules = association_rules(frequent_itemsets, metric=\"lift\",  min_threshold = 1.5)\n    \n    print(\"Reglas identificadas: \", len(rules))\n\n    display(rules.sort_values(by=['confidence'],ascending = [False]))\n    \n    return","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.643011Z","iopub.execute_input":"2023-05-22T06:22:50.643666Z","iopub.status.idle":"2023-05-22T06:22:50.667877Z","shell.execute_reply.started":"2023-05-22T06:22:50.643614Z","shell.execute_reply":"2023-05-22T06:22:50.666904Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def red_base(df,cant_prod):\n\n    #Agrupar el cluster por producto\n    producto=df[['prod_name','customer_id']].groupby('prod_name').count()\n    producto=pd.DataFrame(producto).reset_index()\n    #producto.sort_values('customer_id',ascending=False)\n\n    #Seleccionar los productos con mayor frecuancia de compra\n    producto=producto[producto['customer_id']>=cant_prod]\n    producto\n\n    #Unir transacciones con clientes\n    tabla=(df.merge(producto,on=['prod_name'], how='left'))\n\n    #Renombrar variables\n    tabla=tabla.dropna()\n    tabla=tabla.rename(columns={'customer_id_x':'customer_id',\n                                           'customer_id_y':'productos_comprados'})\n    #Volvemos a revisar los productos\n    color = plt.cm.rainbow(np.linspace(0, 1, 40))\n    tabla['prod_name'].value_counts().head(40).plot.bar(color = color, figsize=(13,5))\n    plt.title('Frecuencia de los items más populares', fontsize = 20)\n    plt.xticks(rotation = 90 )\n    plt.grid()\n    plt.show()\n    \n    return tabla","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.669565Z","iopub.execute_input":"2023-05-22T06:22:50.670743Z","iopub.status.idle":"2023-05-22T06:22:50.693020Z","shell.execute_reply.started":"2023-05-22T06:22:50.670697Z","shell.execute_reply":"2023-05-22T06:22:50.691909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se definen funciones para obtener el market basket analysis y el rfm.","metadata":{}},{"cell_type":"code","source":"def rfm(tabla):\n    \n    tabla=tabla.loc[:, ['t_dat', 'customer_id', 'prod_name','price']]\n\n    #Se parte la población\n    primer_periodo = tabla[(tabla.t_dat < pd.Timestamp(2020,7,20)) & \n                          (tabla.t_dat >= pd.Timestamp(2018,9,19))].reset_index(drop=True)\n\n    segundo_periodo = tabla[(tabla.t_dat < pd.Timestamp(2020,9,20)) \n                               & (tabla.t_dat >= pd.Timestamp(2020,7,20))].reset_index(drop=True)\n\n    #Obtener un dataframe\n    clientes_unicos = pd.DataFrame(primer_periodo['customer_id'].unique())\n\n    #Renombramos la columna\n    clientes_unicos.columns = ['customer_id']\n\n\n    #Fecha de la compra más reciente\n    Primer_compra_segundo_periodo = segundo_periodo.groupby('customer_id').t_dat.min().reset_index()\n    Primer_compra_segundo_periodo.columns = ['customer_id', 'Ultima_compra']\n    Primer_compra_segundo_periodo.head()\n\n\n    #Fecha de la compra más antigua\n    Ultima_compra_primer_periodo= primer_periodo.groupby('customer_id').t_dat.max().reset_index()\n    Ultima_compra_primer_periodo.columns = ['customer_id', 'Primer_compra']\n    Ultima_compra_primer_periodo.head()\n\n\n    # Unir tablas de compras\n    fecha_compras = pd.merge(Ultima_compra_primer_periodo, Primer_compra_segundo_periodo, on='customer_id', how='left')\n\n    # Diferencias entre la última y primer compra de cada cliente\n    fecha_compras['Proxima_fecha_compra'] = (fecha_compras['Ultima_compra'] - fecha_compras['Primer_compra']).dt.days\n\n    clientes_unicos = pd.merge(clientes_unicos, fecha_compras[['customer_id', 'Proxima_fecha_compra']], on='customer_id', how='left')\n\n    # Convetir NA a 9999\n    clientes_unicos = clientes_unicos.fillna(999)\n    clientes_unicos.head()\n\n    #-----------------------------------------------------------------------\n    #Obtener Recencia, Frecuencia y Montos\n\n    #Recencia\n\n    # La compra más antigua del cliente\n    compra_antigua = primer_periodo.groupby('customer_id').t_dat.max().reset_index()\n    compra_antigua.columns = ['customer_id','Primer_compra']\n\n    # Revisar la distancia entre compras\n    compra_antigua['Recencia'] = (compra_antigua['Primer_compra'].max() - compra_antigua['Primer_compra']).dt.days\n\n    # Unir columnas a clientes únicos\n    clientes_unicos = pd.merge(clientes_unicos, compra_antigua[['customer_id', 'Recencia']], on='customer_id')\n\n    kmeans = KMeans(n_clusters=4)\n    kmeans.fit(clientes_unicos[['Recencia']])\n    clientes_unicos['Recencia_Cluster'] = kmeans.predict(clientes_unicos[['Recencia']])\n    #clientes_unicos.groupby('Recencia_Cluster')['Recencia'].describe()\n\n    #-----------------------------------------------------------------------\n    #Frecuencia\n\n    # Frecuencia de compras por cliente\n    frecuencia_compras = primer_periodo.groupby('customer_id').t_dat.count().reset_index()\n    frecuencia_compras.columns = ['customer_id', 'Frecuencia']\n\n    # Unir tablas\n    clientes_unicos = pd.merge(clientes_unicos, frecuencia_compras, on='customer_id')\n\n    kmeans = KMeans(n_clusters=4)\n    kmeans.fit(clientes_unicos[['Frecuencia']])\n    clientes_unicos['Frecuencia_Cluster'] = kmeans.predict(clientes_unicos[['Frecuencia']])\n\n    #-----------------------------------------------------------------------\n    #Monto de la compra\n\n    # Create a new label, Revenue of each item bought\n    primer_periodo['Monto'] = primer_periodo.price \n\n    # Get the revenue from each customer and sum them.\n    monto_compras = primer_periodo.groupby('customer_id').Monto.sum().reset_index()\n\n    # Merge the dataframe ctm_revenue with ctm_dt\n    clientes_unicos = pd.merge(clientes_unicos, monto_compras, on='customer_id')\n    clientes_unicos.head()\n\n    kmeans = KMeans(n_clusters=4)\n    kmeans.fit(clientes_unicos[['Monto']])\n    clientes_unicos['Monto_Cluster'] = kmeans.predict(clientes_unicos[['Monto']])\n\n\n    clientes_unicos['Puntaje'] = clientes_unicos['Recencia_Cluster'] + clientes_unicos['Frecuencia_Cluster'] + clientes_unicos['Monto_Cluster']\n    clientes_unicos.groupby('Puntaje')['Recencia', 'Frecuencia', 'Monto'].mean()\n    \n    return clientes_unicos, primer_periodo","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.694669Z","iopub.execute_input":"2023-05-22T06:22:50.695319Z","iopub.status.idle":"2023-05-22T06:22:50.715534Z","shell.execute_reply.started":"2023-05-22T06:22:50.695287Z","shell.execute_reply":"2023-05-22T06:22:50.714577Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def puntajes(df,rango1,Rango2):\n    df['Categoría'] = 'Bajo'\n    df.loc[df['Puntaje'] > rango1, 'Categoría'] = 'Medio'\n    df.loc[df['Puntaje'] > Rango2, 'Categoría'] = 'Alto'\n    df.head()\n    \n    return df","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.716970Z","iopub.execute_input":"2023-05-22T06:22:50.718248Z","iopub.status.idle":"2023-05-22T06:22:50.737384Z","shell.execute_reply.started":"2023-05-22T06:22:50.718205Z","shell.execute_reply":"2023-05-22T06:22:50.736196Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Se crean nuevas variables a partir del rfm para un modelo que indica si el cliente compra en los siguientes n días.","metadata":{}},{"cell_type":"code","source":"def creaion_var_fecha(clientes_unicos,primer_periodo,rango1,rango2):\n    \n    #Obtenemos un dattaframe con las columnas fecha y clientes_id\n    dias_compras_primer_periodo = primer_periodo[['customer_id','t_dat']]\n    dias_compras_primer_periodo = dias_compras_primer_periodo.sort_values(['customer_id','t_dat'])\n\n    #Eliminar duplicados\n    dias_compras_primer_periodo = dias_compras_primer_periodo.drop_duplicates(subset=['customer_id','t_dat'],keep='first')\n\n    #Modificar las últimas 3 compras\n    dias_compras_primer_periodo['fecha_compra_1'] = dias_compras_primer_periodo.groupby('customer_id')['t_dat'].shift(1)\n    dias_compras_primer_periodo['fecha_compra_2'] = dias_compras_primer_periodo.groupby('customer_id')['t_dat'].shift(2)\n    dias_compras_primer_periodo['fecha_compra_3'] = dias_compras_primer_periodo.groupby('customer_id')['t_dat'].shift(3)\n\n    #Obtener la distancia entre la fecha de compra y las últimas 3 comrpas\n    dias_compras_primer_periodo['DD_fecha_compra_1'] = (dias_compras_primer_periodo['t_dat'] - dias_compras_primer_periodo['fecha_compra_1']).dt.days\n    dias_compras_primer_periodo['DD_fecha_compra_2'] = (dias_compras_primer_periodo['t_dat'] - dias_compras_primer_periodo['fecha_compra_1']).dt.days\n    dias_compras_primer_periodo['DD_fecha_compra_3'] = (dias_compras_primer_periodo['t_dat'] - dias_compras_primer_periodo['fecha_compra_1']).dt.days\n\n    #Agrupar\n    dias_compras_diferencias = dias_compras_primer_periodo.groupby('customer_id').agg({'DD_fecha_compra_1': ['mean','std']}).reset_index()\n    dias_compras_diferencias.columns = ['customer_id', 'Diferencia_dias_Mean','Diferencia_dias_Std']\n\n    #Eliminar duplicados\n    dias_ultima_compra = dias_compras_primer_periodo.drop_duplicates(subset=['customer_id'],keep='last')\n    dias_ultima_compra = dias_ultima_compra.dropna()\n\n    dias_ultima_compra = pd.merge(dias_ultima_compra, dias_compras_diferencias, on='customer_id')\n    clientes_unicos = pd.merge(clientes_unicos, dias_ultima_compra[['customer_id','DD_fecha_compra_1','DD_fecha_compra_2','DD_fecha_compra_3','Diferencia_dias_Mean','Diferencia_dias_Std']], on='customer_id')\n\n    #Crear nueva llave con el índice\n    clientes_unicos=clientes_unicos.reset_index()\n    clientes_unicos_index=clientes_unicos.drop(['customer_id'], axis=1)\n    clientes_unicos_index = clientes_unicos_index.rename(columns={'index':'customer_id'})\n\n    # Crear una copia\n    c_unicos = clientes_unicos_index.copy()\n\n    # Se convierte la data a numerica\n    c_unicos = pd.get_dummies(c_unicos)\n\n    #Rangosde cuando el cleinte va a realziar la próxima compra\n    c_unicos['Rango_proxima_compra'] = 2\n    c_unicos.loc[c_unicos.Proxima_fecha_compra>rango1,'Rango_proxima_compra'] = 1\n    c_unicos.loc[c_unicos.Proxima_fecha_compra>rango2,'Rango_proxima_compra'] = 0\n\n    print(c_unicos.Rango_proxima_compra.value_counts()/len(clientes_unicos))\n\n    # Matiz de correlación\n    corr_matrix = c_unicos[c_unicos.columns].corr()\n\n    # Mínimos coeficeintes de correlación\n    corr_df = pd.DataFrame(corr_matrix.min())\n    corr_df.columns = ['MinCorrelationCoeff']\n\n    # Calcular el máximo coeficiente de correlación\n    corr_df['MaxCorrelationCoeff'] = corr_matrix[corr_matrix < 1].max()\n\n    plt.figure(figsize = (40, 30))\n    sns.heatmap(corr_matrix, annot = True, linewidths=0.2, fmt=\".2f\")\n    \n    return c_unicos","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.738702Z","iopub.execute_input":"2023-05-22T06:22:50.740358Z","iopub.status.idle":"2023-05-22T06:22:50.758144Z","shell.execute_reply.started":"2023-05-22T06:22:50.740322Z","shell.execute_reply":"2023-05-22T06:22:50.756584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"La siguiente función compara modelo para quedarnos con el mejor.","metadata":{}},{"cell_type":"code","source":"def modelos(c_unicos):\n    c_unicos = c_unicos.drop('Proxima_fecha_compra', axis=1)\n\n    X, y = c_unicos.drop('Rango_proxima_compra',axis=1), c_unicos.Rango_proxima_compra\n    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=44)\n\n    models = []\n    models.append((\"RF\",RandomForestClassifier()))\n    models.append((\"Dtree\",DecisionTreeClassifier()))\n    models.append((\"XGB\",xgb.XGBClassifier()))\n\n    for name,model in models:\n        kfold = KFold(n_splits=2)\n        cv_result = cross_val_score(model,X_train,y_train, cv = kfold, scoring = \"accuracy\")\n        print(name, cv_result)\n        \n    return X_train, X_test, y_train, y_test","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.760318Z","iopub.execute_input":"2023-05-22T06:22:50.761018Z","iopub.status.idle":"2023-05-22T06:22:50.785409Z","shell.execute_reply.started":"2023-05-22T06:22:50.760980Z","shell.execute_reply":"2023-05-22T06:22:50.784460Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### 2. Modelo de datos no supervisado","metadata":{}},{"cell_type":"code","source":"# Considera realizar las recomendaciones a partir de los artículos más vendidos\nconteo = transactions_df['article_id'].value_counts().rename('conteo').reset_index()\nmasvendidos=conteo['conteo']>=7000\nr=conteo[masvendidos]\nr.rename(columns = {'index':'article_id'}, inplace = True)\n#r","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:50.786619Z","iopub.execute_input":"2023-05-22T06:22:50.787146Z","iopub.status.idle":"2023-05-22T06:22:52.912860Z","shell.execute_reply.started":"2023-05-22T06:22:50.787113Z","shell.execute_reply":"2023-05-22T06:22:52.911593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Obtiene las transacciones con los artículos más vendidos\ntrans_masvend = pd.merge(transactions_df, r)\ntrans_masvend","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:52.914590Z","iopub.execute_input":"2023-05-22T06:22:52.915617Z","iopub.status.idle":"2023-05-22T06:22:56.572382Z","shell.execute_reply.started":"2023-05-22T06:22:52.915565Z","shell.execute_reply":"2023-05-22T06:22:56.570785Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Agrega la columna tipo de producto, para que las recomendaciones de los \n# artículos se realicen con base en el mismo tipo de producto \nsample_df = trans_masvend.merge(articulos[[\"product_type_name\", \"article_id\"]], on=['article_id','article_id'])\nsample_df = sample_df.drop(['conteo'], axis=1)\n\n# Genera un agrupador para la transacción, para identificar qué artículos \n# compra el cliente dentro de la misma transacción u orden\nsample_df['order_id'] = sample_df.t_dat.str.cat(sample_df.customer_id)\n#sample_df","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:56.574219Z","iopub.execute_input":"2023-05-22T06:22:56.574698Z","iopub.status.idle":"2023-05-22T06:22:59.537523Z","shell.execute_reply.started":"2023-05-22T06:22:56.574656Z","shell.execute_reply":"2023-05-22T06:22:59.536105Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Genera tabla pivote, cada renglón es una orden y cada columna es un artículo;\n# se realiza el conteo de los artículos en cada orden\npivot_df = pd.pivot_table(sample_df,index = 'order_id',columns = 'article_id',values = 'product_type_name',aggfunc = 'count')\npivot_df.reset_index(inplace=True)\npivot_df = pivot_df.fillna(0)\npivot_df = pivot_df.drop('order_id', axis=1)\n# pivot_df","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:22:59.539431Z","iopub.execute_input":"2023-05-22T06:22:59.539938Z","iopub.status.idle":"2023-05-22T06:23:18.799106Z","shell.execute_reply.started":"2023-05-22T06:22:59.539869Z","shell.execute_reply":"2023-05-22T06:23:18.797399Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Se transforma la tabla pivote en una matriz de co-ocurrencia,tomando el \n# producto escalar de la tabla pivote y su transpuesta\nco_matrix = pivot_df.T.dot(pivot_df)\nnp.fill_diagonal(co_matrix.values, 0)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:23:18.800727Z","iopub.execute_input":"2023-05-22T06:23:18.801123Z","iopub.status.idle":"2023-05-22T06:23:20.236641Z","shell.execute_reply.started":"2023-05-22T06:23:18.801088Z","shell.execute_reply":"2023-05-22T06:23:20.233922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Se transforma la matriz de co-ocurrencia en una matriz de similitudes \n# de coseno entre los artículos\ncos_score_df = pd.DataFrame(cosine_similarity(co_matrix))\ncos_score_df.index = co_matrix.index\ncos_score_df.columns = np.array(co_matrix.index)\ncos_score_df","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:23:20.240164Z","iopub.execute_input":"2023-05-22T06:23:20.243042Z","iopub.status.idle":"2023-05-22T06:23:20.325064Z","shell.execute_reply.started":"2023-05-22T06:23:20.242940Z","shell.execute_reply":"2023-05-22T06:23:20.323182Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# De la matriz de similitudes, para cada artículo, toma los top 12 artículos \n# recomendados que no sean el artículo original\n\nproduct_recs = []\nfor i in cos_score_df.index:\n    product_recs.append(cos_score_df[cos_score_df.index!=i][i].sort_values(ascending = False)[0:12].index)\n    \nproduct_recs_df = pd.DataFrame(product_recs)\nproduct_recs_df['recs_list'] = product_recs_df.values.tolist()\nproduct_recs_df.index = cos_score_df.index\n# product_recs_df","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:23:20.327519Z","iopub.execute_input":"2023-05-22T06:23:20.328516Z","iopub.status.idle":"2023-05-22T06:23:20.486578Z","shell.execute_reply.started":"2023-05-22T06:23:20.328459Z","shell.execute_reply":"2023-05-22T06:23:20.485599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Función para obtener las recomendaciones con base en el artículo recibido\ndef get_recs(article_id):\n    return product_recs_df.loc[article_id]['recs_list']","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:23:20.487879Z","iopub.execute_input":"2023-05-22T06:23:20.488413Z","iopub.status.idle":"2023-05-22T06:23:20.492305Z","shell.execute_reply.started":"2023-05-22T06:23:20.488382Z","shell.execute_reply":"2023-05-22T06:23:20.491572Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Obtiene el producto más comprado por customer_id y frecuencia\nmaxcompra=sample_df.groupby(['customer_id','article_id']).size().reset_index(name='counts')\n\n# Obtener el artículo más comprado por customer_id para obtener las \n# recomendaciones con base a ese artículo \nmaxcompra=maxcompra.groupby(['customer_id']).agg({'counts': 'max','article_id':'max'}).reset_index()\n\n# Se agrega una columna con la lista de los 12 artículos recomendados  \n# por cliente, de acuerdo con el artículo consultado\nmaxcompra['recs_list']=maxcompra['article_id'].apply(get_recs)\nmaxcompra = maxcompra.drop('counts', axis=1)\nmaxcompra","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:23:20.493594Z","iopub.execute_input":"2023-05-22T06:23:20.494463Z","iopub.status.idle":"2023-05-22T06:24:47.355276Z","shell.execute_reply.started":"2023-05-22T06:23:20.494431Z","shell.execute_reply":"2023-05-22T06:24:47.353971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Evaluación y comparación de los modelos","metadata":{}},{"cell_type":"markdown","source":"Para evaluar los resultados se corren las funciones previamente definidas.","metadata":{}},{"cell_type":"markdown","source":"Para evaluar los resultados se corren las funciones previamente definidas.","metadata":{}},{"cell_type":"markdown","source":"Vamos a pronosticar la proxima compra y recomendar ciertos productos al clúster que contiene a clientes que tienen una edad mayor a 65 años.","metadata":{}},{"cell_type":"code","source":"cluster=poblacion(transactions,1,40)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:24:47.356781Z","iopub.execute_input":"2023-05-22T06:24:47.357267Z","iopub.status.idle":"2023-05-22T06:25:13.858664Z","shell.execute_reply.started":"2023-05-22T06:24:47.357234Z","shell.execute_reply":"2023-05-22T06:25:13.857364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Los clientes de una edad de 16 a 24 años compran más el producto Jade HW Skinny Denin, además que compran más los últimos días de Junio y Julio.\n\nA continuación, se aplica una reducción de dimensiones para realizar el market basket analysis.","metadata":{}},{"cell_type":"code","source":"tabla=red_base(cluster,200)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:25:13.860930Z","iopub.execute_input":"2023-05-22T06:25:13.861662Z","iopub.status.idle":"2023-05-22T06:25:45.864596Z","shell.execute_reply.started":"2023-05-22T06:25:13.861623Z","shell.execute_reply":"2023-05-22T06:25:45.863247Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#mba(tabla)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:25:45.866588Z","iopub.execute_input":"2023-05-22T06:25:45.867232Z","iopub.status.idle":"2023-05-22T06:25:45.872098Z","shell.execute_reply.started":"2023-05-22T06:25:45.867194Z","shell.execute_reply":"2023-05-22T06:25:45.870835Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clientes_unicos, primer_periodo=rfm(tabla)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:25:45.873743Z","iopub.execute_input":"2023-05-22T06:25:45.874184Z","iopub.status.idle":"2023-05-22T06:25:55.075589Z","shell.execute_reply.started":"2023-05-22T06:25:45.874145Z","shell.execute_reply":"2023-05-22T06:25:55.074188Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"clientes_unicos=puntajes(clientes_unicos,4,6)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:25:55.077217Z","iopub.execute_input":"2023-05-22T06:25:55.077625Z","iopub.status.idle":"2023-05-22T06:25:55.089730Z","shell.execute_reply.started":"2023-05-22T06:25:55.077591Z","shell.execute_reply":"2023-05-22T06:25:55.088288Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"c_unicos=creaion_var_fecha(clientes_unicos,primer_periodo,20,50)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:25:55.091121Z","iopub.execute_input":"2023-05-22T06:25:55.091481Z","iopub.status.idle":"2023-05-22T06:26:03.828826Z","shell.execute_reply.started":"2023-05-22T06:25:55.091450Z","shell.execute_reply":"2023-05-22T06:26:03.827723Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"X_train, X_test, y_train, y_test=modelos(c_unicos)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:26:03.830108Z","iopub.execute_input":"2023-05-22T06:26:03.831030Z","iopub.status.idle":"2023-05-22T06:26:34.919724Z","shell.execute_reply.started":"2023-05-22T06:26:03.830996Z","shell.execute_reply":"2023-05-22T06:26:34.918814Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"xgb_model = xgb.XGBClassifier().fit(X_train, y_train)\n\nprint('Accuracy of XGB classifier on training set: {:.2f}'\n       .format(xgb_model.score(X_train, y_train)))\nprint('Accuracy of XGB classifier on test set: {:.2f}'\n       .format(xgb_model.score(X_test[X_train.columns], y_test)))\n\ny_pred = xgb_model.predict(X_test)\nprint(classification_report(y_test, y_pred))","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:26:34.921345Z","iopub.execute_input":"2023-05-22T06:26:34.922004Z","iopub.status.idle":"2023-05-22T06:26:51.356667Z","shell.execute_reply.started":"2023-05-22T06:26:34.921967Z","shell.execute_reply":"2023-05-22T06:26:51.355382Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"El modelo de datos tipo no supervisado es un modelo relativamente fácil de implementar; sin embargo, presenta una desventaja en el tamaño de las transacciones para generar la tabla pivote, ya que existe un número limitado de celdas para generar la tabla, por lo que se utilizó la delimitación de los artículos más vendidos para generar las recomendaciones. Otra desventaja que presenta este modelo es que no funciona para la recomendación de nuevos artículos (problema de arranque en frío), esto se debe a que no ha habido interacción con ese artículo.","metadata":{}},{"cell_type":"markdown","source":"# Modelo XGBoosting a partir de las compras de los clientes","metadata":{}},{"cell_type":"code","source":"# Contar los valores de la variable \"mi_variable\" y quedarme con los articulos que se vendieron mas de 10,000 piezas\nconteo = transactions['article_id'].value_counts().rename('conteo').reset_index()\nmasvendidos=conteo['conteo']>=7000\nr=conteo[masvendidos]","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:26:51.358358Z","iopub.execute_input":"2023-05-22T06:26:51.358823Z","iopub.status.idle":"2023-05-22T06:26:52.188043Z","shell.execute_reply.started":"2023-05-22T06:26:51.358778Z","shell.execute_reply":"2023-05-22T06:26:52.186781Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"r = r.rename(columns={'index': 'article_id'})","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:26:52.189533Z","iopub.execute_input":"2023-05-22T06:26:52.189910Z","iopub.status.idle":"2023-05-22T06:26:52.199444Z","shell.execute_reply.started":"2023-05-22T06:26:52.189855Z","shell.execute_reply":"2023-05-22T06:26:52.198046Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Resumir la tabla de transacciones\n# Agrupamos por 'id' y 'categoria' y contamos los valores\ncompras = transactions.groupby(['customer_id', 'article_id']).size().reset_index(name='counts')\ncomprasutilizar = (r.merge(compras,on=['article_id'], how='left'))\n#Creamos el prefijo para transponer las variables \n# Transponer tabla\ncomprasfinal = comprasutilizar.pivot_table(index='customer_id', columns='article_id', values='counts', aggfunc='sum',fill_value=0)\ncomprasfinal = comprasfinal.reset_index()","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:26:52.200876Z","iopub.execute_input":"2023-05-22T06:26:52.201277Z","iopub.status.idle":"2023-05-22T06:27:07.817793Z","shell.execute_reply.started":"2023-05-22T06:26:52.201232Z","shell.execute_reply":"2023-05-22T06:27:07.816215Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Creamos la tabla donde se van a guardar los resultados del modelo con la probabilidad de compra de cada prenda\ncalificacion = comprasfinal[['customer_id']]","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:27:07.819581Z","iopub.execute_input":"2023-05-22T06:27:07.820678Z","iopub.status.idle":"2023-05-22T06:27:07.836396Z","shell.execute_reply.started":"2023-05-22T06:27:07.820618Z","shell.execute_reply":"2023-05-22T06:27:07.834689Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(22):\n    #Creamos el modelo XGBoosting para predecir la probabilidad de comprar el artículo i\n    #Articulo i mas vendido\n    prenda = r['article_id'][i] \n    #Obtener los casos que compraron articulo en particular\n    masvendido = transactions['article_id']==prenda\n    sicompraron=transactions[masvendido]\n    sicompraron['target']=1\n    # Obtener la cantidad de filas para balancear los que no compraron esa prenda\n    cantidad_filas = sicompraron.shape[0]\n    #Obtener una muestra aleatoria del mismo tamaño que el articulo que compraron en particular\n    muestra = transactions.query('article_id != \"valor\"').sample(n=cantidad_filas, random_state=42)\n    muestra['target']=0\n    # Concatenamos las dos tablas\n    tabla_combinada = pd.concat([sicompraron, muestra], axis=0)\n    # eliminar duplicados por llave y conservar las variables de interés\n    tabla_combinada = tabla_combinada.drop_duplicates(subset=['customer_id'])[['customer_id', 'target']]\n    cantidad_filas_t = tabla_combinada.shape[0]*.8\n    cantidad_filas_t = int(cantidad_filas_t)\n    #Unir transacciones con clientes para modelo\n    universo=(tabla_combinada.merge(comprasfinal,on=['customer_id'], how='left'))\n    # Dividir los datos en conjunto de entrenamiento y conjunto de prueba\n    variables_a_eliminar = ['customer_id','target',prenda]\n    X_train = universo.drop(variables_a_eliminar, axis=1).iloc[:cantidad_filas_t]\n    y_train = universo['target'].iloc[:cantidad_filas_t]\n    X_test = universo.drop(variables_a_eliminar, axis=1).iloc[cantidad_filas_t:]\n    y_test = universo['target'].iloc[cantidad_filas_t:]\n    # Crear un objeto DMatrix para el conjunto de entrenamiento y el conjunto de prueba\n    dtrain = xgb.DMatrix(X_train, label=y_train)\n    dtest = xgb.DMatrix(X_test, label=y_test)\n    # Definir los parámetros del modelo\n    params = {\n        'objective': 'binary:logistic',\n        'max_depth': 3,\n        'eta': 0.1,\n        'eval_metric': 'error'\n    }\n    # Ajustar el modelo XGBoost a los datos de entrenamiento\n    model = xgb.train(params, dtrain, num_boost_round=100)\n    # Evaluar el modelo en el conjunto de prueba\n    predictions = model.predict(dtest)\n    predictions = np.round(predictions)\n    # Calcular la precisión del modelo en el conjunto de prueba\n    accuracy = np.mean(predictions == y_test)\n    print('Precisión del modelo: {:.2f}%'.format(accuracy * 100))\n    #Calificamos a nuestros clientes con nuestro modelo \n    var_a_eliminar = [prenda,'customer_id']\n    cf = comprasfinal.drop(var_a_eliminar, axis=1)\n    evalua2 = xgb.DMatrix(cf)\n    y_pred = model.predict(evalua2)\n    calificacion[prenda] = y_pred","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:27:07.838845Z","iopub.execute_input":"2023-05-22T06:27:07.839503Z","iopub.status.idle":"2023-05-22T06:30:19.499041Z","shell.execute_reply.started":"2023-05-22T06:27:07.839400Z","shell.execute_reply":"2023-05-22T06:30:19.498050Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Asignar la variable 'ID' como índice\ncalificacion = calificacion.set_index('customer_id')\n# Obtener los 10 valores con probabilidad máxima de compra\nresult = calificacion.apply(lambda row: row.nlargest(10), axis=1)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:30:19.500508Z","iopub.execute_input":"2023-05-22T06:30:19.501163Z","iopub.status.idle":"2023-05-22T06:33:01.382454Z","shell.execute_reply.started":"2023-05-22T06:30:19.501127Z","shell.execute_reply":"2023-05-22T06:33:01.380734Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Recorrer cada celda del DataFrame\nfor fila in result.index:\n    for columna in result.columns:\n        # Reemplazar el valor no nulo con el nombre de la variable correspondiente\n        if not pd.isnull(result.loc[fila, columna]):\n            result.loc[fila, columna] = columna\n\n# Crear las nuevas columnas\nresult['Valores_No_NaN'] = result.apply(lambda row: ', '.join(row.dropna().astype(str)), axis=1)\nrecomendacion = result[['Valores_No_NaN']]\nrecomendacion['customer_id'] = recomendacion.index.astype(str)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:33:01.384506Z","iopub.execute_input":"2023-05-22T06:33:01.385017Z","iopub.status.idle":"2023-05-22T06:37:11.671593Z","shell.execute_reply.started":"2023-05-22T06:33:01.384968Z","shell.execute_reply":"2023-05-22T06:37:11.670329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Visualizamos las recomendaciones de los primeros 15 clientes\nrecomendacion.head(15)","metadata":{"execution":{"iopub.status.busy":"2023-05-22T06:37:11.673743Z","iopub.execute_input":"2023-05-22T06:37:11.674177Z","iopub.status.idle":"2023-05-22T06:37:11.688725Z","shell.execute_reply.started":"2023-05-22T06:37:11.674144Z","shell.execute_reply":"2023-05-22T06:37:11.687791Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Fin del modelo XGBoosting a partir de las compras de los clientes","metadata":{}}]}