{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#

Recommendation system for H and M Fashion

","metadata":{}},{"cell_type":"markdown","source":"

Terminologies

\n\nThere are certain terminologies which needs to be understood before moving forward.\n\n**Apache Spark:** Apache Spark is an open-source distributed general-purpose cluster-computing framework.It can be used with Hadoop too.\n\n**Collaborative filtering:** Collaborative filtering is a method of making automatic predictions (filtering) about the interests of a user by collecting preferences or taste information from many users. Consider example if a person A likes item 1, 2, 3 and B like 2,3,4 then they have similar interests and A should like item 4 and B should like item 1.\n\n**Alternating least square(ALS) matrix factorization:** The idea is basically to take a large (or potentially huge) matrix and factor it into some smaller representation of the original matrix through alternating least squares. We end up with two or more lower dimensional matrices whose product equals the original one.ALS comes inbuilt in Apache Spark.\n\n**PySpark:** PySpark is the collaboration of Apache Spark and Python. PySpark is the Python API for Spark.","metadata":{}},{"cell_type":"markdown","source":"

1.Initialize spark session

","metadata":{}},{"cell_type":"code","source":"!pip install pyspark","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-02-22T21:16:10.603441Z","iopub.execute_input":"2023-02-22T21:16:10.603854Z","iopub.status.idle":"2023-02-22T21:16:53.347273Z","shell.execute_reply.started":"2023-02-22T21:16:10.603766Z","shell.execute_reply":"2023-02-22T21:16:53.346329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

2-Load libraries

","metadata":{}},{"cell_type":"code","source":"import time\nstart_time = time.time()\n\nimport pyspark\nfrom pyspark.sql import SparkSession\nfrom pyspark.sql.types import StructType,StructField, StringType, IntegerType \nfrom pyspark.sql.types import ArrayType, DoubleType, BooleanType\nfrom pyspark.sql.functions import col,array_contains\nfrom pyspark.sql import SQLContext \nfrom pyspark.ml.recommendation import ALS\nfrom pyspark.sql.functions import udf,col,when\nfrom pyspark.sql.functions import to_timestamp,date_format\nimport numpy as np\nimport pandas as pd\nfrom pyspark.sql.types import *\nfrom pyspark.sql.functions import *\nfrom pyspark.sql.window import *\n\nsc = SparkSession.builder.appName(\"Recommendations\").config(\"spark.sql.files.maxPartitionBytes\", 5000000).getOrCreate()\nspark = SparkSession(sc)\n# start_time = time.time()\nprint(\"--- %s seconds ---\" % (time.time() - start_time))\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-22T21:16:53.349811Z","iopub.execute_input":"2023-02-22T21:16:53.350091Z","iopub.status.idle":"2023-02-22T21:17:00.306876Z","shell.execute_reply.started":"2023-02-22T21:16:53.350053Z","shell.execute_reply":"2023-02-22T21:17:00.306034Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

3-Load Dataset in Apache Spark

","metadata":{}},{"cell_type":"code","source":"transaction = spark.read.option(\"header\",True) \\\n .csv(\"/kaggle/input/transaction-data-shop/orders_payments.csv\")\ntransaction.printSchema()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:00.308973Z","iopub.execute_input":"2023-02-22T21:17:00.309504Z","iopub.status.idle":"2023-02-22T21:17:07.729605Z","shell.execute_reply.started":"2023-02-22T21:17:00.309454Z","shell.execute_reply":"2023-02-22T21:17:07.728053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pyspark.sql.functions import min, max\nfrom pyspark.sql.functions import unix_timestamp, lit\nfrom pyspark.sql.types import IntegerType\n\ntransaction = transaction.withColumn(\"user_id\", transaction[\"user_id\"].cast(IntegerType()))\ntransaction = transaction.withColumn(\"product_id\", transaction[\"product_id\"].cast(IntegerType()))\ntransaction = transaction.withColumn(\"quantity\", transaction[\"quantity\"].cast(IntegerType()))\n# min_date, max_date = transaction.select(min(\"t_dat\"), max(\"t_dat\")).first()\n# min_date, max_date","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:07.732589Z","iopub.execute_input":"2023-02-22T21:17:07.736397Z","iopub.status.idle":"2023-02-22T21:17:07.834919Z","shell.execute_reply.started":"2023-02-22T21:17:07.736353Z","shell.execute_reply":"2023-02-22T21:17:07.834097Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Edit model","metadata":{}},{"cell_type":"code","source":"# training_RDD, validation_RDD, test_RDD = viewer_product_prepare_model.randomSplit([6, 2, 2], seed=0)\n# validation_for_predict_RDD = validation_RDD.map(lambda x: (x[0], x[1]))\n# test_for_predict_RDD = test_RDD.map(lambda x: (x[0], x[1]))\n\n# from pyspark.mllib.recommendation import ALS\n# import math\n\n# seed = 5L\n# iterations = 10\n# regularization_parameter = 0.1\n# ranks = [4, 8, 12]\n# errors = [0, 0, 0]\n# err = 0\n# tolerance = 0.02\n\n# min_error = float('inf')\n# best_rank = -1\n# best_iteration = -1\n# for rank in ranks:\n# model = ALS.train(training_RDD, rank, seed=seed, iterations=iterations,\n# lambda_=regularization_parameter)\n# predictions = model.predictAll(validation_for_predict_RDD).map(lambda r: ((r[0], r[1]), r[2]))\n# rates_and_preds = validation_RDD.map(lambda r: ((int(r[0]), int(r[1])), float(r[2]))).join(predictions)\n# error = math.sqrt(rates_and_preds.map(lambda r: (r[1][0] - r[1][1])**2).mean())\n# errors[err] = error\n# err += 1\n# print('For rank %s the RMSE is %s' % (rank, error))\n# if error < min_error:\n# min_error = error\n# best_rank = rank\n\n# print('The best model was trained with rank %s' % best_rank)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pyspark.sql.functions import min, max\nfrom pyspark.sql.functions import unix_timestamp, lit\n","metadata":{"execution":{"iopub.status.busy":"2023-02-22T16:31:07.631610Z","iopub.execute_input":"2023-02-22T16:31:07.632362Z","iopub.status.idle":"2023-02-22T16:31:07.636555Z","shell.execute_reply.started":"2023-02-22T16:31:07.632324Z","shell.execute_reply":"2023-02-22T16:31:07.635675Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

5-Select data for recommendation

","metadata":{}},{"cell_type":"code","source":"# Prepare the dataset\ntransaction_group = transaction.groupby('user_id', 'product_id').agg(sum(\"quantity\").alias(\"count\"))\n# hm = hm.groupby('customer_id', 'article_id').sum(\"name cot\")\ntransaction_group.show(5)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:07.835974Z","iopub.execute_input":"2023-02-22T21:17:07.836216Z","iopub.status.idle":"2023-02-22T21:17:09.928696Z","shell.execute_reply.started":"2023-02-22T21:17:07.836184Z","shell.execute_reply":"2023-02-22T21:17:09.927424Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print((transaction_group.count(), len(transaction_group.columns)))\nhm = transaction_group","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:09.929777Z","iopub.execute_input":"2023-02-22T21:17:09.932436Z","iopub.status.idle":"2023-02-22T21:17:10.925120Z","shell.execute_reply.started":"2023-02-22T21:17:09.932392Z","shell.execute_reply":"2023-02-22T21:17:10.924378Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"numerator = hm.select(\"count\").count()\n\nnum_users = hm.select(\"user_id\").distinct().count()\nnum_articles = hm.select(\"product_id\").distinct().count()\n\n# Đặt mẫu số bằng số lượng khách hàng nhân với số lượng bài viết\ndenominator = num_users * num_articles\n\n# Chia tử số cho mẫu số\nsparsity = (1.0 - (numerator *1.0)/denominator)*100\nprint(\"Sparsity: \", \"%.2f\" % sparsity + \"%.\")","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:10.926113Z","iopub.execute_input":"2023-02-22T21:17:10.926358Z","iopub.status.idle":"2023-02-22T21:17:12.786069Z","shell.execute_reply.started":"2023-02-22T21:17:10.926309Z","shell.execute_reply":"2023-02-22T21:17:12.785380Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"userId_count = hm.groupBy(\"user_id\").count().orderBy('count', ascending=False)\nuserId_count.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T17:41:50.906342Z","iopub.execute_input":"2023-02-22T17:41:50.907070Z","iopub.status.idle":"2023-02-22T17:41:51.212899Z","shell.execute_reply.started":"2023-02-22T17:41:50.907032Z","shell.execute_reply":"2023-02-22T17:41:51.212054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"userId_count.count()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T17:40:06.896031Z","iopub.execute_input":"2023-02-22T17:40:06.896317Z","iopub.status.idle":"2023-02-22T17:40:07.405355Z","shell.execute_reply.started":"2023-02-22T17:40:06.896286Z","shell.execute_reply":"2023-02-22T17:40:07.404554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articleId_count = hm.groupBy(\"product_id\").count().orderBy('count', ascending=False)\narticleId_count.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T17:40:19.454859Z","iopub.execute_input":"2023-02-22T17:40:19.455141Z","iopub.status.idle":"2023-02-22T17:40:19.956207Z","shell.execute_reply.started":"2023-02-22T17:40:19.455109Z","shell.execute_reply":"2023-02-22T17:40:19.955016Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"articleId_count.count()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T17:40:30.714364Z","iopub.execute_input":"2023-02-22T17:40:30.714658Z","iopub.status.idle":"2023-02-22T17:40:31.081803Z","shell.execute_reply.started":"2023-02-22T17:40:30.714625Z","shell.execute_reply":"2023-02-22T17:40:31.080977Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

5-Importing important modules

","metadata":{}},{"cell_type":"code","source":"from pyspark.ml.evaluation import RegressionEvaluator\nfrom pyspark.ml.recommendation import ALS","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:17:12.787092Z","iopub.execute_input":"2023-02-22T21:17:12.787363Z","iopub.status.idle":"2023-02-22T21:17:12.798075Z","shell.execute_reply.started":"2023-02-22T21:17:12.787311Z","shell.execute_reply":"2023-02-22T21:17:12.797371Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

6-Data for train

\n","metadata":{}},{"cell_type":"code","source":"# from pyspark.ml.feature import StringIndexer\n# from pyspark.ml import Pipeline\n# from pyspark.sql.functions import col\n# indexer = [StringIndexer(inputCol=column, outputCol=column+\"_index\") for column in list(set(hm.columns)-set(['count'])) ]\n# pipeline = Pipeline(stages=indexer)\n# transformed = pipeline.fit(hm).transform(hm)\n# transformed.show()\n\ntransformed = hm\ntransformed.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:19:18.261379Z","iopub.execute_input":"2023-02-22T21:19:18.261657Z","iopub.status.idle":"2023-02-22T21:19:18.606757Z","shell.execute_reply.started":"2023-02-22T21:19:18.261627Z","shell.execute_reply":"2023-02-22T21:19:18.605939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

7-Creating training and test data

","metadata":{}},{"cell_type":"code","source":"(training, test)=transformed.randomSplit([0.8, 0.2])\n# (training,validation, test)=transformed.randomSplit([0.6,0.2, 0.2])","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:19:23.208119Z","iopub.execute_input":"2023-02-22T21:19:23.208404Z","iopub.status.idle":"2023-02-22T21:19:23.242355Z","shell.execute_reply.started":"2023-02-22T21:19:23.208370Z","shell.execute_reply":"2023-02-22T21:19:23.241540Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

Tạo mô hình ALS và khớp dữ liệu

\n\nĐể xây dựng mô hình chỉ định rõ ràng các cột. Đặt giá trị không âm là 'True', vì chúng tôi đang tìm kiếm số lượng lớn hơn 0. Mô hình cũng cung cấp tùy chọn để chọn xếp hạng ẩn. Vì chúng tôi đang làm việc với tính rõ ràng, hãy đặt nó thành 'Sai' hoặc theo mặc định, nó sẽ rõ ràng.\n\nKhi sử dụng các phân tách ngẫu nhiên đơn giản như trong Spark's CrossValidator hoặc TrainValidationSplit, thực tế rất phổ biến khi gặp phải người dùng và/hoặc các mục trong tập đánh giá không có trong tập huấn luyện. Theo mặc định, Spark chỉ định các dự đoán NaN trong ALSModel.transform khi yếu tố người dùng và/hoặc mục không có trong mô hình. Chúng tôi đặt chiến lược khởi động nguội thành 'thả' để đảm bảo chúng tôi không nhận được chỉ số đánh giá NaN.","metadata":{}},{"cell_type":"code","source":"from pyspark.ml.evaluation import RegressionEvaluator\nfrom pyspark.ml.recommendation import ALS\nfrom pyspark.ml.tuning import CrossValidator, ParamGridBuilder\n\n\n#create ALS model\nals=ALS(userCol=\"user_id\",itemCol=\"product_id\",ratingCol=\"count\",coldStartStrategy=\"drop\",nonnegative=True)\n\n#tune model using ParamGridBuilder\nparam_grid = ParamGridBuilder()\\\n .addGrid(als.rank, [15,20,25])\\\n .addGrid(als.maxIter,[5,10,15])\\\n .addGrid(als.regParam,[0.09,0.14,0.19])\\\n .build()\n#define evaluator as RMSE\nevaluator = RegressionEvaluator(metricName = \"rmse\",labelCol = 'count', predictionCol = 'prediction')\n\n#Build cross validation using CrossValidator\ncv = CrossValidator(estimator=als,estimatorParamMaps=param_grid, evaluator=evaluator,numFolds=3)\n\n\n#Fit ALS model to training data\nmodel = cv.fit(training)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-22T21:19:27.056102Z","iopub.execute_input":"2023-02-22T21:19:27.056381Z","iopub.status.idle":"2023-02-22T21:41:58.588778Z","shell.execute_reply.started":"2023-02-22T21:19:27.056348Z","shell.execute_reply":"2023-02-22T21:41:58.587517Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# \"\"\"\"\"\"als=ALS(maxIter=5,regParam=0.09,rank=25,userCol=\"customer_id_index\",itemCol=\"article_id_index\",ratingCol=\"count\",coldStartStrategy=\"drop\",nonnegative=True)\n# model=als.fit(training)\"\"\"\"\"\"","metadata":{"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

9-Evaluate rmse

","metadata":{}},{"cell_type":"code","source":"#Extract best model from the tuning exercise using ParamGridBuilder\nbest_model = model.bestModel\n\n#Generate predictions and evaluate using RMSE\npredictions = best_model.transform(test)\nrmse = evaluator.evaluate(predictions)","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-02-22T21:41:58.594616Z","iopub.execute_input":"2023-02-22T21:41:58.596944Z","iopub.status.idle":"2023-02-22T21:41:59.486843Z","shell.execute_reply.started":"2023-02-22T21:41:58.596903Z","shell.execute_reply":"2023-02-22T21:41:59.485868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#print evaluation metrics and model parameters\nprint(\"RMSE =\" + str(rmse))\nprint(\"**Best Model**\")\nprint(\"Rank : {}\".format(best_model.rank))\nprint(\"MaxIter: {}\".format(best_model._java_obj.parent().getMaxIter()))\nprint(\"RegParam: {}\".format(best_model._java_obj.parent().getRegParam()))","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:41:59.488490Z","iopub.execute_input":"2023-02-22T21:41:59.488800Z","iopub.status.idle":"2023-02-22T21:41:59.499927Z","shell.execute_reply.started":"2023-02-22T21:41:59.488764Z","shell.execute_reply":"2023-02-22T21:41:59.499250Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

10-Providing Recommendations by Article id

","metadata":{}},{"cell_type":"code","source":"user_recs=best_model.recommendForAllItems(12).show(10)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T18:25:51.403272Z","iopub.execute_input":"2023-02-22T18:25:51.403556Z","iopub.status.idle":"2023-02-22T18:25:55.957439Z","shell.execute_reply.started":"2023-02-22T18:25:51.403525Z","shell.execute_reply":"2023-02-22T18:25:55.956793Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n

11-Providing Recommendations by Customer id

","metadata":{}},{"cell_type":"code","source":"df_recom = best_model.recommendForAllUsers(12)\ndf_recom.show(10)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:00.003956Z","iopub.execute_input":"2023-02-22T21:58:00.004232Z","iopub.status.idle":"2023-02-22T21:58:04.656484Z","shell.execute_reply.started":"2023-02-22T21:58:00.004202Z","shell.execute_reply":"2023-02-22T21:58:04.655780Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_recom = df_recom.select(\"user_id\",\"recommendations.product_id\")\ndf_recom.show(10)\ndf_recom_pandas = df_recom.toPandas()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:04.657698Z","iopub.execute_input":"2023-02-22T21:58:04.657925Z","iopub.status.idle":"2023-02-22T21:58:10.882084Z","shell.execute_reply.started":"2023-02-22T21:58:04.657895Z","shell.execute_reply":"2023-02-22T21:58:10.881383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# recommend by user id dataframe spark\n# df_recom.write.csv(\"/kaggle/working/df_recom.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:42:09.483231Z","iopub.status.idle":"2023-02-22T21:42:09.485515Z","shell.execute_reply.started":"2023-02-22T21:42:09.485227Z","shell.execute_reply":"2023-02-22T21:42:09.485256Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_recom_pandas.sort_values('user_id')","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:17.848764Z","iopub.execute_input":"2023-02-22T21:58:17.849030Z","iopub.status.idle":"2023-02-22T21:58:17.864197Z","shell.execute_reply.started":"2023-02-22T21:58:17.848999Z","shell.execute_reply":"2023-02-22T21:58:17.863547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_recom_pandas['product_id'] = [' '.join(map(str, l)) for l in df_recom_pandas['product_id']]","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:23.063922Z","iopub.execute_input":"2023-02-22T21:58:23.064494Z","iopub.status.idle":"2023-02-22T21:58:23.070491Z","shell.execute_reply.started":"2023-02-22T21:58:23.064450Z","shell.execute_reply":"2023-02-22T21:58:23.069714Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_recom_pandas.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:26.583756Z","iopub.execute_input":"2023-02-22T21:58:26.584019Z","iopub.status.idle":"2023-02-22T21:58:26.593793Z","shell.execute_reply.started":"2023-02-22T21:58:26.583988Z","shell.execute_reply":"2023-02-22T21:58:26.592591Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# convert to csv file from pandas\n# df_recom_pandas.to_csv(\"/kaggle/working/recommenr for user_id.csv\", encoding='utf-8', index=False)\n# df_recom_pandas.to_csv(\"/kaggle/working/recommend for user_id_1234.csv\",index=False)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:48:20.273415Z","iopub.execute_input":"2023-02-22T21:48:20.273707Z","iopub.status.idle":"2023-02-22T21:48:20.284700Z","shell.execute_reply.started":"2023-02-22T21:48:20.273674Z","shell.execute_reply":"2023-02-22T21:48:20.283715Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df_recom_pandas.to_csv(\"/kaggle/working/recommend_no_header.csv\",index=False,header=False)","metadata":{"execution":{"iopub.status.busy":"2023-02-22T22:00:32.504743Z","iopub.execute_input":"2023-02-22T22:00:32.505022Z","iopub.status.idle":"2023-02-22T22:00:32.513738Z","shell.execute_reply.started":"2023-02-22T22:00:32.504990Z","shell.execute_reply":"2023-02-22T22:00:32.512984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"ok export\")","metadata":{"execution":{"iopub.status.busy":"2023-02-22T21:58:39.948470Z","iopub.execute_input":"2023-02-22T21:58:39.948738Z","iopub.status.idle":"2023-02-22T21:58:39.952916Z","shell.execute_reply.started":"2023-02-22T21:58:39.948708Z","shell.execute_reply":"2023-02-22T21:58:39.952214Z"},"trusted":true},"execution_count":null,"outputs":[]}]}