{"cells":[{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"pip install pyecharts -U","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom pyecharts.charts import Bar,Page,Line\nfrom pyecharts.charts import PictorialBar\nfrom pyecharts import options as opts\nfrom pyecharts.options import ComponentTitleOpts\nfrom pyecharts.globals import SymbolType\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"------------*12.11 更新至 version7*------------\n\n更新了describe函数的用法；\n\n添加了notebook提交结果的方法\n\n------------12.14 更新至 version8------------\n\n修改了env里的代码（.res改成.answered_correctly）错误\n\nps: 虽然修改了，我的output看上去也很正常，但还是在几个小时后Submission Scoring Error了, 持续寻找原因中，等我成功了就来分享模型\n\n----------------这里是更新分界线-----------------\n\n读取数据"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_df = pd.read_csv('/kaggle/input/riiid-test-answer-prediction/train.csv',\n                       low_memory=False, \n                       nrows=10**6)\ntrain_df.describe()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"使用describe函数了解表格数据属性：\n* 根据count看到：数据总量为100w条，prior_question_elapsed_time有缺失值（实际上用unique函数可以看到prior_question_had_explanation也有缺失值）。\n* 根据mean看到，大部分都是question行，lecture占比很小，用户答案略微偏向于2，3（可能大家更偏向于蒙C，也可能是答案本身的倾向），用户整体答对题目的概率为0.6175，题目平均花费的时间为25s（25303ms）\n* 根据min和max可以看到，这些用户中做题最快的用户与相应题目只花了16s，最慢的花了300s也就是5min（不知道这位老兄做了这么长时间做对没有）！"},{"metadata":{},"cell_type":"markdown","source":"标签理解：\n\n**train_df:**\n* timestamp：用户完成题目时间的时间戳，完成第一个题目（0）开始计算\n* task_containner_id：一组题目被用户开始看到的id，其与content_id的关系（几个题一起出现，共用一个id）和bundle_id与content_id的关系具有一致性。\n    然而这个id是事先分配好然后按顺序排好拿给用户去看的，但是用户可以选择先做哪个题。所以它和时间戳并不一致。（用户不一定先回答第一个题目）\n* content_id：内容的id，对应 lecture_id和question_id\n* content_type：内容类型，对应讲座（1）跟做问题（0）\n* prior_question_elapsed_time：（如果上一个bundle有很多question，那么这个时间是这么多个题目的平均时间，如果是第一个题目，则没有这个时间）用户回答前一个问题包中的每个问题所花费的平均时间(以毫秒为单位)\n* prior_question_had_explanation：用户是否看到上一个问题的答案，第一个题目为null。通常前几个都为false，因为那是测试。\n* user_answer & answerd_correctly：Read -1 as null（用户看过的讲座不会有回答）\n\n**lecture：**\n*       tag：题目标签，猜想大致含义是一个问题所考到的知识点或能力\n*       typeof：讲座讲解的题目类型\n*       part：题目所属部分（托业考试分为听力（4个）和阅读（3个），细分总共有7个部分）\n      \n**question：**\n*       tags：同上\n*       part：同上\n*       correct_answer：正确答案\n*       bundle_id：猜想是有些问题属于同一组（比如一个reading有几个问题,这些问题需要一起给出），所以个别问题共用一个bundle_id，但是大部分和question_id一样。也就是说如果几个问题共用一个bundle_id，那么下一个bundle_id不是顺序增长，而是和question_id保持一致\n\n另外有一些我还不是很确定的理解：\n* prior_group_answers_correct：前一个组的所有answered_correct字段（0代表用户答错了，1代表用户答对了？）\n* prior_group_responses：前一个组的所有user_answer条目 （那些正确答案）（eval是想表达什么？）\n* group_num：问题的组数（可能只是用来判断prior_group_answers_correct和prior_group_responses属于哪里的）\n\n\n作图看几个标签之间的关系"},{"metadata":{"trusted":true},"cell_type":"code","source":"#先选出类型为question的数据，用0填充null，把true和false换成0跟1\ntdf = train_df[train_df['content_type_id'] == 0]\ntdf.fillna(0.0,inplace=True)\ntdf['prior_question_had_explanation'] = tdf.prior_question_had_explanation.apply(lambda x: 1 if x == True else 0)\n#做图\ng = sns.pairplot(tdf,\n                 vars = ['prior_question_elapsed_time', 'content_id', 'answered_correctly','prior_question_had_explanation'],\n                 kind='scatter',\n                 markers = '.')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"上图可以看出用户一般会选择看问题的解答，答对题目的概率大概在67%左右，个别题目花费的时间比较长，但是这些因素与用户实际作答正确与否并无太大关系"},{"metadata":{},"cell_type":"markdown","source":"接下来统计一下用户的刷题数量与类型分布"},{"metadata":{"trusted":true},"cell_type":"code","source":"#计算每个用户做过多少努力（看讲座与做重复题目也算在内）\ndf1 = train_df[['user_id']]\ndf1['content_cnt'] = df1.user_id.apply(lambda x: 1)\ndf1 = df1.groupby('user_id').sum().reset_index()\ndf1 = df1.sort_values('user_id',ascending = True)\nprint(df1.head())\n\n#各用户做题数量分布\nusers = df1['user_id'].tolist()\ncontent_cnt = df1['content_cnt'].tolist()\n\n#用pyecharts作图\nd1 = (\n    Bar()\n        .add_xaxis(users)\n        .add_yaxis('content_cnt', content_cnt)\n        .set_series_opts(label_opts = opts.LabelOpts(is_show = False))\n        .set_global_opts(\n            yaxis_opts = opts.AxisOpts(name = 'content_num'),\n            xaxis_opts = opts.AxisOpts(name = 'user_id')\n        )\n)\nd1.render_notebook()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"可以看出大部分用户的做题量在1k以内，但是有很多用户非常热衷于刷题，甚至可以做到1w题（在下佩服）"},{"metadata":{"trusted":true},"cell_type":"code","source":"#计算每个用户是否看过演讲（0:只做过题；1:只看过讲座；2:都做过）\ndf2 = train_df[['user_id','content_type_id']].drop_duplicates()\ndf2['content_type_id'] = df2.content_type_id.apply(lambda x: 1 if x == 0 else 2)\ndf2 = df2.groupby('user_id').sum().reset_index()\ndf2 = df2.rename(index = str, columns = {\"content_type_id\":\"content_type_cnt\"})\ndf2['content_type_cnt'] = df2.content_type_cnt.apply(lambda x: x-1)\nprint(df2['content_type_cnt'].unique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"这个结果表明，没有只看讲座不刷题的懒学生。也可能因为测试题是一定要做的吧。"},{"metadata":{"trusted":true},"cell_type":"code","source":"#计算是否看演讲与用户答题正确率的关系\ncol = ['user_id','proba']\ndf3 = pd.DataFrame(columns = col)\nuser_id = df2['user_id'].unique()\ni=0\nfor uid in user_id:\n    tdf = train_df[train_df['user_id'] == uid]\n    tdf = tdf[tdf['content_type_id'] == 0]\n    tdf = tdf[['user_id','answered_correctly']]\n    proba = tdf['answered_correctly'].sum()/tdf.shape[0]\n    df3.loc[i] = {col[0]:uid,col[1]:proba}\n    i += 1\ndf3 = pd.merge(df2,df3,how = 'outer')\ndf3 = df3.drop('user_id',1)\ndf3 = df3.groupby('content_type_cnt').mean().reset_index()\ndf3 = df3.rename(index = str, columns = {\"answered_correctly\":\"correct_proba\"})\nprint(df3)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"毫不意外，看过讲座的同学他的答题正确率比那些只输出不输入的同学要高，这里居然高出10%！但是考虑到有些用户也许是机构老师（不需要看讲座），也有些学生只是不在这个平台看讲座（线下学习什么的），所以这个结果可能还有争议。"},{"metadata":{"trusted":true},"cell_type":"code","source":"#做出各种努力的量化指标（just求和）对应的用户人数\ndf4 = df1.drop('user_id',1)\ndf4['user_num'] = df4.content_cnt.apply(lambda x: 1)\ndf4 = df4.groupby('content_cnt').sum().reset_index()\ndf4 = df4.sort_values('content_cnt',ascending = True)\n\n#只做题与还看讲座，两种类型对应的用户人数\ndf5 = df2.drop('user_id',1)\ndf5['user_num'] = df5.content_type_cnt.apply(lambda x: 1)\ndf5 = df5.groupby('content_type_cnt').sum().reset_index()\ndf5 = df5.sort_values('content_type_cnt',ascending = True)\n\n#作图展示结果，滑动条可以拉","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"b1 =(\n    Bar()\n        .add_xaxis(df4['content_cnt'].tolist())\n        .add_yaxis('user_num',df4['user_num'].tolist())\n        .set_series_opts(label_opts = opts.LabelOpts(is_show = False))\n        .set_global_opts(\n            xaxis_opts = opts.AxisOpts(name = 'content_cnt'),\n            yaxis_opts = opts.AxisOpts(name = 'user_num'),\n            datazoom_opts=opts.DataZoomOpts(is_show = True,\n                                            is_realtime = True,\n                                            orient = \"horizontal\")\n        )\n)\nb2 = (\n    Bar()\n        .add_xaxis(df5['content_type_cnt'].tolist())\n        .add_yaxis('user_num',df5['user_num'].tolist())\n        .set_global_opts(\n            xaxis_opts = opts.AxisOpts(name = 'types')\n        )\n)\n\npage = Page()\npage.add(b1,b2)\npage.render_notebook()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"虽然看讲座对于学习很有帮助（前面的统计得出），但是实际上不看讲座的学生更多，看讲座的同学只占1475/3824≈38.57%"},{"metadata":{},"cell_type":"markdown","source":"有的人不喜欢用pyecharts，那么我就用sns写一遍吧，效果上还是觉得pyecharts好看一些"},{"metadata":{"trusted":true},"cell_type":"code","source":"#用sns的方法写\nfig,ax = plt.subplots(2,figsize=(10,15))\ng1 = sns.lineplot(x = 'content_cnt', y = 'user_num', data = df4, ax = ax[0])\ng2 = sns.barplot(x = 'content_type_cnt', y = 'user_num', data = df5, ax = ax[1])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"可能new kaggler尝试写一个模型之后会找不到如何提交模型，这里给新手一个解答：\n\n先运行notebook保存一个version（save&run，注意选择关掉internet），跑完后选择view查看你的notebook版本。\n在你的notebook右上角有个省略号，里面可以选择提交。\n![image.png](attachment:image.png)\n\n关于Submission Scoring Error的问题，本竞赛在https://www.kaggle.com/sohier/quick-sample-submission 有做解答.\n如果不是env的问题，在另一个discussion里也有讨论：https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/192124 \n针对获取测试集提交结果的方式，我这里有我自己的代码推荐：\n","attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAU4AAACkCAYAAAD4+8c5AAAgAElEQVR4Ae2d+W8UaXrHJ5cSbRQlUpSVIiVKIiWKFG0i5dwlye5kJzMQ2OWaYWEAcwynwQz3mMOAuTwYbIzBxsZgMBh84At8gLEx86+90eeFp3n7paqry13dXd08P5Squ+o9n/d5P+/3fd/qrk9++OEHo4faQH1AfUB9ILcPzMzMGDk+UWPlNpbaR+2jPqA+gA8INDkrOFVx64xDfUB9IA8fUHDmYSRVGaoy1AfUB1wfUHAqOFVhqA+oD8T0AQVnTIO5o45+VhWiPvBx+oCCU8GpakN9QH0gpg9UJTifP580Q8PDZmBwSI8U24A2oq2SUm2rWx6ZH9XdMJ/sbdUjZTagXWifpNq63OlUHTjpiM+ePzcLCwtV00jldpJi5U8b0VZJwJNOqcBM/4BRLfDMAmc5FNrg0LCZmJgwc3PziYAOFaPQrJx1J9qKNisUzqo00w9NBjbaqdC2TkP8LHCWo0B0HAoBQJOAJ/AvRz00z8XDOok2U7VZGeCknaqhr5QdnGJECoLylO+LPSfRCRebt8ZbHDyTaDMFp4KzlP1PwNne3l7eXw6hPFGdhVY+iU5YaBk0fjyAJtFmCk4FZyn7XWrASaWT6EBJpFHKBtC8kml3BaeCs5R9ScEZ8/mtUjbOx5JXEoOdglPBWcr+ouBUcBa8PFKowyo4Kwd6SQxQhfpLGuIrOBWcCs6UPSyeBJzSnEYawFdoGRScCk4Fp4KzpD8eKBRaaYiv4FRwKjgVnArOmBxQcMY0WBpGu2org65x6hpnpfm0glPBqYpTFacqzpgcUHDGNFiljYyVUF5VnKo4K8FP3TIqOBWcqjhVcarijMkBBWdMg7mjjn6O99PKMHtVu+L8ScMds/v2kPn6xmPzp4dulhRSaXwsKcwPKum6gjNPcM7Pz5uHDx+alpYWe/CZa3Ebe/b1gukYmTYtg5Nm8tXrrPgz86/N2Mt5s/DmTdb1uHm44d+8+cGcezRuDt99auYXkkvXzaPQz9UMzr8/1WVqu4bN0Xsj5si9EbOp/UlOeBL+iysPPjg+a7pv/uxwMHR/VNdmfnGp1/zs/D0L5S++f2BuDE+aNa3p/I/SQv0lDfFjgfPVq1f2H4w4F6PwSXSgJNLw68YfkPT29pq2tjYLT6DJZ65xzw8f9B2A7e8eNr9f15ZRHL9de918db3fzL9LY3vHgPnxkVtm5MVcXmkG5eNfezG/YP7pzB3z1/Wd5tlsfND76RXjexJtlkZlhd3XtvbbNj58dyQDT9o8rLzHe0cD235mfsHCNCjeP5/tsT7T9/ylTRdIM/he7J+w3//rYq+p7x01y5r7QvMNSrdY14rhQ6VOMy9wvn792vT09Jj6+npz9OhRe3R1dRmuJ1ngJDpQEmn4dZqcnLQqc2pqKlNfPqM+ueeHD/p+ZeC5+f39bXa6Nju/YGF5sOepvXao56lNoxjgpCyoXNRsULnScC2JNitWJ19sukzPUZr7uofNxvYnGXiiOqPAyUDKDMFVnrkUpw9OFOiSC/cyCvWbjgHrb0B5sfVJMl4afK7QMuQFzidPnpizZ8+a0dFR8+bNG6s6z58/b9VXoQVw4yfRgZJIwy0Tn0dGRkxnZ2fW1JxpOte454cP+h4ERQC6rWPA+OBkKr+iuc/87Ykuq1IJJ2nyGfWAmuH4tmfYyH2m/nTKM33jprZryPzdyS7TMz5j09/RMWAB+mRq1qy89tC0DU+amvYnNg/yGvVUbu/EC/PfF3sNU0em+qgX0vaXF6RchZyTaLMkO3ahaQk0sTmzDJmif9naH7nOCdwAJ7ALKwdT9pMPRs34y3nzaHLWhmWWIopzVcsjc2d8xmy7NWAP2hwFOjzzyrQMTdo2DUu7FNcL8ZW0xI0EJ4BgWtrd3W2hKQVnmoriWsw6n6Thn5PoQEmk4ZcrCXCeuD9qfqf2ulnf1m+Ydvl58B24/kFdm/nL4x3ml033zb+c6zG/VXvdrGx5aJjqA8h/b7xr/ujbdgsx1sv+8MAN83/NfXb9ks7DVJ/7f1N/26y42mc7E2n9pKHbqs7bozM2DpsU/9bYY/7z4j27fCD3KUcz6riuzfzVd51m6ZW3AOdz0ssIYoMk2qwUHT6fPHxoMm2W9c1fX3sYCkNJOwqcqMn2p1MWhAPTs7Z9J1/Nm5fzCxlwugoTeCo4k9nEFH/lHAnOFy9emEuXLpmhoexXUqA+UZ3u9NVNeDGfk+hASaThln18fNw0NzebpqamwIN7hHHjBH1GRaDwWNfkAEQ7OwfN9Nz7KbSA8/rw2+k/mzk/v9RrVSGdg+k2yg9lIXlsuTlg/vxohxl9OWfXuYAba1qybkq4IHACcGDMfRQRAAaqTOv/tbHHqlVZE52ae23+4XS3gjPisaVc0IzaFHLB+YbllXmWV94fXWMzFrps+ADJWyNTBogSb1fnoG1vUZwuOLnvf5e8ynUW363ks4CT8ydBFQGMAHJsbCzTWQnH948BnKjNMGjK9Xyn69gNCJ3uG7NrUL+3r838ycF20zk6bW0bNJ33rzHlYhrNVI03BgJKUYKiOInjtmUQOM/3v39NCZ8FnAAYEP+mrT8rDQAt+bhpJ/E5icGuXBCQfJOAJmmhOGljVCLtLIds9KBeGVBZP5W8GehoNwVn8soyzL8LAmdjY2PVK86kwek2BJ0DSKEq6Qw+JAnrXgO6TN8B7j+euWNYM2MdUoCWBDjD0nDL4dYhic+VDk7UOCDjWU3Wq93peb5KUyAYNVUPuu9vDvkK0/8ueZXrXKjPDA8PmxMnTthNatl74MzGNde5//z5c3Pu3Dl7rbW1NdElRcofCc6wqfrTp0/NhQsXbAKFGkLiJ9GBkkhDysM5CXDOLSyYz79/YNcT3d1tpmM4PZs8PDYUBCf3GsqQ16t2vFOolM9VgmHQi6M4KR/rnZRLNp2Y9v/0/N0MoF37JPE5iTYrFwTIl3Vill145IuNNx49kg2huA+8B4HRrZtA8OzD8YziZOaB/3wsivPx48cWiIDyypUrZnp62p7liR/uMyOW74RJ+hHKSHDyyBFvcrt9+3bmmUV21tksYtOo2jeHkgAncKm7M2w3h9hswcEHpmbtBs/v1l639wjjQlKA5F6TTRs61+uFN6Z7bMYq1iQVJ/mSPhtZ/9v0wDT0jRkeqGZJQfKRsiV1rgZwojT3dA3ZmcDerqHIB91dGLqfpW1bhybNqQdjmQMVi7IVdcnaOGubbBziT0zvw8DJL5bsDy9Gpw2PNcnaqJtvKT8X6jcuONmgRtxxFlCmApxUEnXJ40j9/f1mdnbWnvk+ODiYtQ5WqEGS6EBJpOHWI8nNIVQJilGclJ1rHlmRjRwXklIG9xrh2Clnc4k0/uJYh+0IArQkFCf5smlEpyX9Hx++Zdc72diSfKRsSZ2TaDOxaTnOKE6Zoi9WaUq5AWeQXd0H4AHh83c/ZmAjiV12HicLAyftyCNuhGWT8dPLvRkflHxLeQ6qX5xriDUUJRvUKD/icuY717mP4KPvci3fZ63jlCFScZIYv44Bmg0NDZbqrCP09fVlFGicDHOFTaIDJZFGrjIWeg9lwM8qC/lpJdPpQuJH1QFwMvVzw7m79+71JD4n0Wal7PhpyAvVyM8sUaBpKE+cMiThM+VOIy9wSiEB6NzcXOLAlPST6EBJpCHl+VjPG9sfmz8+2G5VJyr2zMNxqzyZshfj9+5JtFmcjqthy/s3dtXQr2KBs9gVTqIDJZFGseuZ9vTZFFrX1p/5XT1LCqyluc+cJlmHJNpMYVheGMaxf5K+U660FJx5/jtSuRroY8hXwVk50IsDyLCw1eDTCk4FZ9ZaZjmcWsGp4CyH3xWSp4JTwangjPgpZZhy0uuLA34hwEpLXAWnglPBqeAs6c58WuBXSDkUnApOBaeCU8EZkwMKzpgGK2SU0rjBf8Kga5yLm/JW6lJBNfQDBaeCUxWnKk5VnDE5oOCMabBqGC3TVgdVnKo40+aTUeVRcCo4VXGq4lTFGZMDqQEnP+ccHBouuBMnoV6iRhu9H7xWuVi7JNFmlbre9zGWe7F+kisePwXnyBUmyXuLAid/K5f0MT0zY8bHJwpOd2BgsOA0kq6bppfbX5JoM/tXaXtazCd6pNoGtFPS/YH/2uRNFRx8Dko/SWiSVhY4gzIs9jWUJtBEdbx6NRdY6ThlQLXyl1Jx4mjY3GArpn1oK9qs0DxWtzxMNTAU6G8HNdqp0LZ24ws0Obuf3TCL+RwF2ixwPhkYMKU+UBtjY+NmdvaV/dclQFrIMTHxzL6+mP/kKyQdjVtYO+RjP9poYmLC0Gb5hI8Ks/pan/nR/jbzye4WPVJmA9qF9olqwzj3+W9g/muTs4DOvRYnraCwAlxJ2z1ngVMiowIq+WDKzwDQ//iJHim2AW1EW1Wyr2nZy8OKly9f2vcKudAUsHGNdw4RJt/2EfYFnYMAmgVOFIAeagP1AfWBNPsAr8p49uxZltIUaMoZeBKGsHHrIrD1IeoCNAucsjMl6wV6frtuonZQO6gPpMMHeDEbr8QAiALJsDNhCEucXO0n3OPsQxaIugAVeGaBE0rroTZQH1AfSLMPALcwWPrXCRu3LkDWhagLz0Bw+pnq92SfV1R7qj3VByrDB1CZQFcA6sMzS3Fqo1ZGo2o7aTupD5TGB4LgiepUcMb8qZU6bGkcVu2sdk6DD6A8mboDUA5Z71RwKjjzXi9KgyNrGRSopfYB1kgFnjJlV3AqOBWc6gPqAzl8gOdBgSfrnYCTQ8GZw2ClHtk0P1VT6gPp8wEea/JVp4JTwalqQ31AfSCHDwBJBWcOA+lon77RXttE26TcPgA43ek6m0SqOBWkqjbUB9QHcvgAvzyS6bo816ngzGGwco90mr+qLfWB8vsA//PpT9cVnApOVRvqA+oDOXyAv64DlEzX5bEkBWcOg+loX/7RXttA26DcPnDv3j1z//598/DhQ/Po0SPT39+va5zlbhTNX8GgPpBuH0Bx+uucqjhVceo0TX1AfSCHD/CnyKxzuhtECs4cBlMlkG4loO2j7VMKH1BwKiRVWagPqA/E9AHAKRtE8iC8Ks6YRizFCKd5qJJSH0iPD/AKjqKA8+nTp+bcuXPm6NGjOY9r167l9Zf36jTpcRptC22Lj90HigbOnp4eMzY2FjkFyDfcYhqKPxdl8fbx48f2JU38b95i0klzHOoob+4LKqf/ry0ynSAOcYPi+Gkysg4MDISG99MQu5N30D3yLmZb+OX3y5DUd57f40gqPU2ncgaksoNzeHjYbusn7TQ8KnDgwAHz2Wefmc2bN5uVK1eadevW2Weuks6rnOnxG9mDBw+avr6+wA7M9SVLlnxwEIe4QWX302xqajJ79uyxf2pAeN57TrsFxeUacN66das5derUB4Dk3t69e83IyEhofD/dqPz88H75/ftJfT9z5ozhSCo9TacywSkPwYso4fzJYhvTVZKlnrajAg4fPmwaGhoyigAV0tXVZTZt2mRGR0erxtmjIAE4gRiNmW9bRqXZ1taWExjkRZ6ffvqpuX37dla+3IsLzqj8/HpFld8Pv9jvCs7KAd1i2zgsnqs4iwZOF6JhBeF6vuFypcE9AFlXV5dRSBIeeF68eNFcunQp05mBbGtrq9m3b5+FAepGwjNFbWlpsQqLeIQhrEzPBgcHDZ2ajipx2GG7evWqjSPX3DNxr1+/bst38uRJq4ApF2H4BQJlJ11ZG/anyOR19+5dGx9Fh3KLUpxR4IxKk3K1t7fbP2ulfNhh27Zt1o7YyK0fnwWc58+f/2CgCgJnrjYIyw+b8csNBkgOPosdBZxc6+zsDGxbypkrX+6THvannfAnBgG3rX1w0pm+//77qhqY/bbV728Hi6oDJ86O0gROYY1MGA6m86gfNrF6e3stHDZs2JCZ9gKlLVu22I4HOPiZ1bFjx2xHpdMB2Z07d9r3NEtehGGJgPtyTc6st+7evdumIZBEAcs0GwgDJDok6TQ3N5u1a9faNVrSYF0QGNXW1lp40pHp0CxFSBqSl5yjFGc+aVIumdozRce+AmsGCslLzgJO8mYQAWxiDx+cUW0QlB9lZiBz7bBjxw47yNGuAk4A39jYaNsWW2JrWWKIype63Lx507YHdpa2P3LkSAaeLjh5PIXlDPyIMogt9FydqrTqwCmdJgwkriMDBJSdqyJQvQI+wPnVV19ZJSzxAAWwoiPRgVEjpMN9vp8+fdp2YAnvnrk/Pj6elR8dms5NONIBBkh/vtMBz549m1HIdHoAQSeVdFGnq1atyglO7gM7oCsH4CaNfNJ0wSnlBBpSBv/sghNgAtkbN27Y+vjgjGqDoPywIQOeaweuYTsUsPiAu8aKLYE47UM7ROVLObE1v0GW+rltzzUBp0Dzzp07Cs2P5FG+qgUn0zRx+KCzQA8l6d5HRaIKOQNOFKC/JgrsZLoPQA8dOmQ7K52WzksndtN0P9OBCYcyuXLliu2cAiEfUMTjmtxnio7idXer6cwoq7CBguvr16+3SwDkKYcsSeSTpl8ut0xu3eSzC06uAeft27fbswvOfNrAtwHfKTN2xvZSH6b0NTU1dplDwIlSlDJx5ukK4lEGBrxcbc+AhB8wS3DToN2l7WkXBlEAS1qqNKtTXbrtL5+rDpxUDAUH3KSS7plOhaLjz0dlbcy9zxSODgM0OXbt2mVB54ZxwUEnJDxwkOmcq2DdePy2lbBABOWHuqUTChh9QBHXzYvOSYcHOJIu9fj2229zgjPXGmc+afrlcssk5XDPPjgBCooTyIhaxLbYKaoNfBvwnTIzoIl6ds8McgJOf/BkkxL7oxCj8kWRo5Sxr1s3loCkvTizlHL8+HG7ZEP7umH1c/WCtCrBSYdh7dHfuAA4TN8Eqq56ECd3lQade+PGjVbFyH1fJQEF1ttIE6ihhiSsf0YB+UsDxJOO6AOK+C6k6Mx0fJnKc586uuukfp4ozlzgzCdNv1xumfz8+O6Dk2vAjPXB+vp6Wwdsy/WoNiCMnx9l3r9//webf4SVvICetLNcp20AJmWJytedeUh8v+1pNw6uA2/SlrVciaPn6oRn0cDJyM+mC+oAYJTyYXhxZKZl5AvcmNICOIAqygBIsvkzNDRkOxxhcP7Lly/bOHRuFAXXuEc6TA2JI1NdOgZTQKbDKEkf1m7HAWJ0eAEfaRAnX3AybWQDApBQFgBAh/38888XrTjzSTMInN99953N362ffA4CJ/eoL5tv2FTAGdUGxCN/Nz/aAvUqdpB2YVAhb1GcDHrStqhM2l6m71H54kMMssxeSE/ycNtewEkZZaYia7liCz0rOEOVVJRz5PuYUb7hovLjPs4OAJcvX24f/uaZQjobI4XEl87Ag/ErVqwwS5cuzXQUwtC56SisRa5Zs8ZuwgQ9RE9HZp2RjRzSlPT9M2qEzkg+bDrR0QFfvuAkPQYkYEt5eaifx6Vkh9vPj+/AOugBeFeFRqXpg1OWHKgHg4afbxg4CUcbU3cBZ1QbECcoP9qRQYj2xRZAEiVKeAEnu+LYmLajrLQjQCRMPvlSjxMnTti4y5Yts2uoT548ydTXBSdpslyD+mdwzeUHhNWjsm1QNMXpOka+QMw3nJt21GccGFXlbqj4ccLCCDg50+FQikEdQnaO3R1YPw/3Ox07LC03XNhnykB80gkLE/d6MdKMU4awNohKA9vnsqXUK6z988m30PaKqoPerzyIlgScMm1nfS/XgXpCvaXFkVxwBpUJmKJKOjo6rOrT9a3K6wBB7arXtB2jfKAk4IwqRFrvsz7KjnXY+iyQZyDgcSQMmdZ6aLkUBOoDyfqAglPXmxT46gPqAzF9QMEZ02A6cic7cqs91Z6V6AMKTgWnqg31AfWBmD6g4IxpsEocHbXMqurUB5L1AQWnglPVhvqA+kBMH1BwxjSYjtzJjtxqT7VnJfqAglPBqWpDfUB9IKYPKDhjGqwSR0cts6o69YFkfUDBqeBUtaE+oD4Q0wcUnDENpiN3siO32lPtWYk+oOBUcKraUB9QH4jpAwrOmAarxNFRy6yqTn0gWR9QcCo4VW2oD6gPxPQBBWdMg+nI/fYf7+Utn1H2wMH4I+Qk/zs0Kk+9n6y6Unt+aM+igZM/JeYv1/hLNvmc67845V7YX7jl23j8YS1/XMwf1Ppx5E9t6cQc/Dt6qTs0/9k5MDBgX1oW9ue6frnjfOfVHaQfVP846YSFxbb847z/Jk/syOsoyN/Nm/8s5ZW8gDYsTb3+YcdUm6TbJlUHTjour2ulE/vOxztveDsirzgALrz7hrMfrhjfASavzeD9QLzm4csvv7SvvuBPkF3QFJp3U1OTfS9Rsf4Qmnf2uG/apOzUgdd4bN682daLVxi7f+rMAMW1Ug9ShdpS46cbXuVsn6KBs1yVEoXjv+GQ8sg/tZe6A1Mm3i2EUuOlXpQF4PCqjVxvqCyXDcPyRSEDQHnhGeHkX/JloAKYvNzu6tWrmQGBwYzXLBM2LG29rpCqJB8oGjhlel7qqTrGB0i84ZJ30UhjAEuWA4An1+jMwJUz3wEZ6hM1xUvdgIMAlulpa2urfU2GpNfV1WVu3bqVefkXr9Dg1R8CRgnHmXR5wRpvWnSv85nyACOZtlMOXm8MfDj4zDXCAiXyYJrMee/evQaF6ebJC8t4s2g+6UmahCct6i4A9MvJd+zAy8/cMNiQqTiDg8Tx7Y8dGTRyvTpZ4upZAVoJPlCV4KSD8xpd9+VpwIbXw8prfUUpiQrijYhM4wEmHZw3KAoQ6Pi8HoMpJ40qb7UkPMDkGvfC3vV9/fp109DQkAGg7xiAkQP48ApjoE8ZOPh84cIFe4+8UG5cA1i8TZFXMLuvPHbfSOmnR91YxgC65Ee9eO0u4AaGDAaylOGXke/YivhiQ1GggNcNL6/KFdtyjzdCUjY3nH5WSFaqDxQNnOU2yKVLlyyERK3RaQWElM0FJ6AFeq6SkmushxIe2Mjrf4mLgiKOvB6X/IKWB4ibLzRIF2XqbrygUoEV5QCcvNbXnQYDR9ZOKR95ueAkHZSkq3S5BnhR2qSHgnz69GkGaIQVtUp67kH5SE8GC1GSMqBIWAG8a898bSBp6Dnb9mqPdNmjaOAs51QdJwM0QIFOzBT3wIEDWTu7Ljjp4MCpu7vbqjiUHOXnmkwvCQMomf6j3IAksOKMAiV9gajv5EAD1elf97+TF9NzgCT3RNWRJ3VBEcr7wyUM8WS674JTFCs72tSJA1VZU1NjBwlRnMCYJQNgKgONpO2esRk2laWBXOD0gazgTFfHd9tVP8dvm6oFp8ASNeRCVJzEBScgYqcbRckmjnsIpACmKMz6+noLSWCK2uPMuigqVdJ3z0xl3Z1o9x5gJB7AIpw/pec61wBikJIjLeqIAgZkLjhJD9C69ZHPvLKZuMQBqJSPnX6ANzU1FVgPwOpu8qB2iUeebp0IRzqSh8Cf8rjh9HP8Dqs2S4fNigbONDQwKorpOaqQtUNXTbngpIPT0enwucrNdJw1RV4ZDMSA6ZEjR+wGjUzjg+JLXjLtlzCUh2m3QBXFyjTa3dRCzQJsAE6eqENRwZIO9WtsbLRld8FJHOLmejSJMohdRIFST0nbPc/Nzdm6s2El18mPTTfiyjUULvaUenCmXmGKXOLpOR1Q0HaIboeigbPcU3Uan/U8pqXr1q37oNMKzDjLOiHwEQAMDQ3ZqTr3xZFQdkuXLrUqkzgAB5ByLdcD3oS7ceOGVX/Ag+/kA3R4HEmACuBQrlwnDAef2ehClQo4WUKQdUvKSRoCJcKL+gxKD3UJ1EiLurmbQYRnUCANqbN/BtIuWCkH67IsAVBevrNZ5apLFPm+fftCFbmfh36P7rhqo/LaqKrBKVNEYMTU3XU2F5xcByQnTpywEFy1apVZs2aN6ezstDCQeIQBNKw3yjXUH+CKUquAFpiQ7pIlS+wBYPwH8GkQVCIw5gBysost4AReLC2QFg+eo6yBFmVywcl3SW/58uVmxYoVNp4sPxCHuKtXrzbr16+3+bH04NtK6soZu/nLEqRHeajXZ599Zq5cuZL1eBJrwbJ55aaln8vb+dX+i7d/0cBZqY2CEpQ1x2LUAVgxdRVlG5YH8PIBJuBE+Uo6ADksDfc6aZGvANa9RxrUmYHGvR70mbAse9y5cycrLdJFsfrlYUBBAYtCDkpTry2+A6vtymM7Baf3yE2aHdEFZ5rLqWUrT2dWu5fO7grOCgInqpA1SFnP1I5Suo6itlZbuz6g4KwgcLoNp5+1I6sPlM8HFJwKzsh1Te2g5eugavt02l7BqeBUcKoPqA/E9AEFZ0yDqQJIpwLQdtF2KaUPKDgVnKo21AfUB2L6gIIzpsFKOappXqqi1AfS6QOR4Jydnzd6qA3UB9QH1Afe+4CCUwcGHRjVB9QHYvqAgjOmwXTUfT/qqi3UFh+rDyg4FZyqNtQH1Adi+oCCM6bBPtYRVuut6lJ94L0PKDgVnKo21AfUB2L6gIIzpsF01H0/6qot1BYfqw8oOBWcqjbUB9QHYvpAScA5NjFh2trbTf3JU+bgocPmXON503Wnx0zOzBS1waZfvDR9j/pN/8CAmZmdjZUXcZMu35PBITMwNByrHEmO6Nhg6OlTa5Ok65ZkOaPSirIjbdfT22s4R6Wl91U1L8YHigrOl6/mTM/de+bIsWMWmEDTPc6cPWc78mIKnivOi9lX5uLly2bFr35lNm7abNav32B+zSsmbneZ2bn8HOVqS4s5eep0Yh0PaB09dtwefMY2QH382fPQPPIJk8sOmXtz83bgwh6r16yx9uAVF9/Vn0h8cMjkGXMED4s383LW3O97aCan3w6y+dgRu65avcbaNyxdvZ5fP1A7BdupqODE4cOgKQBFfaJIk2ygru47Zts335iR0bFMur19faZm69a8QZ00OP36oYbqDhywysi/J9/zCSNhc53v9Nw1G77eaLCBDBzjE8/M/ro6cytuOrkAAAq+SURBVOZco4V4rvjlvDfxfNLs3L3boDKDypGUjYLS1mvB0FC7zNt3efFaGN7KwCtpeDUNn+X4ZLFGwuGBogDSP1+4eNEAJ6633Wg3qMTF5uXHa2pu/kAt+urtZkeH4ZC4KJmW1uuZqbSA82H/Y1vGA98eNABIwMNU9/vmZjMwPGzV7e69e82N9pt2eni7q9vsq6szZxsbDYCSPCRP4lL/db9Zbw4ePGTzJX8JxzlXGO5duXrVkCeqeHhkJCuum87zqWmza/du09nV9UEYpu0MMJn4c/Omr7/fHPuu3uytrTXtN28ZFJ+kR/mZArPMQt7Uj3YGaqhX2pL4YqN79x9YG4fZ0KabI8/R8QnTcOasWbP2S6vUpb2i7Ei8S01NhnNUHtwnvbu99+3siHpTl4xNElLOYkM9VweMi6Y4Hzx8FArNpitX7NQLmNE5UT25pqxxnQ3FWVOzxa5tSif20wA47lTcVy6AkzS+PXjQAELKyfdrLa0WDABj+44dFpCs33J89dU6qyIvNzUZynD48BH7nbTJX/IERkB42/btFrpMLbGFW8awMIAYBXaq4Yzp7umx0AXAAM2NL58fDwyaHTt3ZgFc7tkzSxfvli9a29pMzZYttq5SfsDulp+0WAbhfu3+/Wbrtm22Xnyn3pRF1GGUDcnfzRM776mtNfUnTtqBlOl5R+dtu9xCWqxVEyfKjuS/aXNNphxuHqH12rXL+iE2xbbrN3yt8NRBI7BP4YNFAyfriSgQHJ+OR6fgOypN1qtGx8YzqnRwOLlNE9Qr6pE1PToAa6m2Mzvrm9L5LDzm520Z3akzHfXrjRuzpvsoJxQaSgZwbq6psfWSNJqvXbOwFPWIGv1mxw5DPd0Oz2cf1JKGew4KQ7mwo6sEUZNADCXqxuczQKVeAj//vnxn4AKCvQ/6MmmgVlFgwIZw2Aw1KpAHyix/iDojD5Q5bU34SBtOTlpQYlcpx8S7a/1P3kISO/tTdbftgmzkgjPferH+LLMeSZPBUsql5+pQikm1Y9HBadfVeOXu7Cu7xhYETUCQJDjFOOSJSgGcbIq4myFu5yO8dBZRbnR6FKNAkDDPJqfstJe1WwGnhOc+cUhX8vc7vZunn5/Ecc9+GIAFuFqut2XyIDzgAtACMDcNyseywdTMi6w4bhg+Ay/UJOBy7zWev2A4uOaWn+8ACqhRT4lDGOzAd86AlHrIfdofyGND8gTWtzo6rXpG7TEIoMQFvr4NSccth28j7rvgXEy9JA+ph5RdzwpP8YGigVOm6jyCZNe9HNnvKk2gmfRUXSrnnlGJdFJXPdEBJYzfAek0lF2msYQDPkAIGJUDnJQxs9bq2JPpO9Czqtq5Tpl91Sv1tee5eQtKBgfWI1GmPmBR0WInF1jEzwecrkIljmtn8uSJh1OnG+zgxgAnB/cIXyg4F1Mv8qWuCk4FJb4QdBQNnDi8bA658PShCTiT3BwCAkzTfVhzHQUpao31OAEChnGhyHc6DRsgopC5RtlRdsCoHOCkDK4C5DtHmKrinoCqGRXoLFVIPAYT6hWkWn2Fuxhw5rJhTqi/q1uh4FxMvbCNgjMYGNhGjyKucWJc93Ek4MkGCooCWMpRjMeR2FVn7U2m/0zZ2291ZG1coDzZ7KHzAhfifP7FF5lNFsDJdwEOa4qnz5wxh48es+uLSYHT7hR7QBPHFOi5YYAkGx+PHj+xDgzYUaGNFy5+AEZJh+UKHkeiLqQJQGkbd7MLGx2vr7d1tOunc/N295y8ZAlgMeD8+S9+EWrDD/Kcn7f1AuainrEzm3CUV+rjliPIRu5U/YM88qgX+Sg4FZDib0HnoilOMkOxlOMBeDoTyux/fvlL8x8//Zk9mBLa9ch3kGIjhTVPOjbhACfPNcqaJeDkPssIbDItW7bMrs3J40WFghP7sLNO2qhY1k+DGuiDMO86/tovvzLLli+3cAfoFog51AA792xsiT1WrlptlTlgkXyp07Hj39k0v1i61GzcuMmw5CL3XWBxLZ+pOptL7JIH2ZA03Dx/tXKloVw81pVRx3Pztpy0kaw5++XwbeSC088jn3oRR8Gp4MQPwo6iglMyLddPLoECmx1Bu81StnzOQKnQNPLJJ1YYZ30yTjzqYiEdonJJK58w+eTJ4CM7+lE25L7dmMpRrnzyzBUmqXrlykPvhcOmmmxTEnBWk8G0Lvl3DBecarf87aa2Sr+tFJw55Lg6cGEOzHOQLHeg9NSWhdlS7Zcu+yk4FZwKNfUB9YGYPqDgjGkwHfnTNfJre2h7lMMHFJwKTlUb6gPqAzF9QMEZ02DlGN00T1VV6gPp8gEFp4JT1Yb6gPpATB9QcMY0mI786Rr5tT20PcrhA5Hg/OGHH4weagP1AfUB9YH3PqDg1IFBB0b1AfWBmD6g4IxpMB1134+6agu1xcfqAwpOBaeqDfUB9YGYPqDgjGmwj3WE1XqrulQfeO8DCk4Fp6oN9QH1gZg+oOCMaTAddd+PumoLtcXH6gMlBefdu3fN0aNHTX19vZmYmNBRTqGtPqA+UJE+UFJw9vT0WHACz7GxsaIZbH5+3szMzGSOly9fmjdv3hQtv3xGXfJ/8eKFef36ddHKsbCwYEpdV+pT6jzzsbeGUTVcTB+oSnC2tbWZJUuWZB1r1qwx7e3tBrgU06BhaQPyrVu3mr6+vqLlPzIyYvbu3WsHjLByJH2d+hw8eNAwWCWdtqan8EurD1QtON3OjNobHR0127dvN5cvXy6L+lRwKgTSCgEtV3zfLDo4AcbFixczU3Sm6e7R0dGRuApEcbrgFMcYHh628BwfH8+oo+npadPU1GSVGme+E55pdWtra5Z66+rqMrdu3cqUl7q18ObI6WkzOTlpP7N2S3337dtnqJsosSBwhuUt5SWt8+fP27JdunTJ5iH3OJNmc3OzvU85BgYGcipOBhDCnDx50tTV1Znbt29nykd6UfcJI2Wifp2dneb+/ftZts4nDbcO+jl+p1Wbld9mRQcnjYzaa2hoyAIm8Lx27ZpdH0vaEcLAyXrcsWPHLDCkXDU1NRY+vb299rxp0yZbXoB36NChzNR6dnbWwnDbtm0ZmDJN3b9/v+Ee0+QtW7aYw4cP2/SBEt9F4frgxCZheVO2x48fm7Vr19oyATvAuXv37gzYp6amzM6dO825c+eMlL22ttbs2rUrUz7frjdv3jQbNmywSxb37t2zwDt9+nRmIOA+9aPs3MdWR44cycCVgQf7AGvybGxstDZxByk3DTYDsY+bh18m/V5+CGgbxG+DkoCThvHhWSxoklcYOLl35swZex9ldPbsWasO+cw9zlevXs10dFQcYbgOGAEEIABqhAdmQITP3N+4caNVdHznAKzAjs0TF5z55P3q1Su7gSZlA87k/ejRI5s2ZTt16lQGelJ21lHJS8ogZxT0nj17TH9/f+Ye4UQx83nHjh1Z98kTZQpEWRsGgNhHysQ1yiDgJA/KODg4mMlDrgFdKYue43dUtVm6bFYycNLwAs9iQpN88gGnDyJxTKCIcgN2AAAQ8BkVBiQBDWfiHzhwIANRUZycJS0+y2aNC8588iYNFDLAYYkAZYkCBcYAi+k2ZZK8OFPeMMXp1sWNI5+5D+QBnVzjzODA4ddXwqAqBZykAXy7u7utIkWV8iQF1wgncfScLghoe8Rvjyhw/j8kanSBUlkzGgAAAABJRU5ErkJggg=="}}},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"import riiideducation\nenv = riiideducation.make_env()\niter_test = env.iter_test()\n\nfor (sample_test,sample_prediction_df) in iter_test:\n    sample_test = sample_test[sample_test['content_type_id'] == 0]\n    # 这里做你的处理\n    res = model.predict(sample_test)\n    sample_prediction_df.answered_correctly = res\n    env.predict(sample_prediction_df)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}