{"cells":[{"metadata":{},"cell_type":"markdown","source":"**These are some Q&As present in the https://www.kaggle.com/yanamal/questions-tags-and-lectures-riiid-metadata-eda/comments \nwitch i found outstanding and i wanted to chip in with some spare time.**\n"},{"metadata":{},"cell_type":"markdown","source":"Here are the questions i hope to adress:"},{"metadata":{},"cell_type":"markdown","source":"* -Diagnostic questions: According to the dataset description, each new user is asked a series of diagnostic questions to determine their current level.\n    * Which questions or tags get asked the most in the diagnostic part?\n    * -Is there a separate set of diagnostic questions, or are all questions used for both diagnosis and training?\n    * -Is there a deterministic decision tree of what diagnostic question gets asked next, given the answers so far?\n* -Do different users answer roughly the same sequences of questions? what are common subsequences of questions?"},{"metadata":{},"cell_type":"markdown","source":"* Which questions or tags get asked the most in the diagnostic part?"},{"metadata":{},"cell_type":"markdown","source":"In this notbook I refer to assestment questions as those that are the first n questions posed to the user to determin his level"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time\n#get Data\nimport pandas as pd\nimport numpy as np\ndtypes = {\n    \"row_id\": \"uint32\",\n    \"timestamp\": \"float32\",\n    \"user_id\": \"int32\",\n    \"content_id\": \"int16\",\n    \"content_type_id\": \"boolean\",\n    \"task_container_id\": \"int16\",\n    \"user_answer\": \"int8\",\n    \"answered_correctly\": \"int8\",\n    \"prior_question_elapsed_time\": \"float16\", \n    \"prior_question_had_explanation\": \"boolean\"\n}\n\ndata = pd.read_csv(\"../input/riiid-test-answer-prediction/train.csv\", dtype=dtypes)\ndata.drop(columns=['timestamp'],inplace=True)\n#subset\na=data.loc[:int(20e6)]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First question is 7900 38% of the time. 0 is to be Interpreted as chance it being any of the less frequent questions(not in the top 20 frequent qs). "},{"metadata":{"trusted":true,"_kg_hide-input":false},"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nsns.set()\n# data = deliveries.groupby(\"type\")[\"del_tip\"].sum()\n\nFirst_1_q=a.groupby('user_id').head(1).groupby('content_id').answered_correctly.count().sort_values(ascending=False)\nsf= First_1_q.head(20)\nsf=sf.append(pd.Series((max(0,First_1_q.sum()-sf.sum()))))\n\nfig= plt.figure(figsize=(10,10))\nsf.plot.pie(autopct=\"%.1f%%\")\nplt.title(\"First question is\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"First and second question "},{"metadata":{},"cell_type":"markdown","source":"Reading it clockwise gives an idea of the progression and how much variotion is at play ( order is not accounted for in a single chart but if seen in progression ..)"},{"metadata":{},"cell_type":"markdown","source":"second question is often  7876 sometimes .. 7860"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n%matplotlib inline\nsns.set()\n# data = deliveries.groupby(\"type\")[\"del_tip\"].sum()\n\nFirst_2_q=a.groupby('user_id').head(2).groupby('content_id').answered_correctly.count().sort_values(ascending=False)\nfig= plt.figure(figsize=(10,10))\nsf= First_2_q.head(10)\nsf=sf.append(pd.Series((max(0,First_2_q.sum()-sf.sum()))))\nsf.plot.pie(autopct=\"%.1f%%\")\nplt.title(\"First two questions are composed by \")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Already a majority of the chart is taken up but questions that individually would have less than the smallest part of the pie. I call them the umpredicatbles."},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"First_3_q=a.groupby('user_id').head(3).groupby('content_id').answered_correctly.count().sort_values(ascending=False)\nsf= First_3_q.head(20)\nsf=sf.append(pd.Series((max(0,First_3_q.sum()-sf.sum()))))\nfig= plt.figure(figsize=(10,10))\nsf.plot.pie(autopct=\"%.1f%%\")\nplt.title(\"First 3 questions are composed by \")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The ratio beetween the first most probable question for question number n and its second most probable seems constant thougout. "},{"metadata":{"trusted":true},"cell_type":"code","source":"First_3_q=a.groupby('user_id').head(4).groupby('content_id').answered_correctly.count().sort_values(ascending=False)\n\nsf= First_3_q.head(20)\nsf=sf.append(pd.Series((max(0,First_3_q.sum()-sf.sum()))))\nfig= plt.figure(figsize=(10,10))\nsf.plot.pie(autopct=\"%.1f%%\")\nplt.title(\"First 4 questions are composed by \")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"and then caos"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"a=data.loc[:int(20e6)]\nFirst_5_q=a.groupby('user_id').head(5).groupby('content_id').answered_correctly.count().sort_values(ascending=False)\nsf= First_5_q.head(20)\nsf=sf.append(pd.Series((max(0,First_5_q.sum()-sf.sum()))))\nfig= plt.figure(figsize=(10,10))\nsf.plot.pie(autopct=\"%.1f%%\")\nplt.title(\"First 5 questions are composed by \")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# -So Yes some questions are used much more than others in Assesment !\n\n"},{"metadata":{},"cell_type":"markdown","source":"#  -Are they only used in asssestment  ?"},{"metadata":{},"cell_type":"markdown","source":"-turns out no \n"},{"metadata":{},"cell_type":"markdown","source":"Here is of times the poplar firsst questions  are used in assestment over their total usage \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"from seaborn import barplot \nm= []\nFirst_10_q=a.groupby('user_id').head(10).groupby('content_id').answered_correctly.count().sort_values(ascending=False).head(10)\nfor c in range(First_10_q.shape[0]):\n    totusage=a[a['content_id']==First_10_q.index[c]]['answered_correctly'].count()\n    m.append((First_10_q.iloc[c])*100/totusage)\nfig= plt.figure(figsize=(25,3))\nplt.ylim(0, 100)\nplt.title('% of times the question is used for assessment purposes(first 10 questions) ')\nsns.barplot(x=First_10_q.index.to_list(), y=m)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from seaborn import barplot \nm= []\nFirst_10_q=a.groupby('user_id').head(30).groupby('content_id').answered_correctly.count().sort_values(ascending=False).head(30)\nfor c in range(First_10_q.shape[0]):\n    totusage=a[a['content_id']==First_10_q.index[c]]['answered_correctly'].count()\n    m.append((First_10_q.iloc[c])*100/totusage)\nfig= plt.figure(figsize=(25,3))\nplt.ylim(0, 100)\nplt.title('% of times the question is used for assessment purposes(first30 questions) ')\nsns.barplot(x=First_10_q.index.to_list(), y=m)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from seaborn import barplot \nm= []\nFirst_10_q=a.groupby('user_id').head(40).groupby('content_id').answered_correctly.count().sort_values(ascending=False).head(40)\nfor c in range(First_10_q.shape[0]):\n    totusage=a[a['content_id']==First_10_q.index[c]]['answered_correctly'].count()\n    m.append((First_10_q.iloc[c])*100/totusage)\nfig= plt.figure(figsize=(25,3))\nplt.ylim(0, 100)\nplt.title('% of times the question is used for assessment purposes(first40 questions) ')\nsns.barplot(x=First_10_q.index.to_list(), y=m)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The First 30 most poplar questions seem assessment specific, afterthat the questions get usred all about !"},{"metadata":{},"cell_type":"markdown","source":"**-Is there a deterministic decision tree of what diagnostic question gets asked next, given the answers so far?**"},{"metadata":{},"cell_type":"markdown","source":"The following graph show that the question that you get next is highly determined by the answer you give. But not totally, in the below case (99% times it is 7876 if you answer 7900 with ans 2)\nat times it is a unpredictable question that is posed"},{"metadata":{"trusted":true},"cell_type":"code","source":"#second question answers and third question n of observations\na.loc[:,'next_content_id']=np.int16(a.groupby('user_id')['content_id'].shift(-1).values)\nprint('Answering Question 7900 with ans 2 the next contet id is - with occurences - :',\n      a.groupby('user_id').head(2).groupby(['content_id','user_answer','next_content_id'])['answered_correctly'].count()[7900][2],\n      'out of total ',a.groupby('user_id').head(2).groupby(['content_id','user_answer','next_content_id'])['answered_correctly'].count()[7900][2].sum())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The Most Frequent cause and effect are below."},{"metadata":{"trusted":true},"cell_type":"code","source":"a.loc[:,'next_content_id']=np.int16(a.groupby('user_id')['content_id'].shift(-1).values)\na.groupby('user_id').head(2).groupby(['content_id','user_answer','next_content_id'])['answered_correctly'].describe().sort_values(by='count', axis=0,ascending=False).head(30)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Average user answers about 250 questions"},{"metadata":{"trusted":true},"cell_type":"code","source":"a.groupby('user_id')['content_id'].count().mean()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Take aways :\n* the questions posed next are correlated to the answer given. Not in a deterministic Fashion though. Just about 99% if the time\n* Certain questions turn up much more than others in and neerly only in assestment realted activities.(first 30 Questions) -they seem nearly assestment specific\n* How different each persons first 30 questions are changes a lot and quicly. Starting from the very first question.\n-later found this confirming it \n* Average user answers about 250 questions and these \"assestment specific questions\" seem to be at max 30\n* Patterns given the variability of the Users answers, the deterministic and none deterministic factors at play would be probaly very small ( only a couple odf qs to have a high frquency)\n    or possibly task contained.\n    \n"},{"metadata":{},"cell_type":"markdown","source":"Thirty Question Analyses turns out confimed here \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F835774%2F42f22abccf1a0450443564a3aac831fa%2FScreenshot%202020-10-31%20at%2010.21.27%20AM.png?generation=1604119908011993&alt=media)\nA Thank you to Aditya Soni for sharing this knowledge \n"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}