which makes total sense when you think about it, but what if we could only make 3 or 4 different models to get the effiency price ?
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2Fde6b8ea924c3451b4047cefd5899de9c%2FScreenshot_12.jpg?generation=1678427940097359&alt=media\")
\nAs you can see here, they are question who have similar success rates, so why not make models to predict batches of questions and have a 2 minutes infer ??
\nTake question 14 and 16 for example, and you will understand why even if the success rates are the same, you can't deal will them the same way.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2Ff1113cd09ba9790f969e0c2211bf24b7%2FScreenshot_14.jpg?generation=1678428362913194&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2Fca2a5d4051339fefd4c1ffc6b25a8080%2FScreenshot_13.jpg?generation=1678428140524685&alt=media\")
Suprising spike no ? My opinion on the matter is that question 13 difficulty caught them offguard, meaning that the students who got it wrong (from accuracy 0.3 to 0.5 most likely), where paying more attention, we can see that the relationship with previous accuracy isn't the same for both question, indicating that we can't deal with them the same way 👍
\nNote:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2Fba07eb4add17738263e593015b2b6859%2FScreenshot_15.jpg?generation=1678428678968040&alt=media\")
Notice for question 13 (the hardest question) that it's not just a straight line but lower than for the other questions, we can clearly see here that the average student who was paying medium attention didn't get that one right, and only the ones that where already nailing the rest got this one right. I don't really know what to do with that information, but i suggest you observe the behavior of the students that got question 13 right, might give you feature ideas !
", "messages": [ { "id": 2175744, "postDate": "2023-03-10T06:18:41.273Z", "content": "which makes total sense when you think about it, but what if we could only make 3 or 4 different models to get the effiency price ?
\n
\nAs you can see here, they are question who have similar success rates, so why not make models to predict batches of questions and have a 2 minutes infer ??
\nTake question 14 and 16 for example, and you will understand why even if the success rates are the same, you can't deal will them the same way.
\n
Suprising spike no ? My opinion on the matter is that question 13 difficulty caught them offguard, meaning that the students who got it wrong (from accuracy 0.3 to 0.5 most likely), where paying more attention, we can see that the relationship with previous accuracy isn't the same for both question, indicating that we can't deal with them the same way 👍
\nNote:
\n
Notice for question 13 (the hardest question) that it's not just a straight line but lower than for the other questions, we can clearly see here that the average student who was paying medium attention didn't get that one right, and only the ones that where already nailing the rest got this one right. I don't really know what to do with that information, but i suggest you observe the behavior of the students that got question 13 right, might give you feature ideas !
", "rawMarkdown": "### Most of people's intuition was that it was important to train 18 different models , \nwhich makes total sense when you think about it, but what if we could only **make 3 or 4 different models** to get the effiency price ? \n\nAs you can see here, they are question who have similar success rates, so why not **make models to predict batches of questions and have a 2 minutes infer ??**\nTake *question 14 and 16* for example, and you will understand why even if the success rates are the same, **you can't deal will them the same way.**\n\n\n\n\nSuprising spike no ? My opinion on the matter is that **question 13 difficulty caught them offguard**, meaning that the students who got it wrong (from accuracy 0.3 to 0.5 most likely), where paying more attention, we can see that the relationship with previous accuracy isn't the same for both question, indicating that we can't deal with them the same way 👍\n\n*Note:*\n\n\n\nNotice for question 13 (the hardest question) that it's not just a *straight line but lower* than for the other questions, we can clearly see here that the average student who was paying medium attention didn't get that one right, and only the ones that where already nailing the rest got this one right. I don't really know what to do with that information, but i suggest you observe the behavior of the students that got question 13 right, might give you feature ideas ! \n\n\n", "votes": 12 }, { "id": 2199735, "postDate": "2023-03-28T01:20:51.660Z", "content": "get the \"accuracy before\" feature:
\n i (,):\n \n column_name = + (i)\n df_final[column_name] = df_final.loc[:, [+(j) j (, i+)]].mean(axis=)\nplots :
\nfig, axes = plt.subplots(, , figsize=(, ))\n i, ax (axes.ravel()):\n question_gb = df_final.groupby()[].mean()\n ax.plot(question_gb.index, question_gb)\n ax.set_xlabel()\n ax.set_ylabel()\n ax.set_title()\nThis assumes you have a DataFrame named df_final that has for a row a session, and for columns the session_id, and their answers named q_i for the ith question like this:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F12870466%2F726b44dcdbd1d36fd17c47eff05aceee%2FScreenshot_16.jpg?generation=1678429701447528&alt=media\")
\ncheck this notebook to find the code to do that if you don't want to bother.
I only used 1 model for all questions, it has the same score with 18 different models :D
", "rawMarkdown": "I only used 1 model for all questions, it has the same score with 18 different models :D ", "replies": [ { "id": 2176830, "postDate": "2023-03-11T00:17:25.560Z", "content": "That's surprising, that means you make the same prediction for each question and still get a good score ?
\nHow do you deal with the difference between say question 13 and question 2 ? Both can't be labeled the same for each question that's very unlikely..
I add the column 'question' in training set when training the model :D
", "rawMarkdown": "I add the column 'question' in training set when training the model :D", "votes": 1 }, { "id": 2177288, "postDate": "2023-03-11T10:57:06.893Z", "content": "Didn't even considered it ! smart idea for effiency
", "rawMarkdown": "Didn't even considered it ! smart idea for effiency" } ] } ] }, { "id": 2175753, "postDate": "2023-03-10T06:28:33.303Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2199735, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-28T01:20:51.660000", "content": "get the \"accuracy before\" feature:
\n i (,):\n \n column_name = + (i)\n df_final[column_name] = df_final.loc[:, [+(j) j (, i+)]].mean(axis=)\nplots :
\nfig, axes = plt.subplots(, , figsize=(, ))\n i, ax (axes.ravel()):\n question_gb = df_final.groupby()[].mean()\n ax.plot(question_gb.index, question_gb)\n ax.set_xlabel()\n ax.set_ylabel()\n ax.set_title()\nThis assumes you have a DataFrame named df_final that has for a row a session, and for columns the session_id, and their answers named q_i for the ith question like this:
\n
\ncheck this notebook to find the code to do that if you don't want to bother.
I only used 1 model for all questions, it has the same score with 18 different models :D
", "votes": 0, "replies": [ { "id": 2176830, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-11T00:17:25.560000", "content": "That's surprising, that means you make the same prediction for each question and still get a good score ?
\nHow do you deal with the difference between say question 13 and question 2 ? Both can't be labeled the same for each question that's very unlikely..
I add the column 'question' in training set when training the model :D
", "votes": 1, "replies": [] }, { "id": 2177288, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-11T10:57:06.893000", "content": "Didn't even considered it ! smart idea for effiency
", "votes": 0, "replies": [] } ] } ] }, { "id": 2175753, "author_name": "", "author_url": "", "post_date": "2023-03-10T06:28:33.303000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2175744": "### Most of people's intuition was that it was important to train 18 different models , \nwhich makes total sense when you think about it, but what if we could only **make 3 or 4 different models** to get the effiency price ? \n\nAs you can see here, they are question who have similar success rates, so why not **make models to predict batches of questions and have a 2 minutes infer ??**\nTake *question 14 and 16* for example, and you will understand why even if the success rates are the same, **you can't deal will them the same way.**\n\n\n\n\nSuprising spike no ? My opinion on the matter is that **question 13 difficulty caught them offguard**, meaning that the students who got it wrong (from accuracy 0.3 to 0.5 most likely), where paying more attention, we can see that the relationship with previous accuracy isn't the same for both question, indicating that we can't deal with them the same way 👍\n\n*Note:*\n\n\n\nNotice for question 13 (the hardest question) that it's not just a *straight line but lower* than for the other questions, we can clearly see here that the average student who was paying medium attention didn't get that one right, and only the ones that where already nailing the rest got this one right. I don't really know what to do with that information, but i suggest you observe the behavior of the students that got question 13 right, might give you feature ideas ! \n\n\n", "2199735": "## here is the code to see for yourself:\n**get the \"accuracy before\" feature:**\n```python\nfor i in range(1,18):\n # Calculate the mean of questions with a number below i\n column_name = 'accuracy_before_' + str(i)\n df_final[column_name] = df_final.loc[:, ['q_'+str(j) for j in range(1, i+1)]].mean(axis=1)\n```\n**plots :**\n```python\nfig, axes = plt.subplots(6, 3, figsize=(30, 40))\nfor i, ax in enumerate(axes.ravel()):\n question_gb = df_final.groupby(f'accuracy_before_{i+1}')[f'q_{i+1}'].mean()\n ax.plot(question_gb.index, question_gb)\n ax.set_xlabel(f'previous questions accuracy (before {i+1})')\n ax.set_ylabel(f'{i+1}th question accuracy')\n ax.set_title(f'question {i+1} accuracy, by previous question accuracy')\n```\n\nThis assumes you have a DataFrame named `df_final` that has for a row a session, and for columns the session_id, and their answers named `q_i` for the `i`th question like this:\n\ncheck [this notebook](https://www.kaggle.com/code/janmpia/student-perf-eda-feature-engineering) to find the code to do that if you don't want to bother.", "2175850": "I only used 1 model for all questions, it has the same score with 18 different models :D ", "2175753": "" } }