{ "id": 199483, "title": "Overlapping lecture_id and question_id", "url": "/competitions/riiid-test-answer-prediction/discussion/199483", "author_name": "", "post_date": "2020-11-25T22:39:10.587367200Z", "votes": 2, "comment_count": 7, "views": 0, "content": "

After doing a quick search, I found there's overlapping between lecture_id and question_id

\n

lectures = pd.read_csv(\"data/lectures.csv\")
\nquestions = pd.read_csv(\"data/questions.csv\")
\nlectures[lectures['lecture_id'].isin(questions['question_id'])]

\n

I'm wondering how can we interpret this result. Does it mean that there are lectures specifically taught for these questions? Any help would be greatly appreciated!

", "messages": [ { "id": "1091267", "postDate": "11/25/2020 22:39:10", "content": "

After doing a quick search, I found there's overlapping between lecture_id and question_id

\n

lectures = pd.read_csv(\"data/lectures.csv\")
\nquestions = pd.read_csv(\"data/questions.csv\")
\nlectures[lectures['lecture_id'].isin(questions['question_id'])]

\n

I'm wondering how can we interpret this result. Does it mean that there are lectures specifically taught for these questions? Any help would be greatly appreciated!

", "rawMarkdown": "After doing a quick search, I found there's overlapping between lecture_id and question_id\n\n`lectures = pd.read_csv(\"data/lectures.csv\")`\n`questions = pd.read_csv(\"data/questions.csv\")`\n`lectures[lectures['lecture_id'].isin(questions['question_id'])]`\n\nI'm wondering how can we interpret this result. Does it mean that there are lectures specifically taught for these questions? Any help would be greatly appreciated!", "votes": null }, { "id": "1092104", "postDate": "11/26/2020 14:46:25", "content": "

I think the common understanding (OK - so it's mostly my understanding) is that question_id and lecture_id are content_id and can be used to merge into a larger file.

\n

What's not clear is the tags. Do the tags in the question file and the tags in the lecture file refer to the same subject matter? I assume this is what you mean by overlapping. If tag 100 in the lecture file was on the topic of American football, than my belief is that anytime question tag 100 appears for a user, than the question touched on American football.

\n

If you follow the links from the discussion topic by the host you arrive at th github linked below. From the description on this page the tags for lectures and questions are the \"same\" grouping of skill sets needed.

\n
\n

Santa provides a total 13,169 problems and 1,021 lectures tagged with 293 types of skills, and each of them has been consumed 95,294,926 times and 601,805 times, respectively>

\n
\n

https://github.com/riiid/ednet

", "rawMarkdown": "I think the common understanding (OK - so it's mostly my understanding) is that question_id and lecture_id are content_id and can be used to merge into a larger file.\n\nWhat's not clear is the tags. Do the tags in the question file and the tags in the lecture file refer to the same subject matter? I assume this is what you mean by overlapping. If tag 100 in the lecture file was on the topic of American football, than my belief is that anytime question tag 100 appears for a user, than the question touched on American football. \n\nIf you follow the links from the discussion topic by the host you arrive at th github linked below. From the description on this page the tags for lectures and questions are the \"same\" grouping of skill sets needed.\n\n> Santa provides a total 13,169 problems and 1,021 lectures tagged with 293 types of skills, and each of them has been consumed 95,294,926 times and 601,805 times, respectively> \n\n\nhttps://github.com/riiid/ednet", "votes": null }, { "id": "1092118", "postDate": "11/26/2020 14:59:40", "content": "

Yes, there are overlapping ids. I don't think any of those lectures are specifically for the corresponding questions because there are 13523 unique questions and 418 unique lectures. If it was the case, there would be more lectures than 418.

\n

However, there is one-to-many relationship between lecture tag and question tags. You can find relationships of questions and lectures from their tags.

", "rawMarkdown": "Yes, there are overlapping ids. I don't think any of those lectures are specifically for the corresponding questions because there are 13523 unique questions and 418 unique lectures. If it was the case, there would be more lectures than 418.\n\nHowever, there is one-to-many relationship between lecture tag and question tags. You can find relationships of questions and lectures from their tags.", "votes": null }, { "id": "1092125", "postDate": "11/26/2020 15:03:32", "content": "

Gunes - I assume your the downvote. Since I think you are wrong - should I down vote you?

", "rawMarkdown": "Gunes - I assume your the downvote. Since I think you are wrong - should I down vote you?", "votes": null }, { "id": "1092132", "postDate": "11/26/2020 15:07:56", "content": "

Nope it wasn't me. I barely use downvote. But why do think am I wrong?

", "rawMarkdown": "Nope it wasn't me. I barely use downvote. But why do think am I wrong?", "votes": null }, { "id": "1092140", "postDate": "11/26/2020 15:11:00", "content": "

The github page - If my assumption is correct that this git and our data set are the same problem, than clear from the text on that git that the tags are the same skill sets.

\n
\n

tags is the expert-annotated tag for each lecture, which is a single integer. The tags that are used for questions and lectures are the same.>

\n
\n

PS - the down vote is gone so maybe it was a mis-click by someone rather than a strong disagreement with my opinion.

", "rawMarkdown": "The github page - If my assumption is correct that this git and our data set are the same problem, than clear from the text on that git that the tags are the same skill sets. \n\n> tags is the expert-annotated tag for each lecture, which is a single integer. The tags that are used for questions and lectures are the same.> \n\nPS - the down vote is gone so maybe it was a mis-click by someone rather than a strong disagreement with my opinion.", "votes": null }, { "id": "1092165", "postDate": "11/26/2020 15:21:31", "content": "

We are on the same page then. I said content_ids of questions and lectures don't have any relationship which is what @lizhaonan asked.

\n

You don't have to make assumptions on that file though. Check this discussion post by Sohier Dane. He wrote I'm updating the dataset now so lecture tags will match the tags in questions.csv. in that post.

", "rawMarkdown": "We are on the same page then. I said content_ids of questions and lectures don't have any relationship which is what @lizhaonan asked.\n\nYou don't have to make assumptions on that file though. Check [this](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/191106) discussion post by Sohier Dane. He wrote **I'm updating the dataset now so lecture tags will match the tags in questions.csv.** in that post.", "votes": null }, { "id": "1092207", "postDate": "11/26/2020 15:56:25", "content": "

Hello @gunesevitan and @pcjimmmy, thank you both for your comments. I agree that you are talking about the same thing that tags are for linking questions to lectures

", "rawMarkdown": "Hello @gunesevitan and @pcjimmmy, thank you both for your comments. I agree that you are talking about the same thing that tags are for linking questions to lectures", "votes": null } ], "comments": [ { "id": 1092104, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/26/2020 14:46:25", "content": "

I think the common understanding (OK - so it's mostly my understanding) is that question_id and lecture_id are content_id and can be used to merge into a larger file.

\n

What's not clear is the tags. Do the tags in the question file and the tags in the lecture file refer to the same subject matter? I assume this is what you mean by overlapping. If tag 100 in the lecture file was on the topic of American football, than my belief is that anytime question tag 100 appears for a user, than the question touched on American football.

\n

If you follow the links from the discussion topic by the host you arrive at th github linked below. From the description on this page the tags for lectures and questions are the \"same\" grouping of skill sets needed.

\n
\n

Santa provides a total 13,169 problems and 1,021 lectures tagged with 293 types of skills, and each of them has been consumed 95,294,926 times and 601,805 times, respectively>

\n
\n

https://github.com/riiid/ednet

", "votes": null, "replies": [] }, { "id": 1092118, "author_name": "gunesevitan", "author_url": "", "post_date": "11/26/2020 14:59:40", "content": "

Yes, there are overlapping ids. I don't think any of those lectures are specifically for the corresponding questions because there are 13523 unique questions and 418 unique lectures. If it was the case, there would be more lectures than 418.

\n

However, there is one-to-many relationship between lecture tag and question tags. You can find relationships of questions and lectures from their tags.

", "votes": null, "replies": [ { "id": 1092125, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/26/2020 15:03:32", "content": "

Gunes - I assume your the downvote. Since I think you are wrong - should I down vote you?

", "votes": null, "replies": [] }, { "id": 1092132, "author_name": "gunesevitan", "author_url": "", "post_date": "11/26/2020 15:07:56", "content": "

Nope it wasn't me. I barely use downvote. But why do think am I wrong?

", "votes": null, "replies": [] }, { "id": 1092140, "author_name": "pcjimmmy", "author_url": "", "post_date": "11/26/2020 15:11:00", "content": "

The github page - If my assumption is correct that this git and our data set are the same problem, than clear from the text on that git that the tags are the same skill sets.

\n
\n

tags is the expert-annotated tag for each lecture, which is a single integer. The tags that are used for questions and lectures are the same.>

\n
\n

PS - the down vote is gone so maybe it was a mis-click by someone rather than a strong disagreement with my opinion.

", "votes": null, "replies": [] }, { "id": 1092165, "author_name": "gunesevitan", "author_url": "", "post_date": "11/26/2020 15:21:31", "content": "

We are on the same page then. I said content_ids of questions and lectures don't have any relationship which is what @lizhaonan asked.

\n

You don't have to make assumptions on that file though. Check this discussion post by Sohier Dane. He wrote I'm updating the dataset now so lecture tags will match the tags in questions.csv. in that post.

", "votes": null, "replies": [] }, { "id": 1092207, "author_name": "lizhaonan", "author_url": "", "post_date": "11/26/2020 15:56:25", "content": "

Hello @gunesevitan and @pcjimmmy, thank you both for your comments. I agree that you are talking about the same thing that tags are for linking questions to lectures

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1091267": "After doing a quick search, I found there's overlapping between lecture_id and question_id\n\n`lectures = pd.read_csv(\"data/lectures.csv\")`\n`questions = pd.read_csv(\"data/questions.csv\")`\n`lectures[lectures['lecture_id'].isin(questions['question_id'])]`\n\nI'm wondering how can we interpret this result. Does it mean that there are lectures specifically taught for these questions? Any help would be greatly appreciated!", "1092104": "I think the common understanding (OK - so it's mostly my understanding) is that question_id and lecture_id are content_id and can be used to merge into a larger file.\n\nWhat's not clear is the tags. Do the tags in the question file and the tags in the lecture file refer to the same subject matter? I assume this is what you mean by overlapping. If tag 100 in the lecture file was on the topic of American football, than my belief is that anytime question tag 100 appears for a user, than the question touched on American football. \n\nIf you follow the links from the discussion topic by the host you arrive at th github linked below. From the description on this page the tags for lectures and questions are the \"same\" grouping of skill sets needed.\n\n> Santa provides a total 13,169 problems and 1,021 lectures tagged with 293 types of skills, and each of them has been consumed 95,294,926 times and 601,805 times, respectively> \n\n\nhttps://github.com/riiid/ednet", "1092118": "Yes, there are overlapping ids. I don't think any of those lectures are specifically for the corresponding questions because there are 13523 unique questions and 418 unique lectures. If it was the case, there would be more lectures than 418.\n\nHowever, there is one-to-many relationship between lecture tag and question tags. You can find relationships of questions and lectures from their tags.", "1092125": "Gunes - I assume your the downvote. Since I think you are wrong - should I down vote you?", "1092132": "Nope it wasn't me. I barely use downvote. But why do think am I wrong?", "1092140": "The github page - If my assumption is correct that this git and our data set are the same problem, than clear from the text on that git that the tags are the same skill sets. \n\n> tags is the expert-annotated tag for each lecture, which is a single integer. The tags that are used for questions and lectures are the same.> \n\nPS - the down vote is gone so maybe it was a mis-click by someone rather than a strong disagreement with my opinion.", "1092165": "We are on the same page then. I said content_ids of questions and lectures don't have any relationship which is what @lizhaonan asked.\n\nYou don't have to make assumptions on that file though. Check [this](https://www.kaggle.com/c/riiid-test-answer-prediction/discussion/191106) discussion post by Sohier Dane. He wrote **I'm updating the dataset now so lecture tags will match the tags in questions.csv.** in that post.", "1092207": "Hello @gunesevitan and @pcjimmmy, thank you both for your comments. I agree that you are talking about the same thing that tags are for linking questions to lectures" }, "source": "meta" }