Knowledge tracking (KT) looks a lot like recommender engines (RE). The main difference would be that in recommender systems, we would be looking for an item that the user would \"like\" looking over the whole item set. But, in Knowledge tracking, the item is already given and we're requested to predict if it will be \"liked\" or not. Apart from this \"small\" difference, both KT and RE looks alike. Hence, we can apply advances in RE to KT as the first domain seems to enjoy more researchs.
\nA common issue with RE is the cold start problem. According to Wikipedia, cold start arises when the system cannot draw any inferences for users or items about which it has not yet gathered sufficient information. This is the case for new items or new users.
\nIn this competition, item related cold start is not so important as the items from test set are completely included in train. But, if you're doing some iterative training, you can still facing the item cold start issue as well.
\nSo, how one could deal with th CS problem ? Well, there is a quite rich literature about the problem.
\nThe item cold-start problem refers to when items added to the catalogue have either none or very little interactions. The system can't make any meaningful inference due to the lack of information.
\nOne way of dealing with it consists in using content-based algorithms. A content-based algorithms relies on the item features (price, description, ingredients, …) and thus remain robust even when there is no interaction. In the context of Knowledge tracking, a content-based algorithm would be any model that uses question/lectures features like tags, bundles and parts.
\nThe new user case refers to when a new user enrolls in the system and for a certain period of time the recommender has to provide recommendation without relying on the user's past interactions, since none has occurred yet.
\nOne can deal with this case by using a user-user content based algorithm which typically relies on user's features (e.g. age, gender, country) to find similar users and recommend the items they interacted with in a positive way, therefore being robust to the new user case. In this competition, a user-user content based algorithm will be any model that uses user features. Unfortunately, we have no user data available and it will be hard to deal with the new user cold start.
\nIn a nutshell, making prediction for new users or new items is very challenging and those who will find the right way of doing it, could get a serious edge over other competitors.
\nAnd you, have you seen the problem, and how are you dealing with it ?
", "messages": [ { "id": "1110191", "postDate": "12/12/2020 14:03:04", "content": "Knowledge tracking (KT) looks a lot like recommender engines (RE). The main difference would be that in recommender systems, we would be looking for an item that the user would \"like\" looking over the whole item set. But, in Knowledge tracking, the item is already given and we're requested to predict if it will be \"liked\" or not. Apart from this \"small\" difference, both KT and RE looks alike. Hence, we can apply advances in RE to KT as the first domain seems to enjoy more researchs.
\nA common issue with RE is the cold start problem. According to Wikipedia, cold start arises when the system cannot draw any inferences for users or items about which it has not yet gathered sufficient information. This is the case for new items or new users.
\nIn this competition, item related cold start is not so important as the items from test set are completely included in train. But, if you're doing some iterative training, you can still facing the item cold start issue as well.
\nSo, how one could deal with th CS problem ? Well, there is a quite rich literature about the problem.
\nThe item cold-start problem refers to when items added to the catalogue have either none or very little interactions. The system can't make any meaningful inference due to the lack of information.
\nOne way of dealing with it consists in using content-based algorithms. A content-based algorithms relies on the item features (price, description, ingredients, …) and thus remain robust even when there is no interaction. In the context of Knowledge tracking, a content-based algorithm would be any model that uses question/lectures features like tags, bundles and parts.
\nThe new user case refers to when a new user enrolls in the system and for a certain period of time the recommender has to provide recommendation without relying on the user's past interactions, since none has occurred yet.
\nOne can deal with this case by using a user-user content based algorithm which typically relies on user's features (e.g. age, gender, country) to find similar users and recommend the items they interacted with in a positive way, therefore being robust to the new user case. In this competition, a user-user content based algorithm will be any model that uses user features. Unfortunately, we have no user data available and it will be hard to deal with the new user cold start.
\nIn a nutshell, making prediction for new users or new items is very challenging and those who will find the right way of doing it, could get a serious edge over other competitors.
\nAnd you, have you seen the problem, and how are you dealing with it ?
", "rawMarkdown": "Knowledge tracking (KT) looks a lot like recommender engines (RE). The main difference would be that in recommender systems, we would be looking for an item that the user would \"*like*\" looking over the whole item set. But, in Knowledge tracking, the item is already given and we're requested to predict if it will be \"*liked*\" or not. Apart from this \"small\" difference, both **KT** and **RE** looks alike. Hence, we can apply advances in **RE** to **KT** as the first domain seems to enjoy more researchs.\n\n\nA common issue with RE is the **cold start problem**. According to Wikipedia, **cold start** arises when the system cannot draw any inferences for users or items about which it has not yet gathered sufficient information. This is the case for new items or new users.\n\nIn this competition, item related cold start is not so important as the items from test set are completely included in train. But, if you're doing some iterative training, you can still facing the item cold start issue as well.\n\nSo, how one could deal with th CS problem ? Well, there is a quite rich literature about the problem.\n\n### New item cold start\nThe item cold-start problem refers to when items added to the catalogue have either none or very little interactions. The system can't make any meaningful inference due to the lack of information. \n\nOne way of dealing with it consists in using **content-based** algorithms. A content-based algorithms relies on the item features (price, description, ingredients, ...) and thus remain robust even when there is no interaction. In the context of Knowledge tracking, a **content-based** algorithm would be any model that uses question/lectures features like tags, bundles and parts.\n\n### New user cold start\nThe new user case refers to when a new user enrolls in the system and for a certain period of time the recommender has to provide recommendation without relying on the user's past interactions, since none has occurred yet.\n\nOne can deal with this case by using a user-user content based algorithm which typically relies on user's features (e.g. age, gender, country) to find similar users and recommend the items they interacted with in a positive way, therefore being robust to the new user case. In this competition, a user-user content based algorithm will be any model that uses user features. Unfortunately, we have no user data available and it will be hard to deal with the **new user cold start**.\n\n\nIn a nutshell, making prediction for new users or new items is very challenging and those who will find the right way of doing it, could get a serious edge over other competitors.\n\nAnd you, have you seen the problem, and how are you dealing with it ?", "votes": null }, { "id": "1110197", "postDate": "12/12/2020 14:06:19", "content": "Nice explanation. Wonder how others are addressing this issue
", "rawMarkdown": "Nice explanation. Wonder how others are addressing this issue", "votes": null }, { "id": "1110204", "postDate": "12/12/2020 14:12:43", "content": "I have one insight, with regards to cold start , does the user , select random topic to learn from(and also has random questions to answer) or does the user follow a learning path , with respect to recommendation engines , the item is completely randomized and the user has a freedom to choose anything , but perhaps with respect to education, he would face same things with respect to the course , he has chosen. Thus , the cold start becomes maybe different in this case ..
\nPlus with recommender systems , we can fill the values of embedding with respect to other users , as to how they have done , but can we do the same with learning ?
I believe that, unlike recommender systems, we could ignore the oracle that chooses the questions. We just need to be able to tell if the chosen question will be \"liked\" or not. The cold start arises here when the oracle selects a question that has little interaction or when the user has no or little history.
\nFor your second point, the answer seems to be Yes : we can predict a learner behaviour by looking to his \"neighbours\".
", "rawMarkdown": "I believe that, unlike recommender systems, we could ignore the oracle that chooses the questions. We just need to be able to tell if the chosen question will be \"*liked*\" or not. The cold start arises here when the oracle selects a question that has little interaction or when the user has no or little history.\n\nFor your second point, the answer seems to be Yes : we can predict a learner behaviour by looking to his \"neighbours\".", "votes": null }, { "id": "1110252", "postDate": "12/12/2020 14:55:51", "content": "I agree with your first part, but if we follow that method for the second part we are disregarding the particular user's heterogeneity concerning the field and his thinking. Wouldn't creating separate user performance measures more useable in that case ??
\nEdit : I think I am wrong about this statement , we can link the same
You have said pretty much all that is to be said…At least in my view, I dont see an easy way out for this other than relying on the exercises, tags & lectures features.
\nThe only other thing I can think of (just thinking aloud) is while it takes some time for the user's history to be built, there could be certain 'profiling' features which can be immediately gauged based on the first few interactions? - e.g. interaction frequency, whether she watches lectures or not, whether she views explanations or not (if correct or if wrong)..response speed and of course response-correctness..All this may give a quick initial profile regarding generic factors like attitude, aptitude, dedication etc before a more deeper one involving knowledge can be built? Not sure though if it will help practically
", "rawMarkdown": "You have said pretty much all that is to be said...At least in my view, I dont see an easy way out for this other than relying on the exercises, tags & lectures features. \n\nThe only other thing I can think of (just thinking aloud) is while it takes some time for the user's history to be built, there could be certain 'profiling' features which can be immediately gauged based on the first few interactions? - e.g. interaction frequency, whether she watches lectures or not, whether she views explanations or not (if correct or if wrong)..response speed and of course response-correctness..All this may give a quick initial profile regarding generic factors like attitude, aptitude, dedication etc before a more deeper one involving knowledge can be built? Not sure though if it will help practically", "votes": null }, { "id": "1110292", "postDate": "12/12/2020 15:46:59", "content": "Thanks @allohvk .
\nYes, your approach could work. Further more, based on those features, we can, for instance, cluster newcomers by a mean of an online clustering algorithm … then the new user will receive similar predictions as the ones from his cluster.
", "rawMarkdown": "Thanks @allohvk .\n\nYes, your approach could work. Further more, based on those features, we can, for instance, cluster newcomers by a mean of an online clustering algorithm ... then the new user will receive similar predictions as the ones from his cluster.", "votes": null }, { "id": "1110318", "postDate": "12/12/2020 16:06:52", "content": "Thanks @kneroma ..The other thought that just stuck me now - I remember seeing somewhere in the data description that the first few questions are diagnostic . I specifically remember that word..'diagnostic' and also they say that these dont have explanations. It is quite likely that with the response to these first few questions, the user's 'initial' profile can quickly be built… then (s)he could be bracketed into a cluster or a nearest-neighbour approach could be used…until this user can be more accurately modelled.
", "rawMarkdown": "Thanks @kneroma ..The other thought that just stuck me now - I remember seeing somewhere in the data description that the first few questions are diagnostic . I specifically remember that word..'diagnostic' and also they say that these dont have explanations. It is quite likely that with the response to these first few questions, the user's 'initial' profile can quickly be built... then (s)he could be bracketed into a cluster or a nearest-neighbour approach could be used...until this user can be more accurately modelled.", "votes": null }, { "id": "1110456", "postDate": "12/12/2020 18:53:50", "content": "You're totally right !
", "rawMarkdown": "You're totally right !", "votes": null } ], "comments": [ { "id": 1110197, "author_name": "louise2001", "author_url": "", "post_date": "12/12/2020 14:06:19", "content": "Nice explanation. Wonder how others are addressing this issue
", "votes": null, "replies": [] }, { "id": 1110204, "author_name": "sahilmaheshwari", "author_url": "", "post_date": "12/12/2020 14:12:43", "content": "I have one insight, with regards to cold start , does the user , select random topic to learn from(and also has random questions to answer) or does the user follow a learning path , with respect to recommendation engines , the item is completely randomized and the user has a freedom to choose anything , but perhaps with respect to education, he would face same things with respect to the course , he has chosen. Thus , the cold start becomes maybe different in this case ..
\nPlus with recommender systems , we can fill the values of embedding with respect to other users , as to how they have done , but can we do the same with learning ?
I believe that, unlike recommender systems, we could ignore the oracle that chooses the questions. We just need to be able to tell if the chosen question will be \"liked\" or not. The cold start arises here when the oracle selects a question that has little interaction or when the user has no or little history.
\nFor your second point, the answer seems to be Yes : we can predict a learner behaviour by looking to his \"neighbours\".
", "votes": null, "replies": [] }, { "id": 1110252, "author_name": "sahilmaheshwari", "author_url": "", "post_date": "12/12/2020 14:55:51", "content": "I agree with your first part, but if we follow that method for the second part we are disregarding the particular user's heterogeneity concerning the field and his thinking. Wouldn't creating separate user performance measures more useable in that case ??
\nEdit : I think I am wrong about this statement , we can link the same
You have said pretty much all that is to be said…At least in my view, I dont see an easy way out for this other than relying on the exercises, tags & lectures features.
\nThe only other thing I can think of (just thinking aloud) is while it takes some time for the user's history to be built, there could be certain 'profiling' features which can be immediately gauged based on the first few interactions? - e.g. interaction frequency, whether she watches lectures or not, whether she views explanations or not (if correct or if wrong)..response speed and of course response-correctness..All this may give a quick initial profile regarding generic factors like attitude, aptitude, dedication etc before a more deeper one involving knowledge can be built? Not sure though if it will help practically
", "votes": null, "replies": [ { "id": 1110292, "author_name": "kneroma", "author_url": "", "post_date": "12/12/2020 15:46:59", "content": "Thanks @allohvk .
\nYes, your approach could work. Further more, based on those features, we can, for instance, cluster newcomers by a mean of an online clustering algorithm … then the new user will receive similar predictions as the ones from his cluster.
", "votes": null, "replies": [] }, { "id": 1110318, "author_name": "allohvk", "author_url": "", "post_date": "12/12/2020 16:06:52", "content": "Thanks @kneroma ..The other thought that just stuck me now - I remember seeing somewhere in the data description that the first few questions are diagnostic . I specifically remember that word..'diagnostic' and also they say that these dont have explanations. It is quite likely that with the response to these first few questions, the user's 'initial' profile can quickly be built… then (s)he could be bracketed into a cluster or a nearest-neighbour approach could be used…until this user can be more accurately modelled.
", "votes": null, "replies": [] }, { "id": 1110456, "author_name": "kneroma", "author_url": "", "post_date": "12/12/2020 18:53:50", "content": "You're totally right !
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1110191": "Knowledge tracking (KT) looks a lot like recommender engines (RE). The main difference would be that in recommender systems, we would be looking for an item that the user would \"*like*\" looking over the whole item set. But, in Knowledge tracking, the item is already given and we're requested to predict if it will be \"*liked*\" or not. Apart from this \"small\" difference, both **KT** and **RE** looks alike. Hence, we can apply advances in **RE** to **KT** as the first domain seems to enjoy more researchs.\n\n\nA common issue with RE is the **cold start problem**. According to Wikipedia, **cold start** arises when the system cannot draw any inferences for users or items about which it has not yet gathered sufficient information. This is the case for new items or new users.\n\nIn this competition, item related cold start is not so important as the items from test set are completely included in train. But, if you're doing some iterative training, you can still facing the item cold start issue as well.\n\nSo, how one could deal with th CS problem ? Well, there is a quite rich literature about the problem.\n\n### New item cold start\nThe item cold-start problem refers to when items added to the catalogue have either none or very little interactions. The system can't make any meaningful inference due to the lack of information. \n\nOne way of dealing with it consists in using **content-based** algorithms. A content-based algorithms relies on the item features (price, description, ingredients, ...) and thus remain robust even when there is no interaction. In the context of Knowledge tracking, a **content-based** algorithm would be any model that uses question/lectures features like tags, bundles and parts.\n\n### New user cold start\nThe new user case refers to when a new user enrolls in the system and for a certain period of time the recommender has to provide recommendation without relying on the user's past interactions, since none has occurred yet.\n\nOne can deal with this case by using a user-user content based algorithm which typically relies on user's features (e.g. age, gender, country) to find similar users and recommend the items they interacted with in a positive way, therefore being robust to the new user case. In this competition, a user-user content based algorithm will be any model that uses user features. Unfortunately, we have no user data available and it will be hard to deal with the **new user cold start**.\n\n\nIn a nutshell, making prediction for new users or new items is very challenging and those who will find the right way of doing it, could get a serious edge over other competitors.\n\nAnd you, have you seen the problem, and how are you dealing with it ?", "1110197": "Nice explanation. Wonder how others are addressing this issue", "1110204": "I have one insight, with regards to cold start , does the user , select random topic to learn from(and also has random questions to answer) or does the user follow a learning path , with respect to recommendation engines , the item is completely randomized and the user has a freedom to choose anything , but perhaps with respect to education, he would face same things with respect to the course , he has chosen. Thus , the cold start becomes maybe different in this case ..\nPlus with recommender systems , we can fill the values of embedding with respect to other users , as to how they have done , but can we do the same with learning ?", "1110227": "I believe that, unlike recommender systems, we could ignore the oracle that chooses the questions. We just need to be able to tell if the chosen question will be \"*liked*\" or not. The cold start arises here when the oracle selects a question that has little interaction or when the user has no or little history.\n\nFor your second point, the answer seems to be Yes : we can predict a learner behaviour by looking to his \"neighbours\".", "1110252": "I agree with your first part, but if we follow that method for the second part we are disregarding the particular user's heterogeneity concerning the field and his thinking. Wouldn't creating separate user performance measures more useable in that case ??\nEdit : I think I am wrong about this statement , we can link the same", "1110283": "You have said pretty much all that is to be said...At least in my view, I dont see an easy way out for this other than relying on the exercises, tags & lectures features. \n\nThe only other thing I can think of (just thinking aloud) is while it takes some time for the user's history to be built, there could be certain 'profiling' features which can be immediately gauged based on the first few interactions? - e.g. interaction frequency, whether she watches lectures or not, whether she views explanations or not (if correct or if wrong)..response speed and of course response-correctness..All this may give a quick initial profile regarding generic factors like attitude, aptitude, dedication etc before a more deeper one involving knowledge can be built? Not sure though if it will help practically", "1110292": "Thanks @allohvk .\n\nYes, your approach could work. Further more, based on those features, we can, for instance, cluster newcomers by a mean of an online clustering algorithm ... then the new user will receive similar predictions as the ones from his cluster.", "1110318": "Thanks @kneroma ..The other thought that just stuck me now - I remember seeing somewhere in the data description that the first few questions are diagnostic . I specifically remember that word..'diagnostic' and also they say that these dont have explanations. It is quite likely that with the response to these first few questions, the user's 'initial' profile can quickly be built... then (s)he could be bracketed into a cluster or a nearest-neighbour approach could be used...until this user can be more accurately modelled.", "1110456": "You're totally right !" }, "source": "meta" }