![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2034058%2F33df7e3141e15432bd7b230a45ab9826%2Fpublic%20lb.png?generation=1566156841644247&alt=media\")
i was trying this kernel for this competition,even though the loss and validation loss decreasing decently i am getting -0.015 public lb score,why is this happening? can anyone please explain?
\n\nsee this : \n
some part of my work includes collecting code from my mate's kernel name : @ratan123
\n\nso i decided to match my model's prediction with his this kernel's model prediction : https://www.kaggle.com/ratan123/aptos-2019-keras-baseline
\n\nwhere he got 0.75 lb score\nafter comparing my result with him i got this : \n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2034058%2F2f788c9d300e52a9d2cf047e1037ec28%2Fprob.png?generation=1566157121248502&alt=media\")
you can see that out of 1928 predicted samples there are 975 predictions that is not same in both my model and his model,but rest of our predictions are exactly same,so why my kernels 5th version is less than 0% can anyone explain? it will be highly appreciated ,i made that kernel public not to get upvotes but to learn from expert kagglers like you,thanks in advance!
", "messages": [ { "id": "602223", "postDate": "08/18/2019 19:42:25", "content": "i was trying this kernel for this competition,even though the loss and validation loss decreasing decently i am getting -0.015 public lb score,why is this happening? can anyone please explain?
\n\nsee this : \n
some part of my work includes collecting code from my mate's kernel name : @ratan123
\n\nso i decided to match my model's prediction with his this kernel's model prediction : https://www.kaggle.com/ratan123/aptos-2019-keras-baseline
\n\nwhere he got 0.75 lb score\nafter comparing my result with him i got this : \n
you can see that out of 1928 predicted samples there are 975 predictions that is not same in both my model and his model,but rest of our predictions are exactly same,so why my kernels 5th version is less than 0% can anyone explain? it will be highly appreciated ,i made that kernel public not to get upvotes but to learn from expert kagglers like you,thanks in advance!
", "rawMarkdown": "i was trying[ this kernel](https://www.kaggle.com/mobassir/keras-cnn?scriptVersionId=19083722) for this competition,even though the loss and validation loss decreasing decently i am getting -0.015 public lb score,why is this happening? can anyone please explain?\n\nsee this : \n\n\nsome part of my work includes collecting code from my mate's kernel name : @ratan123\n\nso i decided to match my model's prediction with his this kernel's model prediction : https://www.kaggle.com/ratan123/aptos-2019-keras-baseline\n\nwhere he got 0.75 lb score\nafter comparing my result with him i got this : \n\n\nyou can see that out of 1928 predicted samples there are 975 predictions that is not same in both my model and his model,but rest of our predictions are exactly same,so why my kernels 5th version is less than 0% can anyone explain? it will be highly appreciated ,i made that kernel public not to get upvotes but to learn from expert kagglers like you,thanks in advance!", "votes": null }, { "id": "602252", "postDate": "08/18/2019 20:27:23", "content": "Found an explanation of what negative kappa score means:\nhttps://www.researchgate.net/post/How_to_deal_with_a_negative_Kappa_in_classification
\n\nBut this still doesn't answer your question why the model is performing so poor.
", "rawMarkdown": "Found an explanation of what negative kappa score means:\nhttps://www.researchgate.net/post/How_to_deal_with_a_negative_Kappa_in_classification\n\nBut this still doesn't answer your question why the model is performing so poor.", "votes": null }, { "id": "602266", "postDate": "08/18/2019 20:52:52", "content": "even the negative kappa score for my model was 0.65 as you said,so there is no reason why i get less than 0% public lb score,,thanks for your confirmation mate
", "rawMarkdown": "even the negative kappa score for my model was 0.65 as you said,so there is no reason why i get less than 0% public lb score,,thanks for your confirmation mate", "votes": null }, { "id": "602311", "postDate": "08/18/2019 22:10:34", "content": "Perhaps you can make a histogram of the predictions, I expect your model may be predicting class 0 most of the time.
", "rawMarkdown": "Perhaps you can make a histogram of the predictions, I expect your model may be predicting class 0 most of the time.", "votes": null }, { "id": "602317", "postDate": "08/18/2019 22:22:16", "content": "no it is not,if my model is predicting 0 most of the times than ratan's model is also predicting 0most of the time right? he got 0.75 lb score where my model has more than 1000 prediction that matches exactly with ratan's model and my model also has 0.65 kappa score that is what my team mate confirms
", "rawMarkdown": "no it is not,if my model is predicting 0 most of the times than ratan's model is also predicting 0most of the time right? he got 0.75 lb score where my model has more than 1000 prediction that matches exactly with ratan's model and my model also has 0.65 kappa score that is what my team mate confirms", "votes": null }, { "id": "602332", "postDate": "08/18/2019 23:48:23", "content": "Please don't just cast away this advice, here's a scenario:
\n\nThere are 2000 samples.\nYour friend's model predicts class 0 a thousand times. Your model predicts class 0 for every sample, you have a thousand in overlap. Your local validation score can vary wildly from LB, perhaps the distribution of labels / the samples in the LB dataset are very different.
\n\nNot related: if he's your teammate, don't forget to team up (before the deadline)!
", "rawMarkdown": "Please don't just cast away this advice, here's a scenario:\n\nThere are 2000 samples.\nYour friend's model predicts class 0 a thousand times. Your model predicts class 0 for every sample, you have a thousand in overlap. Your local validation score can vary wildly from LB, perhaps the distribution of labels / the samples in the LB dataset are very different.\n\nNot related: if he's your teammate, don't forget to team up (before the deadline)!", "votes": null }, { "id": "602335", "postDate": "08/18/2019 23:53:37", "content": "thanks for your advice,i will try different model and yeah ratan is not my team mate,my team mate is \"Aleksandra Deis\" for this competition
", "rawMarkdown": "thanks for your advice,i will try different model and yeah ratan is not my team mate,my team mate is \"Aleksandra Deis\" for this competition", "votes": null } ], "comments": [ { "id": 602252, "author_name": "aleksandradeis", "author_url": "", "post_date": "08/18/2019 20:27:23", "content": "Found an explanation of what negative kappa score means:\nhttps://www.researchgate.net/post/How_to_deal_with_a_negative_Kappa_in_classification
\n\nBut this still doesn't answer your question why the model is performing so poor.
", "votes": null, "replies": [ { "id": 602266, "author_name": "mobassir", "author_url": "", "post_date": "08/18/2019 20:52:52", "content": "even the negative kappa score for my model was 0.65 as you said,so there is no reason why i get less than 0% public lb score,,thanks for your confirmation mate
", "votes": null, "replies": [] } ] }, { "id": 602311, "author_name": "gzuidhof", "author_url": "", "post_date": "08/18/2019 22:10:34", "content": "Perhaps you can make a histogram of the predictions, I expect your model may be predicting class 0 most of the time.
", "votes": null, "replies": [ { "id": 602317, "author_name": "mobassir", "author_url": "", "post_date": "08/18/2019 22:22:16", "content": "no it is not,if my model is predicting 0 most of the times than ratan's model is also predicting 0most of the time right? he got 0.75 lb score where my model has more than 1000 prediction that matches exactly with ratan's model and my model also has 0.65 kappa score that is what my team mate confirms
", "votes": null, "replies": [] }, { "id": 602332, "author_name": "gzuidhof", "author_url": "", "post_date": "08/18/2019 23:48:23", "content": "Please don't just cast away this advice, here's a scenario:
\n\nThere are 2000 samples.\nYour friend's model predicts class 0 a thousand times. Your model predicts class 0 for every sample, you have a thousand in overlap. Your local validation score can vary wildly from LB, perhaps the distribution of labels / the samples in the LB dataset are very different.
\n\nNot related: if he's your teammate, don't forget to team up (before the deadline)!
", "votes": null, "replies": [] }, { "id": 602335, "author_name": "mobassir", "author_url": "", "post_date": "08/18/2019 23:53:37", "content": "thanks for your advice,i will try different model and yeah ratan is not my team mate,my team mate is \"Aleksandra Deis\" for this competition
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "602223": "i was trying[ this kernel](https://www.kaggle.com/mobassir/keras-cnn?scriptVersionId=19083722) for this competition,even though the loss and validation loss decreasing decently i am getting -0.015 public lb score,why is this happening? can anyone please explain?\n\nsee this : \n\n\nsome part of my work includes collecting code from my mate's kernel name : @ratan123\n\nso i decided to match my model's prediction with his this kernel's model prediction : https://www.kaggle.com/ratan123/aptos-2019-keras-baseline\n\nwhere he got 0.75 lb score\nafter comparing my result with him i got this : \n\n\nyou can see that out of 1928 predicted samples there are 975 predictions that is not same in both my model and his model,but rest of our predictions are exactly same,so why my kernels 5th version is less than 0% can anyone explain? it will be highly appreciated ,i made that kernel public not to get upvotes but to learn from expert kagglers like you,thanks in advance!", "602252": "Found an explanation of what negative kappa score means:\nhttps://www.researchgate.net/post/How_to_deal_with_a_negative_Kappa_in_classification\n\nBut this still doesn't answer your question why the model is performing so poor.", "602266": "even the negative kappa score for my model was 0.65 as you said,so there is no reason why i get less than 0% public lb score,,thanks for your confirmation mate", "602311": "Perhaps you can make a histogram of the predictions, I expect your model may be predicting class 0 most of the time.", "602317": "no it is not,if my model is predicting 0 most of the times than ratan's model is also predicting 0most of the time right? he got 0.75 lb score where my model has more than 1000 prediction that matches exactly with ratan's model and my model also has 0.65 kappa score that is what my team mate confirms", "602332": "Please don't just cast away this advice, here's a scenario:\n\nThere are 2000 samples.\nYour friend's model predicts class 0 a thousand times. Your model predicts class 0 for every sample, you have a thousand in overlap. Your local validation score can vary wildly from LB, perhaps the distribution of labels / the samples in the LB dataset are very different.\n\nNot related: if he's your teammate, don't forget to team up (before the deadline)!", "602335": "thanks for your advice,i will try different model and yeah ratan is not my team mate,my team mate is \"Aleksandra Deis\" for this competition" }, "source": "meta" }