Hello,
\nI am new to Keras and Kaggle and when I'm submitting my predictions I'm getting a score of zero. However, when training the model it seemed to be performing fairly well on the validation data, and the predictions from the model also seem, at first glance, fairly sensible (i.e. It isn't predicting the same class for every image in the test set). Does anyone have any ideas what I'm doing wrong?
\nTraining Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-training
\nInference Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-inference
(Note: For the inference notebook version 2 is the correct version. I train in the training notebook, save the model and then use that model in the inference notebook which is run offline)
\nThanks in advance!
", "messages": [ { "id": "1006736", "postDate": "09/11/2020 14:15:37", "content": "Hello,
\nI am new to Keras and Kaggle and when I'm submitting my predictions I'm getting a score of zero. However, when training the model it seemed to be performing fairly well on the validation data, and the predictions from the model also seem, at first glance, fairly sensible (i.e. It isn't predicting the same class for every image in the test set). Does anyone have any ideas what I'm doing wrong?
\nTraining Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-training
\nInference Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-inference
(Note: For the inference notebook version 2 is the correct version. I train in the training notebook, save the model and then use that model in the inference notebook which is run offline)
\nThanks in advance!
", "rawMarkdown": "Hello,\n\nI am new to Keras and Kaggle and when I'm submitting my predictions I'm getting a score of zero. However, when training the model it seemed to be performing fairly well on the validation data, and the predictions from the model also seem, at first glance, fairly sensible (i.e. It isn't predicting the same class for every image in the test set). Does anyone have any ideas what I'm doing wrong?\n\nTraining Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-training \nInference Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-inference\n\n(Note: For the inference notebook version 2 is the correct version. I train in the training notebook, save the model and then use that model in the inference notebook which is run offline)\n\nThanks in advance!", "votes": null }, { "id": "1006742", "postDate": "09/11/2020 14:22:46", "content": "Hello @tchristie
\nI noticed that you're using the code from Pre-trained MobileNetV2 (1000 classes, 1 epoch). The code is using \"categorical_accuracy\" as a metric, but the competition's evaluation metric is \"Global Average Precision\"
", "rawMarkdown": "Hello @tchristie\n\nI noticed that you're using the code from [Pre-trained MobileNetV2 (1000 classes, 1 epoch)](https://www.kaggle.com/socathie/pre-trained-mobilenetv2-1000-classes-1-epoch). The code is using \"categorical_accuracy\" as a metric, but the competition's evaluation metric is \"Global Average Precision\"", "votes": null }, { "id": "1006780", "postDate": "09/11/2020 14:46:37", "content": "Thanks @trikialaaa . What would you suggest I do? As metrics aren't used for training a model I assume that using \"categorical_accuracy\" is just giving me unrealistic expectations for my score rather than causing something to actually be wrong with the model. I would still expect my model to get a score above 0.000 though?
", "rawMarkdown": "Thanks @trikialaaa . What would you suggest I do? As metrics aren't used for training a model I assume that using \"categorical_accuracy\" is just giving me unrealistic expectations for my score rather than causing something to actually be wrong with the model. I would still expect my model to get a score above 0.000 though?", "votes": null }, { "id": "1007926", "postDate": "09/12/2020 16:20:43", "content": "Same problem here. Did you solve this problem?
", "rawMarkdown": "Same problem here. Did you solve this problem?", "votes": null }, { "id": "1008616", "postDate": "09/13/2020 09:12:35", "content": "@guagugu No I haven't solved it yet. I see you have managed to get a score on the board though, did you manage to solve it?
", "rawMarkdown": "guagugu No I haven't solved it yet. I see you have managed to get a score on the board though, did you manage to solve it?", "votes": null }, { "id": "1008760", "postDate": "09/13/2020 11:11:05", "content": "That score is from another kernel, the 0 PB score problem is still there…
", "rawMarkdown": "That score is from another kernel, the 0 PB score problem is still there...", "votes": null } ], "comments": [ { "id": 1006742, "author_name": "trikialaaa", "author_url": "", "post_date": "09/11/2020 14:22:46", "content": "Hello @tchristie
\nI noticed that you're using the code from Pre-trained MobileNetV2 (1000 classes, 1 epoch). The code is using \"categorical_accuracy\" as a metric, but the competition's evaluation metric is \"Global Average Precision\"
", "votes": null, "replies": [ { "id": 1006780, "author_name": "tchristie", "author_url": "", "post_date": "09/11/2020 14:46:37", "content": "Thanks @trikialaaa . What would you suggest I do? As metrics aren't used for training a model I assume that using \"categorical_accuracy\" is just giving me unrealistic expectations for my score rather than causing something to actually be wrong with the model. I would still expect my model to get a score above 0.000 though?
", "votes": null, "replies": [] } ] }, { "id": 1007926, "author_name": "guagugu", "author_url": "", "post_date": "09/12/2020 16:20:43", "content": "Same problem here. Did you solve this problem?
", "votes": null, "replies": [ { "id": 1008616, "author_name": "tchristie", "author_url": "", "post_date": "09/13/2020 09:12:35", "content": "@guagugu No I haven't solved it yet. I see you have managed to get a score on the board though, did you manage to solve it?
", "votes": null, "replies": [] }, { "id": 1008760, "author_name": "guagugu", "author_url": "", "post_date": "09/13/2020 11:11:05", "content": "That score is from another kernel, the 0 PB score problem is still there…
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1006736": "Hello,\n\nI am new to Keras and Kaggle and when I'm submitting my predictions I'm getting a score of zero. However, when training the model it seemed to be performing fairly well on the validation data, and the predictions from the model also seem, at first glance, fairly sensible (i.e. It isn't predicting the same class for every image in the test set). Does anyone have any ideas what I'm doing wrong?\n\nTraining Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-training \nInference Notebook: https://www.kaggle.com/tchristie/landmark-recognition-2020-inference\n\n(Note: For the inference notebook version 2 is the correct version. I train in the training notebook, save the model and then use that model in the inference notebook which is run offline)\n\nThanks in advance!", "1006742": "Hello @tchristie\n\nI noticed that you're using the code from [Pre-trained MobileNetV2 (1000 classes, 1 epoch)](https://www.kaggle.com/socathie/pre-trained-mobilenetv2-1000-classes-1-epoch). The code is using \"categorical_accuracy\" as a metric, but the competition's evaluation metric is \"Global Average Precision\"", "1006780": "Thanks @trikialaaa . What would you suggest I do? As metrics aren't used for training a model I assume that using \"categorical_accuracy\" is just giving me unrealistic expectations for my score rather than causing something to actually be wrong with the model. I would still expect my model to get a score above 0.000 though?", "1007926": "Same problem here. Did you solve this problem?", "1008616": "guagugu No I haven't solved it yet. I see you have managed to get a score on the board though, did you manage to solve it?", "1008760": "That score is from another kernel, the 0 PB score problem is still there..." }, "source": "meta" }