The loading part of the input dataset is working fine on save, but is giving an error on submit.
\nI've lost about 5 days and 15 submits due to this problem.
\nDoes anyone have the same situation or know how to solve it?
The loading part of the input dataset is working fine on save, but is giving an error on submit.
\nI've lost about 5 days and 15 submits due to this problem.
\nDoes anyone have the same situation or know how to solve it?
What kind of error do you get?
\nMake sure you are reading the test images in your notebook, since the test images when submitting are different than the images in the available test set:
This is a synchronous rerun code competition. The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. Additionally, this competition also has two unique characteristics:\nLast thing I could think of is the submission file itself, does it have the correct column names and format. The columns should be names id and landmarks (plural!) and there should be a space between the landmark_id and confidence score.
id,landmarks\n000088da12d664db,8815 0.03\n0001623c6d808702,\n0001bbb682d45002,5328 0.5\nbest course of action in this case would be to do Unit testing of each function you are using. There was a similar thread on the Google Image Retrieval competition. You can refer to some of the ideas there as well.
\nIn general for Code competitions when the submission gives such an error, the only way I out is to do a thorough unit testing to know the true cause of the exception.
\nHope this helps and all the best for the rest of the competition. 👍
", "rawMarkdown": "best course of action in this case would be to do Unit testing of each function you are using. There was a similar [thread](https://www.kaggle.com/c/landmark-retrieval-2021/discussion/269592) on the [Google Image Retrieval competition](https://www.kaggle.com/c/landmark-retrieval-2021/overview). You can refer to some of the ideas there as well. \n\nIn general for Code competitions when the submission gives such an error, the only way I out is to do a thorough unit testing to know the true cause of the exception.\n\nHope this helps and all the best for the rest of the competition. 👍", "votes": null }, { "id": "1505954", "postDate": "09/07/2021 16:59:18", "content": "Apparently, you're running out of RAM. I had the same error when I was saving all the probabilities for each sample and only then counting the argmax and confidence. I solved this by saving only argmax and confidence from each prediction.
", "rawMarkdown": "Apparently, you're running out of RAM. I had the same error when I was saving all the probabilities for each sample and only then counting the argmax and confidence. I solved this by saving only argmax and confidence from each prediction.", "votes": null }, { "id": "1507126", "postDate": "09/08/2021 21:01:54", "content": "You may need to think about the assumptions you have in your notebook. I got an exception error at first, however, I quickly fixed my exception error by adjusting some assumptions I made. For example, Train.csv is different during the re-run. So just be careful about your assumptions.
\nIf anyone is running out of memory during inference, that can easily solved by inferencing on batches of test instead of trying to inference on the entire test set.
", "rawMarkdown": "You may need to think about the assumptions you have in your notebook. I got an exception error at first, however, I quickly fixed my exception error by adjusting some assumptions I made. For example, Train.csv is different during the re-run. So just be careful about your assumptions. \n\nIf anyone is running out of memory during inference, that can easily solved by inferencing on batches of test instead of trying to inference on the entire test set.", "votes": null }, { "id": "1508903", "postDate": "09/10/2021 17:09:31", "content": "Thank you Tim Yee!
\nAs you commented, the error was caused by the fact that train.csv is different between save and submit.
\nI don't know why it's spec'd this way, but thanks to you, I can now enter this competition!
Only the test dataset is different or also the train dataset is different?
", "rawMarkdown": "Only the test dataset is different or also the train dataset is different?", "votes": null }, { "id": "1530244", "postDate": "10/01/2021 03:49:46", "content": "Are you sure that the train dataset is different too? I think only the test dataset should be different.
", "rawMarkdown": "Are you sure that the train dataset is different too? I think only the test dataset should be different.", "votes": null }, { "id": "1530633", "postDate": "10/01/2021 09:57:39", "content": "The private train dataset is also different as stated in the data description. The private training set is much smaller, with only 100K images, instead of 1.5M.
\nTo facilitate recognition-by-retrieval approaches, the private training set contains only a 100k subset of the total public training set. This 100k subset contains all of the training set images associated with the landmarks in the private test set. You may still attach the full training set as an external data set if you wish.\nI have the same error too. When I replace: y_pred = model.predict(test_image_ds)
\nwith this line: y_pred = np.random.random((global_test_dataset_size, n_classes))
\nthe error disappears.
I discovered another strange behavior.
\nWhen I save notebook, close it, go to the leaderboard and create a new submission from there - the submission fails after 5 seconds.
\nWhen I click submit from the very notebook - the submission succeeds and I also can see the submitting logs in a real time.
I don't know the detailed cause of your error, but it could be a problem on the kaggle side.
", "rawMarkdown": "I don't know the detailed cause of your error, but it could be a problem on the kaggle side.", "votes": null } ], "comments": [ { "id": 1504204, "author_name": "markwijkhuizen", "author_url": "", "post_date": "09/06/2021 07:09:27", "content": "What kind of error do you get?
\nMake sure you are reading the test images in your notebook, since the test images when submitting are different than the images in the available test set:
This is a synchronous rerun code competition. The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. Additionally, this competition also has two unique characteristics:\nLast thing I could think of is the submission file itself, does it have the correct column names and format. The columns should be names id and landmarks (plural!) and there should be a space between the landmark_id and confidence score.
id,landmarks\n000088da12d664db,8815 0.03\n0001623c6d808702,\n0001bbb682d45002,5328 0.5\nOnly the test dataset is different or also the train dataset is different?
", "votes": null, "replies": [] }, { "id": 1530633, "author_name": "markwijkhuizen", "author_url": "", "post_date": "10/01/2021 09:57:39", "content": "The private train dataset is also different as stated in the data description. The private training set is much smaller, with only 100K images, instead of 1.5M.
\nTo facilitate recognition-by-retrieval approaches, the private training set contains only a 100k subset of the total public training set. This 100k subset contains all of the training set images associated with the landmarks in the private test set. You may still attach the full training set as an external data set if you wish.\nbest course of action in this case would be to do Unit testing of each function you are using. There was a similar thread on the Google Image Retrieval competition. You can refer to some of the ideas there as well.
\nIn general for Code competitions when the submission gives such an error, the only way I out is to do a thorough unit testing to know the true cause of the exception.
\nHope this helps and all the best for the rest of the competition. 👍
", "votes": null, "replies": [] }, { "id": 1505954, "author_name": "vadimtimakin", "author_url": "", "post_date": "09/07/2021 16:59:18", "content": "Apparently, you're running out of RAM. I had the same error when I was saving all the probabilities for each sample and only then counting the argmax and confidence. I solved this by saving only argmax and confidence from each prediction.
", "votes": null, "replies": [] }, { "id": 1507126, "author_name": "teeyee314", "author_url": "", "post_date": "09/08/2021 21:01:54", "content": "You may need to think about the assumptions you have in your notebook. I got an exception error at first, however, I quickly fixed my exception error by adjusting some assumptions I made. For example, Train.csv is different during the re-run. So just be careful about your assumptions.
\nIf anyone is running out of memory during inference, that can easily solved by inferencing on batches of test instead of trying to inference on the entire test set.
", "votes": null, "replies": [ { "id": 1508903, "author_name": "kalfirst", "author_url": "", "post_date": "09/10/2021 17:09:31", "content": "Thank you Tim Yee!
\nAs you commented, the error was caused by the fact that train.csv is different between save and submit.
\nI don't know why it's spec'd this way, but thanks to you, I can now enter this competition!
Are you sure that the train dataset is different too? I think only the test dataset should be different.
", "votes": null, "replies": [] } ] }, { "id": 1530642, "author_name": "markbquant", "author_url": "", "post_date": "10/01/2021 10:19:03", "content": "I have the same error too. When I replace: y_pred = model.predict(test_image_ds)
\nwith this line: y_pred = np.random.random((global_test_dataset_size, n_classes))
\nthe error disappears.
I discovered another strange behavior.
\nWhen I save notebook, close it, go to the leaderboard and create a new submission from there - the submission fails after 5 seconds.
\nWhen I click submit from the very notebook - the submission succeeds and I also can see the submitting logs in a real time.
I don't know the detailed cause of your error, but it could be a problem on the kaggle side.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1503896": "The loading part of the input dataset is working fine on save, but is giving an error on submit.\nI've lost about 5 days and 15 submits due to this problem.\nDoes anyone have the same situation or know how to solve it?", "1504204": "What kind of error do you get?\nMake sure you are reading the test images in your notebook, since the test images when submitting are different than the images in the [available test set](https://www.kaggle.com/c/landmark-recognition-2021/data):\n\n```\nThis is a synchronous rerun code competition. The provided test set is a representative set of files to demonstrate the format of the private test set. When you submit your notebook, Kaggle will rerun your code on the private dataset. Additionally, this competition also has two unique characteristics:\n```\n\nLast thing I could think of is the [submission file](https://www.kaggle.com/c/landmark-recognition-2021/overview/evaluation) itself, does it have the correct column names and format. The columns should be names `id` and `landmarks` (plural!) and there should be a space between the landmark_id and confidence score.\n\n```\nid,landmarks\n000088da12d664db,8815 0.03\n0001623c6d808702,\n0001bbb682d45002,5328 0.5\n```", "1504321": "best course of action in this case would be to do Unit testing of each function you are using. There was a similar [thread](https://www.kaggle.com/c/landmark-retrieval-2021/discussion/269592) on the [Google Image Retrieval competition](https://www.kaggle.com/c/landmark-retrieval-2021/overview). You can refer to some of the ideas there as well. \n\nIn general for Code competitions when the submission gives such an error, the only way I out is to do a thorough unit testing to know the true cause of the exception.\n\nHope this helps and all the best for the rest of the competition. 👍", "1505954": "Apparently, you're running out of RAM. I had the same error when I was saving all the probabilities for each sample and only then counting the argmax and confidence. I solved this by saving only argmax and confidence from each prediction.", "1507126": "You may need to think about the assumptions you have in your notebook. I got an exception error at first, however, I quickly fixed my exception error by adjusting some assumptions I made. For example, Train.csv is different during the re-run. So just be careful about your assumptions. \n\nIf anyone is running out of memory during inference, that can easily solved by inferencing on batches of test instead of trying to inference on the entire test set.", "1508903": "Thank you Tim Yee!\nAs you commented, the error was caused by the fact that train.csv is different between save and submit.\nI don't know why it's spec'd this way, but thanks to you, I can now enter this competition!", "1530242": "Only the test dataset is different or also the train dataset is different?", "1530244": "Are you sure that the train dataset is different too? I think only the test dataset should be different.", "1530633": "The private train dataset is also different as stated in the [data description](https://www.kaggle.com/c/landmark-recognition-2021/data). The private training set is much smaller, with only 100K images, instead of 1.5M.\n\n```\nTo facilitate recognition-by-retrieval approaches, the private training set contains only a 100k subset of the total public training set. This 100k subset contains all of the training set images associated with the landmarks in the private test set. You may still attach the full training set as an external data set if you wish.\n```", "1530642": "I have the same error too. When I replace: `y_pred = model.predict(test_image_ds)`\nwith this line: `y_pred = np.random.random((global_test_dataset_size, n_classes))` \nthe error disappears.", "1544134": "I discovered another strange behavior.\nWhen I save notebook, close it, go to the leaderboard and create a new submission from there - the submission fails after 5 seconds.\nWhen I click submit from the very notebook - the submission succeeds and I also can see the submitting logs in a real time.", "1545445": "I don't know the detailed cause of your error, but it could be a problem on the kaggle side." }, "source": "meta" }