\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F11197532%2Ff87d46b3adb41618cd855e2364813e27%2Fpredict.png?generation=1661336742733126&alt=media\")
In my local machince, the model performace is good and the code is the same as the submited, what's the possible reason for it?
", "messages": [ { "id": "1910630", "postDate": "08/23/2022 15:09:51", "content": "In my local machince, the model performace is good and the code is the same as the submited, what's the possible reason for it?
", "rawMarkdown": "In my local machince, the model performace is good and the code is the same as the submited, what's the possible reason for it?", "votes": null }, { "id": "1910945", "postDate": "08/23/2022 19:48:22", "content": "In your local machine you are calculating the model performance on the test data from the public data… On the other hand on lb your model performance is calculated on hidden test data… Now the possible reason is that:
\n0.) You are overfitting badly on the train set
\n1.) You haven't split the data properly
\n2.) The distribution of test set is different from training set (but given the people achieving good score on lb, I'd rule out this possibility)
Even for a poor model I'd expect a bit more than 0.04, perhaps your preprocessing of the test image is poor eg. your train pipeline might be using BGR2RGB but your kaggle submission missed this , maybe image size related issues?
\nAnother could be your RLE function not working as intended.
I would check your submission on the test image they give us and compare to some of the posts in this thread: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/343117
", "rawMarkdown": "Even for a poor model I'd expect a bit more than 0.04, perhaps your preprocessing of the test image is poor eg. your train pipeline might be using BGR2RGB but your kaggle submission missed this , maybe image size related issues?\nAnother could be your RLE function not working as intended.\n\nI would check your submission on the test image they give us and compare to some of the posts in this thread: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/343117", "votes": null }, { "id": "1911777", "postDate": "08/24/2022 09:57:41", "content": "@maxvandijck I think my model prediction is not quite different from those posts, thanks very much if you can help me check with it! That's really strange, I set the mask to random values to submit and the lb is 0.12😂, better than mine
", "rawMarkdown": "maxvandijck I think my model prediction is not quite different from those posts, thanks very much if you can help me check with it! That's really strange, I set the mask to random values to submit and the lb is 0.12😂, better than mine", "votes": null }, { "id": "1911809", "postDate": "08/24/2022 10:26:25", "content": "That's my model predict on the test image:
\n
Check if your rle is transposed
", "rawMarkdown": "Check if your rle is transposed", "votes": null }, { "id": "1912096", "postDate": "08/24/2022 14:00:01", "content": "Thanks. I have been thinking about it and here is my rle code:
\ndef rle_encode(img):
\n pixels = img.T.flatten()
\n pixels[0] = 0
\n pixels[-1] = 0
\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 2
\n runs[1::2] -= runs[::2]
\n return ' '.join(str(x) for x in runs)
I have tried to drop .T and the result is still very low ☹️.
", "rawMarkdown": "Thanks. I have been thinking about it and here is my rle code:\ndef rle_encode(img):\n pixels = img.T.flatten()\n pixels[0] = 0\n pixels[-1] = 0\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 2\n runs[1::2] -= runs[::2]\n return ' '.join(str(x) for x in runs)\n\nI have tried to drop **.T** and the result is still very low ☹️.", "votes": null }, { "id": "1912411", "postDate": "08/24/2022 17:48:43", "content": "That looks fine. May be something happens when running on the public test set which results in blank rle 🤔
", "rawMarkdown": "That looks fine. May be something happens when running on the public test set which results in blank rle 🤔", "votes": null }, { "id": "1912449", "postDate": "08/24/2022 18:17:53", "content": "This is a quite good prediction for the hubmap spleen. but your score is quite low.
\nHubmap scales should be considered when making a submission
Solved! Thanks everyone. What I did is avoiding reading test.csv，all the information are obtained from dir test_images
", "rawMarkdown": "Solved! Thanks everyone. What I did is avoiding reading test.csv，all the information are obtained from dir test_images", "votes": null }, { "id": "1913257", "postDate": "08/25/2022 09:02:04", "content": "there is some bug in you code.
\nusing csv file is ok for me.
\n'../input/hubmap-organ-segmentation/test.csv'
I doubt the possible bug is that I used 'os.path.join' to create the path of test.csv instead of '../input/hubmap-organ-segmentation/test.csv', but I haven't checked it.
", "rawMarkdown": "I doubt the possible bug is that I used 'os.path.join' to create the path of test.csv instead of '../input/hubmap-organ-segmentation/test.csv', but I haven't checked it.", "votes": null } ], "comments": [ { "id": 1910945, "author_name": "anitho2910", "author_url": "", "post_date": "08/23/2022 19:48:22", "content": "In your local machine you are calculating the model performance on the test data from the public data… On the other hand on lb your model performance is calculated on hidden test data… Now the possible reason is that:
\n0.) You are overfitting badly on the train set
\n1.) You haven't split the data properly
\n2.) The distribution of test set is different from training set (but given the people achieving good score on lb, I'd rule out this possibility)
Even for a poor model I'd expect a bit more than 0.04, perhaps your preprocessing of the test image is poor eg. your train pipeline might be using BGR2RGB but your kaggle submission missed this , maybe image size related issues?
\nAnother could be your RLE function not working as intended.
I would check your submission on the test image they give us and compare to some of the posts in this thread: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/343117
", "votes": null, "replies": [] }, { "id": 1911777, "author_name": "resonwang", "author_url": "", "post_date": "08/24/2022 09:57:41", "content": "@maxvandijck I think my model prediction is not quite different from those posts, thanks very much if you can help me check with it! That's really strange, I set the mask to random values to submit and the lb is 0.12😂, better than mine
", "votes": null, "replies": [] }, { "id": 1911809, "author_name": "resonwang", "author_url": "", "post_date": "08/24/2022 10:26:25", "content": "That's my model predict on the test image:
\n
This is a quite good prediction for the hubmap spleen. but your score is quite low.
\nHubmap scales should be considered when making a submission
Check if your rle is transposed
", "votes": null, "replies": [ { "id": 1912096, "author_name": "resonwang", "author_url": "", "post_date": "08/24/2022 14:00:01", "content": "Thanks. I have been thinking about it and here is my rle code:
\ndef rle_encode(img):
\n pixels = img.T.flatten()
\n pixels[0] = 0
\n pixels[-1] = 0
\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 2
\n runs[1::2] -= runs[::2]
\n return ' '.join(str(x) for x in runs)
I have tried to drop .T and the result is still very low ☹️.
", "votes": null, "replies": [] }, { "id": 1912411, "author_name": "riyasvk", "author_url": "", "post_date": "08/24/2022 17:48:43", "content": "That looks fine. May be something happens when running on the public test set which results in blank rle 🤔
", "votes": null, "replies": [] } ] }, { "id": 1912911, "author_name": "resonwang", "author_url": "", "post_date": "08/25/2022 03:27:29", "content": "Solved! Thanks everyone. What I did is avoiding reading test.csv，all the information are obtained from dir test_images
", "votes": null, "replies": [ { "id": 1913257, "author_name": "hengck23", "author_url": "", "post_date": "08/25/2022 09:02:04", "content": "there is some bug in you code.
\nusing csv file is ok for me.
\n'../input/hubmap-organ-segmentation/test.csv'
I doubt the possible bug is that I used 'os.path.join' to create the path of test.csv instead of '../input/hubmap-organ-segmentation/test.csv', but I haven't checked it.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1910630": "In my local machince, the model performace is good and the code is the same as the submited, what's the possible reason for it?", "1910945": "In your local machine you are calculating the model performance on the test data from the public data... On the other hand on lb your model performance is calculated on hidden test data... Now the possible reason is that:\n0.) You are overfitting badly on the train set\n1.) You haven't split the data properly\n2.) The distribution of test set is different from training set (but given the people achieving good score on lb, I'd rule out this possibility)", "1911009": "Even for a poor model I'd expect a bit more than 0.04, perhaps your preprocessing of the test image is poor eg. your train pipeline might be using BGR2RGB but your kaggle submission missed this , maybe image size related issues?\nAnother could be your RLE function not working as intended.\n\nI would check your submission on the test image they give us and compare to some of the posts in this thread: https://www.kaggle.com/competitions/hubmap-organ-segmentation/discussion/343117", "1911777": "maxvandijck I think my model prediction is not quite different from those posts, thanks very much if you can help me check with it! That's really strange, I set the mask to random values to submit and the lb is 0.12😂, better than mine", "1911809": "That's my model predict on the test image:\n", "1912015": "Check if your rle is transposed", "1912096": "Thanks. I have been thinking about it and here is my rle code:\ndef rle_encode(img):\n pixels = img.T.flatten()\n pixels[0] = 0\n pixels[-1] = 0\n runs = np.where(pixels[1:] != pixels[:-1])[0] + 2\n runs[1::2] -= runs[::2]\n return ' '.join(str(x) for x in runs)\n\nI have tried to drop **.T** and the result is still very low ☹️.", "1912411": "That looks fine. May be something happens when running on the public test set which results in blank rle 🤔", "1912449": "This is a quite good prediction for the hubmap spleen. but your score is quite low. \nHubmap scales should be considered when making a submission", "1912911": "Solved! Thanks everyone. What I did is avoiding reading test.csv，all the information are obtained from dir test_images", "1913257": "there is some bug in you code.\nusing csv file is ok for me.\n'../input/hubmap-organ-segmentation/test.csv'", "1913929": "I doubt the possible bug is that I used 'os.path.join' to create the path of test.csv instead of '../input/hubmap-organ-segmentation/test.csv', but I haven't checked it." }, "source": "meta" }