These days massive progresses are in LB.
\nSome notebooks say that the larger input image size leads higher score.
\nHowever, as comments in those notebooks say, the bigger img size needs the more gpu power, memory, and time to train and predict.
\nSo, I think that splitting the original image into 4 or 9 grid small images and resizing them to original size will also give us high lb score.
What do you think about my Idea?
\n[Update] I use SAHI lib to split test images.But my trained model(using not split image to train) score dropped.
\nNext, I split my train/val dataset and I am training yolo model with these dataset.
These days massive progresses are in LB.
\nSome notebooks say that the larger input image size leads higher score.
\nHowever, as comments in those notebooks say, the bigger img size needs the more gpu power, memory, and time to train and predict.
\nSo, I think that splitting the original image into 4 or 9 grid small images and resizing them to original size will also give us high lb score.
What do you think about my Idea?
\n[Update] I use SAHI lib to split test images.But my trained model(using not split image to train) score dropped.
\nNext, I split my train/val dataset and I am training yolo model with these dataset.
Sounds interesting. but how can we deal with COTS divided by grid? There is a chance to detect the same COTS twice without advanced postprocessing. In that case, it is possible to union such predictions. But what about an opposite case where the model will not detect half of COTS?
\nAnyway, I believe that such an approach (along with grid augmentation) will produce exponentially more data samples, so it should reduce overfitting, isn't it?
", "rawMarkdown": "Sounds interesting. but how can we deal with COTS divided by grid? There is a chance to detect the same COTS twice without advanced postprocessing. In that case, it is possible to union such predictions. But what about an opposite case where the model will not detect half of COTS?\n\nAnyway, I believe that such an approach (along with grid augmentation) will produce exponentially more data samples, so it should reduce overfitting, isn't it?", "votes": null }, { "id": "1662275", "postDate": "01/24/2022 06:48:39", "content": "I'm so sorry for replying so late.
\nThanks for your helpful suggestions!
\nI'll think more about the problem you said…
I would suggest a naive approach to deal with half of COTS. What do you think about splitting images into grids with overlaps? I mean, each cell will overlap a neighbor for N pixels. We may choose N according to the average COTS size. Working like that ensures that each COTS will hit at least one cell. Having that, we need just to union prediction along cell edges.
\nWhat is your opinion about that idea?
i have done some exps,but they droped the lb…
", "rawMarkdown": "i have done some exps,but they droped the lb...", "votes": null }, { "id": "1662465", "postDate": "01/24/2022 09:45:06", "content": "Is it due to missing COTS on the cell edges?
", "rawMarkdown": "Is it due to missing COTS on the cell edges?", "votes": null }, { "id": "1662598", "postDate": "01/24/2022 12:14:47", "content": "Thank you for replying!
\nBefore I saw your reply, I had found this notebook which uses SAHI lib.
\nThis library does almost the same thing I wanted to do.
\nThis lib splits an image with overlapping (as @meowmeowmeowmeowmeow said!).
And I tried to use this library but the lb score dropped(as @junyun1002 said)…
\nI watched the predicted vs label anno video, but I think the reason that the scores dropped is others…
Wait a little bit …. I am investigating …. I see where potential problem could be … Checking :)
", "rawMarkdown": "Wait a little bit .... I am investigating .... I see where potential problem could be ... Checking :)", "votes": null }, { "id": "1662772", "postDate": "01/24/2022 14:16:21", "content": "Oh! Hi @remekkinas! Thank you for your very very helpful notebooks and discussions!!
\nI learned a lot from them and they saved much time for me!
I used public best single model weight(as this https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer notebook uses.) and changed img_size(1920~9000).
\nI think train images are very different from the images which my model predicts(for example, cots size).
\nSo I will change my train data to more similar to the data for prediction!
\nI believe some experiments lead better score!
Great! Let us know if you find way to jump. Thank you!
", "rawMarkdown": "Great! Let us know if you find way to jump. Thank you!", "votes": null }, { "id": "1663575", "postDate": "01/25/2022 08:20:11", "content": "Given my observation that inference size is proportionate to the square of the resolution, grid-wise inference will be just as slow (if not slower) than single inference of the whole image.
\nPersonally, I found SAHI gave lower score than single inference.
", "rawMarkdown": "Given my observation that inference size is proportionate to the square of the resolution, grid-wise inference will be just as slow (if not slower) than single inference of the whole image.\n\nPersonally, I found SAHI gave lower score than single inference.", "votes": null }, { "id": "1663577", "postDate": "01/25/2022 08:20:43", "content": "correction: Inference time is proportionate to square of resolution
", "rawMarkdown": "correction: Inference time is proportionate to square of resolution", "votes": null } ], "comments": [ { "id": 1653059, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "01/17/2022 07:58:52", "content": "Sounds interesting. but how can we deal with COTS divided by grid? There is a chance to detect the same COTS twice without advanced postprocessing. In that case, it is possible to union such predictions. But what about an opposite case where the model will not detect half of COTS?
\nAnyway, I believe that such an approach (along with grid augmentation) will produce exponentially more data samples, so it should reduce overfitting, isn't it?
", "votes": null, "replies": [ { "id": 1662275, "author_name": "umepon0626", "author_url": "", "post_date": "01/24/2022 06:48:39", "content": "I'm so sorry for replying so late.
\nThanks for your helpful suggestions!
\nI'll think more about the problem you said…
I would suggest a naive approach to deal with half of COTS. What do you think about splitting images into grids with overlaps? I mean, each cell will overlap a neighbor for N pixels. We may choose N according to the average COTS size. Working like that ensures that each COTS will hit at least one cell. Having that, we need just to union prediction along cell edges.
\nWhat is your opinion about that idea?
i have done some exps,but they droped the lb…
", "votes": null, "replies": [] }, { "id": 1662465, "author_name": "meowmeowmeowmeowmeow", "author_url": "", "post_date": "01/24/2022 09:45:06", "content": "Is it due to missing COTS on the cell edges?
", "votes": null, "replies": [] }, { "id": 1662598, "author_name": "umepon0626", "author_url": "", "post_date": "01/24/2022 12:14:47", "content": "Thank you for replying!
\nBefore I saw your reply, I had found this notebook which uses SAHI lib.
\nThis library does almost the same thing I wanted to do.
\nThis lib splits an image with overlapping (as @meowmeowmeowmeowmeow said!).
And I tried to use this library but the lb score dropped(as @junyun1002 said)…
\nI watched the predicted vs label anno video, but I think the reason that the scores dropped is others…
Wait a little bit …. I am investigating …. I see where potential problem could be … Checking :)
", "votes": null, "replies": [] }, { "id": 1662772, "author_name": "umepon0626", "author_url": "", "post_date": "01/24/2022 14:16:21", "content": "Oh! Hi @remekkinas! Thank you for your very very helpful notebooks and discussions!!
\nI learned a lot from them and they saved much time for me!
I used public best single model weight(as this https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer notebook uses.) and changed img_size(1920~9000).
\nI think train images are very different from the images which my model predicts(for example, cots size).
\nSo I will change my train data to more similar to the data for prediction!
\nI believe some experiments lead better score!
Great! Let us know if you find way to jump. Thank you!
", "votes": null, "replies": [] }, { "id": 1663575, "author_name": "alexchwong", "author_url": "", "post_date": "01/25/2022 08:20:11", "content": "Given my observation that inference size is proportionate to the square of the resolution, grid-wise inference will be just as slow (if not slower) than single inference of the whole image.
\nPersonally, I found SAHI gave lower score than single inference.
", "votes": null, "replies": [] }, { "id": 1663577, "author_name": "alexchwong", "author_url": "", "post_date": "01/25/2022 08:20:43", "content": "correction: Inference time is proportionate to square of resolution
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1653029": "These days massive progresses are in LB.\nSome notebooks say that the larger input image size leads higher score. \nHowever, as comments in those notebooks say, the bigger img size needs the more gpu power, memory, and time to train and predict.\nSo, I think that splitting the original image into 4 or 9 grid small images and resizing them to original size will also give us high lb score.\n\nWhat do you think about my Idea?\n\n[Update] I use [SAHI lib](https://github.com/obss/sahi) to split test images.But my trained model(using not split image to train) score dropped.\nNext, I split my train/val dataset and I am training yolo model with these dataset.", "1653059": "Sounds interesting. but how can we deal with COTS divided by grid? There is a chance to detect the same COTS twice without advanced postprocessing. In that case, it is possible to union such predictions. But what about an opposite case where the model will not detect half of COTS?\n\nAnyway, I believe that such an approach (along with grid augmentation) will produce exponentially more data samples, so it should reduce overfitting, isn't it?", "1662275": "I'm so sorry for replying so late. \nThanks for your helpful suggestions!\nI'll think more about the problem you said...", "1662282": "I would suggest a naive approach to deal with half of COTS. What do you think about splitting images into grids with overlaps? I mean, each cell will overlap a neighbor for N pixels. We may choose N according to the average COTS size. Working like that ensures that each COTS will hit at least one cell. Having that, we need just to union prediction along cell edges.\nWhat is your opinion about that idea?", "1662438": "i have done some exps,but they droped the lb...", "1662465": "Is it due to missing COTS on the cell edges?", "1662598": "Thank you for replying!\nBefore I saw your reply, I had found [this notebook](https://www.kaggle.com/remekkinas/sahi-slicing-aided-hyper-inference-yv5-and-yx) which uses [SAHI lib](https://github.com/obss/sahi).\nThis library does almost the same thing I wanted to do.\nThis lib splits an image with overlapping (as @meowmeowmeowmeowmeow said!).\n\nAnd I tried to use this library but the lb score dropped(as @junyun1002 said)...\nI watched the predicted vs label anno video, but I think the reason that the scores dropped is others...", "1662654": "Wait a little bit .... I am investigating .... I see where potential problem could be ... Checking :)", "1662772": "Oh! Hi @remekkinas! Thank you for your very very helpful notebooks and discussions!!\nI learned a lot from them and they saved much time for me!\n\nI used public best single model weight(as this https://www.kaggle.com/awsaf49/great-barrier-reef-yolov5-infer notebook uses.) and changed img_size(1920~9000).\n- img_size → public lb score\n- 1920 → **0.455**\n- 3000 → **0.489**\n- 5000 → **0.476**\n- 7000 → time over\n- 9000 → time over\n\nI think train images are very different from the images which my model predicts(for example, cots size).\nSo I will change my train data to more similar to the data for prediction!\nI believe some experiments lead better score!", "1662790": "Great! Let us know if you find way to jump. Thank you!", "1663575": "Given my observation that inference size is proportionate to the square of the resolution, grid-wise inference will be just as slow (if not slower) than single inference of the whole image.\n\nPersonally, I found SAHI gave lower score than single inference.", "1663577": "correction: Inference time is proportionate to square of resolution" }, "source": "meta" }