Has anyone tried EfficientDet? If yes, could you please share your experience? Would it be nice to spend time on that?
", "messages": [ { "id": "1357560", "postDate": "06/19/2021 19:11:22", "content": "Has anyone tried EfficientDet? If yes, could you please share your experience? Would it be nice to spend time on that?
", "rawMarkdown": "Has anyone tried EfficientDet? If yes, could you please share your experience? Would it be nice to spend time on that?", "votes": null }, { "id": "1357683", "postDate": "06/19/2021 22:05:39", "content": "Baseline model with default [corrected] NMS post processing (opacity class only). Effdet library by Wrightman uses default IoU_threshold = 0.5 for batch_nms.
\n\n\nmAP IoU=0.50 = 0.56
\n
\n mAP IoU=0.50:0.95 = 0.18
Try the biggest image size that could fit into your GPU memory. Post-processing is probably needed (and will increase mAP significantly)
", "rawMarkdown": "Baseline model with default [corrected] NMS post processing (opacity class only). Effdet library by Wrightman uses default IoU_threshold = 0.5 for batch_nms.\n>mAP IoU=0.50 = 0.56\n>mAP IoU=0.50:0.95 = 0.18\n\nTry the biggest image size that could fit into your GPU memory. Post-processing is probably needed (and will increase mAP significantly)", "votes": null }, { "id": "1357948", "postDate": "06/20/2021 05:04:43", "content": "seems to be better than vanilla yolov5
", "rawMarkdown": "seems to be better than vanilla yolov5", "votes": null }, { "id": "1357962", "postDate": "06/20/2021 05:20:00", "content": "Yeah, better than I thought. And I manually checked some predictions: most of them are close to the ground truth & make sense.
\nOne thing: the confidence score for bbox prediction is generally very low, highest ~ 0.5. This may be related to the small training set. Or it could be that the negative samples (image w/o bbox) penalize the loss too much by default.
", "rawMarkdown": "Yeah, better than I thought. And I manually checked some predictions: most of them are close to the ground truth & make sense. \n\nOne thing: the confidence score for bbox prediction is generally very low, highest ~ 0.5. This may be related to the small training set. Or it could be that the negative samples (image w/o bbox) penalize the loss too much by default.", "votes": null }, { "id": "1358572", "postDate": "06/20/2021 15:27:39", "content": "@hengck23 Did you also try yolov5 with TTA?
", "rawMarkdown": "hengck23 Did you also try yolov5 with TTA?", "votes": null }, { "id": "1358575", "postDate": "06/20/2021 15:29:11", "content": "Thanks for the info @houndcl. Which image size did you use and obtain these results?
", "rawMarkdown": "Thanks for the info @houndcl. Which image size did you use and obtain these results?", "votes": null }, { "id": "1358628", "postDate": "06/20/2021 16:32:40", "content": "896 x 896 with EfficientDet-D3. Not testing bigger model but I assume that D4 and up won't improve the LB a lot, as there are few small objects and the information encoded in the 6k X-ray images is limited.
", "rawMarkdown": "896 x 896 with EfficientDet-D3. Not testing bigger model but I assume that D4 and up won't improve the LB a lot, as there are few small objects and the information encoded in the 6k X-ray images is limited.", "votes": null }, { "id": "1358848", "postDate": "06/20/2021 20:14:17", "content": "@houndcl I'm trying to train it but I saw that my train loss in the first epochs start very high: from near 130 and goes down to near 10-20 at the end of the first epoch, valid loss is near 1 at the end of epoch one. I thought this is weird. Did you have the same experience w/ EfficientDet? Do you think I'm missing something?
", "rawMarkdown": "houndcl I'm trying to train it but I saw that my train loss in the first epochs start very high: from near 130 and goes down to near 10-20 at the end of the first epoch, valid loss is near 1 at the end of epoch one. I thought this is weird. Did you have the same experience w/ EfficientDet? Do you think I'm missing something?", "votes": null }, { "id": "1359770", "postDate": "06/21/2021 14:05:38", "content": "Same experience. Det is not easy to train. I am using fastai's one cycle fit, and starting with low lr will help stabilize the training. Don't panic on seeing loss > 1.5. It is crossing the local minimum, and loss will drop significantly during the cooling phase.
", "rawMarkdown": "Same experience. Det is not easy to train. I am using fastai's one cycle fit, and starting with low lr will help stabilize the training. Don't panic on seeing loss > 1.5. It is crossing the local minimum, and loss will drop significantly during the cooling phase.", "votes": null }, { "id": "1359816", "postDate": "06/21/2021 14:45:21", "content": "That's interesting. I'd give OneCycle a shot and see how it helps. Btw, after 3 epochs I reached 0.8 valid loss and the predicted bboxes seem reasonable. I've not yet implemented mAP to report quantitively but I'll do it soon.
", "rawMarkdown": "That's interesting. I'd give OneCycle a shot and see how it helps. Btw, after 3 epochs I reached 0.8 valid loss and the predicted bboxes seem reasonable. I've not yet implemented mAP to report quantitively but I'll do it soon.", "votes": null }, { "id": "1359818", "postDate": "06/21/2021 14:49:37", "content": "0.8 is about the same loss as mine. Limit of vanilla model is likely around 0.77, which translates to AP@0.5 ~ 0.57.
", "rawMarkdown": "0.8 is about the same loss as mine. Limit of vanilla model is likely around 0.77, which translates to AP@0.5 ~ 0.57.", "votes": null }, { "id": "1359880", "postDate": "06/21/2021 15:56:35", "content": "Are you referring to any effdet code? I am facing difficulties
", "rawMarkdown": "Are you referring to any effdet code? I am facing difficulties", "votes": null } ], "comments": [ { "id": 1357683, "author_name": "houndcl", "author_url": "", "post_date": "06/19/2021 22:05:39", "content": "Baseline model with default [corrected] NMS post processing (opacity class only). Effdet library by Wrightman uses default IoU_threshold = 0.5 for batch_nms.
\n\n\nmAP IoU=0.50 = 0.56
\n
\n mAP IoU=0.50:0.95 = 0.18
Try the biggest image size that could fit into your GPU memory. Post-processing is probably needed (and will increase mAP significantly)
", "votes": null, "replies": [ { "id": 1357948, "author_name": "hengck23", "author_url": "", "post_date": "06/20/2021 05:04:43", "content": "seems to be better than vanilla yolov5
", "votes": null, "replies": [] }, { "id": 1357962, "author_name": "houndcl", "author_url": "", "post_date": "06/20/2021 05:20:00", "content": "Yeah, better than I thought. And I manually checked some predictions: most of them are close to the ground truth & make sense.
\nOne thing: the confidence score for bbox prediction is generally very low, highest ~ 0.5. This may be related to the small training set. Or it could be that the negative samples (image w/o bbox) penalize the loss too much by default.
", "votes": null, "replies": [] }, { "id": 1358572, "author_name": "eakdag", "author_url": "", "post_date": "06/20/2021 15:27:39", "content": "@hengck23 Did you also try yolov5 with TTA?
", "votes": null, "replies": [] }, { "id": 1358575, "author_name": "eakdag", "author_url": "", "post_date": "06/20/2021 15:29:11", "content": "Thanks for the info @houndcl. Which image size did you use and obtain these results?
", "votes": null, "replies": [] }, { "id": 1358628, "author_name": "houndcl", "author_url": "", "post_date": "06/20/2021 16:32:40", "content": "896 x 896 with EfficientDet-D3. Not testing bigger model but I assume that D4 and up won't improve the LB a lot, as there are few small objects and the information encoded in the 6k X-ray images is limited.
", "votes": null, "replies": [] }, { "id": 1358848, "author_name": "moeinshariatnia", "author_url": "", "post_date": "06/20/2021 20:14:17", "content": "@houndcl I'm trying to train it but I saw that my train loss in the first epochs start very high: from near 130 and goes down to near 10-20 at the end of the first epoch, valid loss is near 1 at the end of epoch one. I thought this is weird. Did you have the same experience w/ EfficientDet? Do you think I'm missing something?
", "votes": null, "replies": [] }, { "id": 1359770, "author_name": "houndcl", "author_url": "", "post_date": "06/21/2021 14:05:38", "content": "Same experience. Det is not easy to train. I am using fastai's one cycle fit, and starting with low lr will help stabilize the training. Don't panic on seeing loss > 1.5. It is crossing the local minimum, and loss will drop significantly during the cooling phase.
", "votes": null, "replies": [] }, { "id": 1359816, "author_name": "moeinshariatnia", "author_url": "", "post_date": "06/21/2021 14:45:21", "content": "That's interesting. I'd give OneCycle a shot and see how it helps. Btw, after 3 epochs I reached 0.8 valid loss and the predicted bboxes seem reasonable. I've not yet implemented mAP to report quantitively but I'll do it soon.
", "votes": null, "replies": [] }, { "id": 1359818, "author_name": "houndcl", "author_url": "", "post_date": "06/21/2021 14:49:37", "content": "0.8 is about the same loss as mine. Limit of vanilla model is likely around 0.77, which translates to AP@0.5 ~ 0.57.
", "votes": null, "replies": [] }, { "id": 1359880, "author_name": "mrinath", "author_url": "", "post_date": "06/21/2021 15:56:35", "content": "Are you referring to any effdet code? I am facing difficulties
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1357560": "Has anyone tried EfficientDet? If yes, could you please share your experience? Would it be nice to spend time on that?", "1357683": "Baseline model with default [corrected] NMS post processing (opacity class only). Effdet library by Wrightman uses default IoU_threshold = 0.5 for batch_nms.\n>mAP IoU=0.50 = 0.56\n>mAP IoU=0.50:0.95 = 0.18\n\nTry the biggest image size that could fit into your GPU memory. Post-processing is probably needed (and will increase mAP significantly)", "1357948": "seems to be better than vanilla yolov5", "1357962": "Yeah, better than I thought. And I manually checked some predictions: most of them are close to the ground truth & make sense. \n\nOne thing: the confidence score for bbox prediction is generally very low, highest ~ 0.5. This may be related to the small training set. Or it could be that the negative samples (image w/o bbox) penalize the loss too much by default.", "1358572": "hengck23 Did you also try yolov5 with TTA?", "1358575": "Thanks for the info @houndcl. Which image size did you use and obtain these results?", "1358628": "896 x 896 with EfficientDet-D3. Not testing bigger model but I assume that D4 and up won't improve the LB a lot, as there are few small objects and the information encoded in the 6k X-ray images is limited.", "1358848": "houndcl I'm trying to train it but I saw that my train loss in the first epochs start very high: from near 130 and goes down to near 10-20 at the end of the first epoch, valid loss is near 1 at the end of epoch one. I thought this is weird. Did you have the same experience w/ EfficientDet? Do you think I'm missing something?", "1359770": "Same experience. Det is not easy to train. I am using fastai's one cycle fit, and starting with low lr will help stabilize the training. Don't panic on seeing loss > 1.5. It is crossing the local minimum, and loss will drop significantly during the cooling phase.", "1359816": "That's interesting. I'd give OneCycle a shot and see how it helps. Btw, after 3 epochs I reached 0.8 valid loss and the predicted bboxes seem reasonable. I've not yet implemented mAP to report quantitively but I'll do it soon.", "1359818": "0.8 is about the same loss as mine. Limit of vanilla model is likely around 0.77, which translates to AP@0.5 ~ 0.57.", "1359880": "Are you referring to any effdet code? I am facing difficulties" }, "source": "meta" }