{ "id": 307634, "title": "28th to 40th: EDA on Bboxes was useful", "url": "/competitions/tensorflow-great-barrier-reef/discussion/307634", "author_name": "Jaideep", "post_date": "2022-02-15T04:03:19.258000", "votes": 15, "comment_count": 0, "views": 0, "content": "

I congratulate to all silver/gold medal holders. It was a quite a considerable shake up towards the end , yet the competition was exciting.
\nWe couldnt select from any of our top 3 solution ( that would have landed us at 40-45 position at private lb ) because of unavailability at the time of completion of our final submissions. Below is some interesting analysis of bboxes based on which we were to select our final submission .

\n

Here is the kernel doing that analysis
\nhttps://www.kaggle.com/jaideepvalani/eda-model-cv-check-rohit?scriptVersionId=87771195

\n

In this kernel I split the training data cots based on their areas so 0-1500- I regarded as tiny cots these cots were dominant not only in public lb but also in Training data , 1500-3500 small and there after it was mid size bboxes. Below is distribution.

\n
label  \n0     18582  -No Cots images\n1      2053\n2      1370\n3       791\n4       330\n5       256\n6        73\n7        22\n9         6\n8         4\n11        2\n12        2\n13        2\n
\n

If we see the distribution tiny cots were dominant in training data and likely 50 percent of private cot objects were also tiny so their impact on final score was not that high as it was in public LB so that would explain why model inference on high im size was working for many but private lb score would have dropped for those whose models were quite average at area label from 3 to 6 that might have contributed 30-45 percent of private LB score .

\n

Our best private solution kernel was mix of models best at public LB , and models doing best in area segment >2.
\nIOU area wise of our best model at public lb based on video id based split fold 2

\n
label\n1    0.637997\n2    0.627082\n3    0.822058\n4    0.784556\nName: iou, dtype: float64\n
\n

IOU area wise breakup of model not scoring so high on public LB , 58.9 with inference size 3600.

\n
label\n1    0.637223\n2    0.586083\n3    0.975207\n4    0.000023\n
\n

Some of our models performed well on high im size while some were performing on only 3600 so we chose to ensemble models performing well at different different im size.

\n

Best solution details
\nOur best solution on private lb was mix of 4 models,we couldnt select because of unavailability :(
\n1) Model with Best public score & Best public lb model using inference size -6400 best score at 9000 alone.
\n2) Remaining 2 at 3600 im size ( not so good public lb score ~58 60)
\nWe chose lower im size for best public lb scoring model because we feared for lot of FPs on rest of private LB data.

\n

CV split :
\n1) Video id based Splits : We used fold 2 as validation fold for best public lb score model . As video ids 0/1 validation fold contained lot of small cots when we checked BBoxes area wise distribution in fold . So we dint ensemble all of them just 1 and 2 to ensure we dont include underfit models in ensemble that would have caused lot false negatives .
\n2)Subsequences : We also Subsequence based splits 90:10 for 2 of models scoring around ~60

\n

Training details
\nYolov5s6
\nTraining im size=3000 to 3200
\nAugmentation : 1 mosaic , mixup 0.5 , UD, HFlip, little bit of CLAHE etc . Mosaic/mixup were crucial to train model with variety of data.

\n

What not worked to expectations

\n

1) Training on Style transfer generated datausing @hengck23 fantastic kernel
\n2) Pretraining using heavy augs like Copy paste , Perspective .
\n3) Pretraining on external datasets

", "messages": [ { "id": 1690683, "postDate": "2022-02-15T04:03:19.257Z", "content": "

I congratulate to all silver/gold medal holders. It was a quite a considerable shake up towards the end , yet the competition was exciting.
\nWe couldnt select from any of our top 3 solution ( that would have landed us at 40-45 position at private lb ) because of unavailability at the time of completion of our final submissions. Below is some interesting analysis of bboxes based on which we were to select our final submission .

\n

Here is the kernel doing that analysis
\nhttps://www.kaggle.com/jaideepvalani/eda-model-cv-check-rohit?scriptVersionId=87771195

\n

In this kernel I split the training data cots based on their areas so 0-1500- I regarded as tiny cots these cots were dominant not only in public lb but also in Training data , 1500-3500 small and there after it was mid size bboxes. Below is distribution.

\n
label  \n0     18582  -No Cots images\n1      2053\n2      1370\n3       791\n4       330\n5       256\n6        73\n7        22\n9         6\n8         4\n11        2\n12        2\n13        2\n
\n

If we see the distribution tiny cots were dominant in training data and likely 50 percent of private cot objects were also tiny so their impact on final score was not that high as it was in public LB so that would explain why model inference on high im size was working for many but private lb score would have dropped for those whose models were quite average at area label from 3 to 6 that might have contributed 30-45 percent of private LB score .

\n

Our best private solution kernel was mix of models best at public LB , and models doing best in area segment >2.
\nIOU area wise of our best model at public lb based on video id based split fold 2

\n
label\n1    0.637997\n2    0.627082\n3    0.822058\n4    0.784556\nName: iou, dtype: float64\n
\n

IOU area wise breakup of model not scoring so high on public LB , 58.9 with inference size 3600.

\n
label\n1    0.637223\n2    0.586083\n3    0.975207\n4    0.000023\n
\n

Some of our models performed well on high im size while some were performing on only 3600 so we chose to ensemble models performing well at different different im size.

\n

Best solution details
\nOur best solution on private lb was mix of 4 models,we couldnt select because of unavailability :(
\n1) Model with Best public score & Best public lb model using inference size -6400 best score at 9000 alone.
\n2) Remaining 2 at 3600 im size ( not so good public lb score ~58 60)
\nWe chose lower im size for best public lb scoring model because we feared for lot of FPs on rest of private LB data.

\n

CV split :
\n1) Video id based Splits : We used fold 2 as validation fold for best public lb score model . As video ids 0/1 validation fold contained lot of small cots when we checked BBoxes area wise distribution in fold . So we dint ensemble all of them just 1 and 2 to ensure we dont include underfit models in ensemble that would have caused lot false negatives .
\n2)Subsequences : We also Subsequence based splits 90:10 for 2 of models scoring around ~60

\n

Training details
\nYolov5s6
\nTraining im size=3000 to 3200
\nAugmentation : 1 mosaic , mixup 0.5 , UD, HFlip, little bit of CLAHE etc . Mosaic/mixup were crucial to train model with variety of data.

\n

What not worked to expectations

\n

1) Training on Style transfer generated datausing @hengck23 fantastic kernel
\n2) Pretraining using heavy augs like Copy paste , Perspective .
\n3) Pretraining on external datasets

", "rawMarkdown": "I congratulate to all silver/gold medal holders. It was a quite a considerable shake up towards the end , yet the competition was exciting.\nWe couldnt select from any of our top 3 solution ( that would have landed us at 40-45 position at private lb ) because of unavailability at the time of completion of our final submissions. Below is some interesting analysis of bboxes based on which we were to select our final submission .\n\nHere is the kernel doing that analysis\nhttps://www.kaggle.com/jaideepvalani/eda-model-cv-check-rohit?scriptVersionId=87771195\n\nIn this kernel I split the training data cots based on their areas so 0-1500- I regarded as tiny cots these cots were dominant not only in public lb but also in Training data , 1500-3500 small and there after it was mid size bboxes. Below is distribution.\n```\nlabel \n0 18582 -No Cots images\n1 2053\n2 1370\n3 791\n4 330\n5 256\n6 73\n7 22\n9 6\n8 4\n11 2\n12 2\n13 2\n\n```\nIf we see the distribution tiny cots were dominant in training data and likely 50 percent of private cot objects were also tiny so their impact on final score was not that high as it was in public LB so that would explain why model inference on high im size was working for many but private lb score would have dropped for those whose models were quite average at area label from 3 to 6 that might have contributed 30-45 percent of private LB score . \n\nOur best private solution kernel was mix of models best at public LB , and models doing best in area segment >2.\nIOU area wise of our best model at public lb based on video id based split fold 2\n```\nlabel\n1 0.637997\n2 0.627082\n3 0.822058\n4 0.784556\nName: iou, dtype: float64\n```\nIOU area wise breakup of model not scoring so high on public LB , 58.9 with inference size 3600.\n```\nlabel\n1 0.637223\n2 0.586083\n3 0.975207\n4 0.000023\n\n```\n\nSome of our models performed well on high im size while some were performing on only 3600 so we chose to ensemble models performing well at different different im size. \n\n**Best solution details**\nOur best solution on private lb was mix of 4 models,we couldnt select because of unavailability :(\n1) Model with Best public score & Best public lb model using inference size -6400 best score at 9000 alone.\n2) Remaining 2 at 3600 im size ( not so good public lb score ~58 60) \nWe chose lower im size for best public lb scoring model because we feared for lot of FPs on rest of private LB data. \n\n**CV split :**\n1) Video id based Splits : We used fold 2 as validation fold for best public lb score model . As video ids 0/1 validation fold contained lot of small cots when we checked BBoxes area wise distribution in fold . So we dint ensemble all of them just 1 and 2 to ensure we dont include underfit models in ensemble that would have caused lot false negatives .\n2)Subsequences : We also Subsequence based splits 90:10 for 2 of models scoring around ~60 \n\n**Training details**\nYolov5s6 \nTraining im size=3000 to 3200 \nAugmentation : 1 mosaic , mixup 0.5 , UD, HFlip, little bit of CLAHE etc . Mosaic/mixup were crucial to train model with variety of data. \n\n**What not worked to expectations**\n\n1) Training on Style transfer generated datausing @hengck23 fantastic kernel\n2) Pretraining using heavy augs like Copy paste , Perspective . \n3) Pretraining on external datasets", "votes": 15 } ], "comments": [], "raw_markdown_by_id": { "1690683": "I congratulate to all silver/gold medal holders. It was a quite a considerable shake up towards the end , yet the competition was exciting.\nWe couldnt select from any of our top 3 solution ( that would have landed us at 40-45 position at private lb ) because of unavailability at the time of completion of our final submissions. Below is some interesting analysis of bboxes based on which we were to select our final submission .\n\nHere is the kernel doing that analysis\nhttps://www.kaggle.com/jaideepvalani/eda-model-cv-check-rohit?scriptVersionId=87771195\n\nIn this kernel I split the training data cots based on their areas so 0-1500- I regarded as tiny cots these cots were dominant not only in public lb but also in Training data , 1500-3500 small and there after it was mid size bboxes. Below is distribution.\n```\nlabel \n0 18582 -No Cots images\n1 2053\n2 1370\n3 791\n4 330\n5 256\n6 73\n7 22\n9 6\n8 4\n11 2\n12 2\n13 2\n\n```\nIf we see the distribution tiny cots were dominant in training data and likely 50 percent of private cot objects were also tiny so their impact on final score was not that high as it was in public LB so that would explain why model inference on high im size was working for many but private lb score would have dropped for those whose models were quite average at area label from 3 to 6 that might have contributed 30-45 percent of private LB score . \n\nOur best private solution kernel was mix of models best at public LB , and models doing best in area segment >2.\nIOU area wise of our best model at public lb based on video id based split fold 2\n```\nlabel\n1 0.637997\n2 0.627082\n3 0.822058\n4 0.784556\nName: iou, dtype: float64\n```\nIOU area wise breakup of model not scoring so high on public LB , 58.9 with inference size 3600.\n```\nlabel\n1 0.637223\n2 0.586083\n3 0.975207\n4 0.000023\n\n```\n\nSome of our models performed well on high im size while some were performing on only 3600 so we chose to ensemble models performing well at different different im size. \n\n**Best solution details**\nOur best solution on private lb was mix of 4 models,we couldnt select because of unavailability :(\n1) Model with Best public score & Best public lb model using inference size -6400 best score at 9000 alone.\n2) Remaining 2 at 3600 im size ( not so good public lb score ~58 60) \nWe chose lower im size for best public lb scoring model because we feared for lot of FPs on rest of private LB data. \n\n**CV split :**\n1) Video id based Splits : We used fold 2 as validation fold for best public lb score model . As video ids 0/1 validation fold contained lot of small cots when we checked BBoxes area wise distribution in fold . So we dint ensemble all of them just 1 and 2 to ensure we dont include underfit models in ensemble that would have caused lot false negatives .\n2)Subsequences : We also Subsequence based splits 90:10 for 2 of models scoring around ~60 \n\n**Training details**\nYolov5s6 \nTraining im size=3000 to 3200 \nAugmentation : 1 mosaic , mixup 0.5 , UD, HFlip, little bit of CLAHE etc . Mosaic/mixup were crucial to train model with variety of data. \n\n**What not worked to expectations**\n\n1) Training on Style transfer generated datausing @hengck23 fantastic kernel\n2) Pretraining using heavy augs like Copy paste , Perspective . \n3) Pretraining on external datasets" } }