{ "id": 114578, "title": "Hints from a late joiner's persepctive", "url": "/competitions/understanding_cloud_organization/discussion/114578", "author_name": "", "post_date": "2019-10-27T17:06:07.849969200Z", "votes": 33, "comment_count": 50, "views": 0, "content": "

I joined the competition around 3 days ago. Here I would like to share some hints from my journey to the current position (52nd, 0.661).

\n\n
    \n

  1. You need to fully understanding the competition metrics and implement it correctly. Correct metrics + good split = stable CV. My CV and LB difference are all within +/- 0.003.

    2. \n

  2. Post-processing can bring huge benefit, but not necessarily needed. I got ~0.04 boost on LB on the first day. But my current position does not involve any post-processing.

    3. \n

  3. Be careful about where the boost comes from. The contributions from a true-negative and a true-positive are different.

    4. \n
\n\n

Good luck!

", "messages": [ { "id": "659440", "postDate": "10/27/2019 17:06:07", "content": "

I joined the competition around 3 days ago. Here I would like to share some hints from my journey to the current position (52nd, 0.661).

\n\n
    \n

  1. You need to fully understanding the competition metrics and implement it correctly. Correct metrics + good split = stable CV. My CV and LB difference are all within +/- 0.003.

    2. \n

  2. Post-processing can bring huge benefit, but not necessarily needed. I got ~0.04 boost on LB on the first day. But my current position does not involve any post-processing.

    3. \n

  3. Be careful about where the boost comes from. The contributions from a true-negative and a true-positive are different.

    4. \n
\n\n

Good luck!

", "rawMarkdown": "I joined the competition around 3 days ago. Here I would like to share some hints from my journey to the current position (52nd, 0.661). \n\n1. You need to fully understanding the competition metrics and implement it correctly. Correct metrics + good split = stable CV. My CV and LB difference are all within +/- 0.003. \n\n2. Post-processing can bring huge benefit, but not necessarily needed. I got ~0.04 boost on LB on the first day. But my current position does not involve any post-processing.\n\n3. Be careful about where the boost comes from. The contributions from a true-negative and a true-positive are different. \n\nGood luck!", "votes": null }, { "id": "659449", "postDate": "10/27/2019 17:24:46", "content": "

0.661 without any post-processing is a custom model?

", "rawMarkdown": "0.661 without any post-processing is a custom model?", "votes": null }, { "id": "659556", "postDate": "10/27/2019 22:10:07", "content": "

I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+

", "rawMarkdown": "I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+", "votes": null }, { "id": "659583", "postDate": "10/28/2019 00:09:37", "content": "

No.

", "rawMarkdown": "No.", "votes": null }, { "id": "659607", "postDate": "10/28/2019 01:13:34", "content": "

Thanks for the share. If you don't mind, did you use classifier to get this score? Or just segmentation model? Thanks in advanced

", "rawMarkdown": "Thanks for the share. If you don't mind, did you use classifier to get this score? Or just segmentation model? Thanks in advanced", "votes": null }, { "id": "659631", "postDate": "10/28/2019 02:11:49", "content": "
\n

DimitreOliveira wrote:

\n \n

I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+

\n
\n\n

I have the same question

", "rawMarkdown": "> **DimitreOliveira wrote:**\n> \n> I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+\n\nI have the same question", "votes": null }, { "id": "659760", "postDate": "10/28/2019 08:26:10", "content": "

when you say post-process, is it for example removing small size masks?

", "rawMarkdown": "when you say post-process, is it for example removing small size masks?", "votes": null }, { "id": "659823", "postDate": "10/28/2019 09:59:29", "content": "

I've got 0.605 with unet-effnet+bce, I'vent done yet any post processing and threshold tuning.

\n\n

upd:\nI also haven't use any TTA.

", "rawMarkdown": "I've got 0.605 with unet-effnet+bce, I'vent done yet any post processing and threshold tuning.\n\nupd:\nI also haven't use any TTA.", "votes": null }, { "id": "659840", "postDate": "10/28/2019 10:20:48", "content": "

@niuddd yes it is\nit seems that some people reached 0.66x only with the network output

", "rawMarkdown": "niuddd yes it is\nit seems that some people reached 0.66x only with the network output", "votes": null }, { "id": "659904", "postDate": "10/28/2019 12:24:38", "content": "

thx for the information, I also remove small masks, but it's not clear how much boost it got

", "rawMarkdown": "thx for the information, I also remove small masks, but it's not clear how much boost it got", "votes": null }, { "id": "659914", "postDate": "10/28/2019 12:36:06", "content": "

I got 0.600+ on validation score using dice_coef removing the smooth param as the eval metrics today with the references of here

", "rawMarkdown": "I got 0.600+ on validation score using dice_coef removing the smooth param as the eval metrics today with the references of [here](https://www.kaggle.com/c/understanding_cloud_organization/discussion/114093#latest-659887)", "votes": null }, { "id": "660201", "postDate": "10/28/2019 21:29:02", "content": "

Only seg.

", "rawMarkdown": "Only seg.", "votes": null }, { "id": "660241", "postDate": "10/28/2019 23:22:54", "content": "

Thanks, that's impressive

", "rawMarkdown": "Thanks, that's impressive", "votes": null }, { "id": "660264", "postDate": "10/29/2019 00:13:12", "content": "

I got 0.660 with 5-fold segmentation models with thresholds (0.5, 20000) without classification, similar structure to my public kernel. I agree and think it is possible to reach higher (0.660+) without classification.

", "rawMarkdown": "I got 0.660 with 5-fold segmentation models with thresholds (0.5, 20000) without classification, similar structure to my public kernel. I agree and think it is possible to reach higher (0.660+) without classification.", "votes": null }, { "id": "660286", "postDate": "10/29/2019 00:44:30", "content": "

@gogo827jz Thanks for the information! I should take some serious thinking about my segmentation model... I only get this place with classifier, without it, I can only get 0.65LB lol

", "rawMarkdown": "gogo827jz Thanks for the information! I should take some serious thinking about my segmentation model... I only get this place with classifier, without it, I can only get 0.65LB lol", "votes": null }, { "id": "660298", "postDate": "10/29/2019 01:06:28", "content": "

I got an 0.006 boost on 5-fold compared with single fold. Very huge.

", "rawMarkdown": "I got an 0.006 boost on 5-fold compared with single fold. Very huge.", "votes": null }, { "id": "660303", "postDate": "10/29/2019 01:13:45", "content": "

@gogo827jz Thanks for your kindly share, I will try that.

", "rawMarkdown": "gogo827jz Thanks for your kindly share, I will try that.", "votes": null }, { "id": "660965", "postDate": "10/29/2019 20:17:06", "content": "

What is post processing exactly here? Ensemble methods??

", "rawMarkdown": "What is post processing exactly here? Ensemble methods??", "votes": null }, { "id": "661068", "postDate": "10/29/2019 23:29:33", "content": "
    \n
  1. Use classifier to filter out less confidence mask
    2. \n
  2. Use threshold and limit min size of mask to filter out less confidence mask
    3. \n
", "rawMarkdown": "1. Use classifier to filter out less confidence mask\n2. Use threshold and limit min size of mask to filter out less confidence mask", "votes": null }, { "id": "661085", "postDate": "10/29/2019 23:49:02", "content": "

You can also alter the masks with post processing. For example, wherever the image is all black (between satellites), you can remove that from your mask. Additionally, you can reshape masks. You can turn them into polygon shapes, or you can analyze the density of blue and white in the images behind the masks and alter the masks accordingly.

", "rawMarkdown": "You can also alter the masks with post processing. For example, wherever the image is all black (between satellites), you can remove that from your mask. Additionally, you can reshape masks. You can turn them into polygon shapes, or you can analyze the density of blue and white in the images behind the masks and alter the masks accordingly.", "votes": null }, { "id": "661157", "postDate": "10/30/2019 00:52:30", "content": "

Fortunately the Unets are pretty good on not predicting on the black areas, but this blue/white pixel seems interesting.

", "rawMarkdown": "Fortunately the Unets are pretty good on not predicting on the black areas, but this blue/white pixel seems interesting.", "votes": null }, { "id": "661268", "postDate": "10/30/2019 04:15:29", "content": "

you mean you got the current position without threshold and minsize Post-preocess?

", "rawMarkdown": "you mean you got the current position without threshold and minsize Post-preocess?", "votes": null }, { "id": "661544", "postDate": "10/30/2019 12:57:54", "content": "

After doing an EDA for the CV , I have seen that my Unet somehow has many False Negatives. Since the classification is used to remove the False Negatives , the public kernel of Classification Model Keras EfficientnetB2 gives me very less to no boost at all .

", "rawMarkdown": "After doing an EDA for the CV , I have seen that my Unet somehow has many False Negatives. Since the classification is used to remove the False Negatives , the public kernel of Classification Model Keras EfficientnetB2 gives me very less to no boost at all .", "votes": null }, { "id": "661567", "postDate": "10/30/2019 13:19:12", "content": "

I thought classifier is for reducing the False Positive? But indeed, After I trained my segmentation with better setting, I can't get any benefit from classifier anymore. But still, I think classifier can be useful in this competition, since we only saw the score from 25% of testing data. Also in airbus competition, the winner sampled the data with large portion of labeled data to train the segmentation model and use classifier to clear the false positive. So I think classifier can still be crucial.

", "rawMarkdown": "I thought classifier is for reducing the False Positive? But indeed, After I trained my segmentation with better setting, I can't get any benefit from classifier anymore. But still, I think classifier can be useful in this competition, since we only saw the score from 25% of testing data. Also in airbus competition, the winner sampled the data with large portion of labeled data to train the segmentation model and use classifier to clear the false positive. So I think classifier can still be crucial.", "votes": null }, { "id": "661663", "postDate": "10/30/2019 15:03:23", "content": "

Sorry .. I mean the Unet is not detecting masks for many of the Fish and Gravel . Therefore Classifier does not have much to remove .

", "rawMarkdown": "Sorry .. I mean the Unet is not detecting masks for many of the Fish and Gravel . Therefore Classifier does not have much to remove .", "votes": null }, { "id": "662204", "postDate": "10/31/2019 08:15:48", "content": "

Could you share some tips around augmentation and image size? I find I start overfitting quickly

", "rawMarkdown": "Could you share some tips around augmentation and image size? I find I start overfitting quickly", "votes": null }, { "id": "662213", "postDate": "10/31/2019 08:30:36", "content": "

This has become a menace :( . Looks simple, but very tricky competition .

", "rawMarkdown": "This has become a menace :( . Looks simple, but very tricky competition .", "votes": null }, { "id": "662222", "postDate": "10/31/2019 09:00:47", "content": "

@khornlund Hah, don't worry.. Maybe one of your submissions is 1st on the private.

", "rawMarkdown": "khornlund Hah, don't worry.. Maybe one of your submissions is 1st on the private.", "votes": null }, { "id": "662582", "postDate": "10/31/2019 18:04:29", "content": "

Augmentations are all taken from the public notebooks, I experimented to find the best parameters for my model.

\n\n

I am not sure whether image size matters here, the mask label is pretty noisy. Though I have not experimented, I doubt that large image size helps in this competition.

", "rawMarkdown": "Augmentations are all taken from the public notebooks, I experimented to find the best parameters for my model. \n\nI am not sure whether image size matters here, the mask label is pretty noisy. Though I have not experimented, I doubt that large image size helps in this competition.", "votes": null }, { "id": "662727", "postDate": "10/31/2019 23:14:37", "content": "

@naivelamb

\n\n

do you have a problem with overfitting? My single fold segmentation with tta scores only 0.657. The network overfits already after 30 epochs...

", "rawMarkdown": "naivelamb \n\ndo you have a problem with overfitting? My single fold segmentation with tta scores only 0.657. The network overfits already after 30 epochs...", "votes": null }, { "id": "662788", "postDate": "11/01/2019 02:14:26", "content": "

What batch size did you use? I find that networks trained on Kaggle kernel with larger batchsize have a 0.015 boost compared with my offline models.

", "rawMarkdown": "What batch size did you use? I find that networks trained on Kaggle kernel with larger batchsize have a 0.015 boost compared with my offline models.", "votes": null }, { "id": "662789", "postDate": "11/01/2019 02:14:47", "content": "

It depends on model. Which model are you using?

", "rawMarkdown": "It depends on model. Which model are you using?", "votes": null }, { "id": "662839", "postDate": "11/01/2019 04:30:02", "content": "

How many epochs do you usually train one model? I trained 30 epochs, but wondering if it would overfit.

", "rawMarkdown": "How many epochs do you usually train one model? I trained 30 epochs, but wondering if it would overfit.", "votes": null }, { "id": "662846", "postDate": "11/01/2019 04:48:22", "content": "

You can always run an experiment of more epochs to check whether 30 is enough or not.

", "rawMarkdown": "You can always run an experiment of more epochs to check whether 30 is enough or not.", "votes": null }, { "id": "662990", "postDate": "11/01/2019 09:44:11", "content": "

With AdamW, even Resnet34 overfits. With SGD little better.

", "rawMarkdown": "With AdamW, even Resnet34 overfits. With SGD little better.", "votes": null }, { "id": "662991", "postDate": "11/01/2019 09:46:09", "content": "

@phoenix9032 which encoder did you use with Unet?

", "rawMarkdown": "phoenix9032 which encoder did you use with Unet?", "votes": null }, { "id": "663014", "postDate": "11/01/2019 10:47:08", "content": "

Couple of .. Resnet34, SERESNET50

", "rawMarkdown": "Couple of .. Resnet34, SERESNET50", "votes": null }, { "id": "663071", "postDate": "11/01/2019 12:16:43", "content": "

What scheduler are you using?

", "rawMarkdown": "What scheduler are you using?", "votes": null }, { "id": "663073", "postDate": "11/01/2019 12:18:25", "content": "

@xiejialun what's your classifier performance, I trained a classifier,but the avg accuracy of 4 classes is only 0.77 in my validation data

", "rawMarkdown": "xiejialun what's your classifier performance, I trained a classifier,but the avg accuracy of 4 classes is only 0.77 in my validation data", "votes": null }, { "id": "663077", "postDate": "11/01/2019 12:26:13", "content": "

step. The bigger LR leads also more overfitting.

", "rawMarkdown": "step. The bigger LR leads also more overfitting.", "votes": null }, { "id": "663083", "postDate": "11/01/2019 12:35:37", "content": "

I also meet the same problem, I only train 20 epoch, but it seems also overfit😂

", "rawMarkdown": "I also meet the same problem, I only train 20 epoch, but it seems also overfit😂", "votes": null }, { "id": "663120", "postDate": "11/01/2019 13:28:29", "content": "

Twenty is plenty.

", "rawMarkdown": "Twenty is plenty.", "votes": null }, { "id": "663126", "postDate": "11/01/2019 13:36:11", "content": "

@tugstugi Why bigger LR leads overfitting??

", "rawMarkdown": "tugstugi Why bigger LR leads overfitting??", "votes": null }, { "id": "663127", "postDate": "11/01/2019 13:37:11", "content": "

@robga I agree, my models also converge around 10~15 epochs usually.

", "rawMarkdown": "robga I agree, my models also converge around 10~15 epochs usually.", "votes": null }, { "id": "663131", "postDate": "11/01/2019 13:40:46", "content": "

what's more, bigger model seems didn't helpful, resnet34 is almost same lb with seresnet50 and b3

", "rawMarkdown": "what's more, bigger model seems didn't helpful, resnet34 is almost same lb with seresnet50 and b3", "votes": null }, { "id": "663139", "postDate": "11/01/2019 13:52:02", "content": "

@hustkevin1037 If you use deeper model , reduce the LR ... A lot depend on LR for convergence.. But , at the end , all it matters what CV score it gives in those epochs it run..

", "rawMarkdown": "hustkevin1037 If you use deeper model , reduce the LR ... A lot depend on LR for convergence.. But , at the end , all it matters what CV score it gives in those epochs it run..", "votes": null }, { "id": "663163", "postDate": "11/01/2019 14:30:14", "content": "

@bamps53 it seems bigger bigger LR destroyes the pretrained weights. I assume that @robga freezes some layers to prevent that... Or stops earlier.

", "rawMarkdown": "bamps53 it seems bigger bigger LR destroyes the pretrained weights. I assume that @robga freezes some layers to prevent that... Or stops earlier.", "votes": null }, { "id": "663687", "postDate": "11/02/2019 14:33:02", "content": "

@hustkevin1037 About 0.8 valid accuracy with regular binary crossentropy loss on densenet169. Maybe you can try some different augmentation, this should be helpful. But I don't use classifier now, I think in this moment, the classifier just make my public score worse. But I'm still working on training a better classifier with different training process and backbone.

", "rawMarkdown": "hustkevin1037 About 0.8 valid accuracy with regular binary crossentropy loss on densenet169. Maybe you can try some different augmentation, this should be helpful. But I don't use classifier now, I think in this moment, the classifier just make my public score worse. But I'm still working on training a better classifier with different training process and backbone.", "votes": null }, { "id": "664743", "postDate": "11/04/2019 06:39:11", "content": "

Thanks for those hints. Really useful.

", "rawMarkdown": "Thanks for those hints. Really useful.", "votes": null }, { "id": "664853", "postDate": "11/04/2019 10:02:37", "content": "

each true neg add 1, each true pos add 0.75? is that right?

", "rawMarkdown": "each true neg add 1, each true pos add 0.75? is that right?", "votes": null }, { "id": "664948", "postDate": "11/04/2019 12:59:17", "content": "

I'm also very curious about it that , seresnet50 performs the same as resnet34 , not overfitt , not underfitting .

", "rawMarkdown": "I'm also very curious about it that , seresnet50 performs the same as resnet34 , not overfitt , not underfitting .", "votes": null } ], "comments": [ { "id": 659449, "author_name": "igormunizims", "author_url": "", "post_date": "10/27/2019 17:24:46", "content": "

0.661 without any post-processing is a custom model?

", "votes": null, "replies": [ { "id": 659583, "author_name": "naivelamb", "author_url": "", "post_date": "10/28/2019 00:09:37", "content": "

No.

", "votes": null, "replies": [] }, { "id": 659760, "author_name": "niuddd", "author_url": "", "post_date": "10/28/2019 08:26:10", "content": "

when you say post-process, is it for example removing small size masks?

", "votes": null, "replies": [] }, { "id": 659840, "author_name": "igormunizims", "author_url": "", "post_date": "10/28/2019 10:20:48", "content": "

@niuddd yes it is\nit seems that some people reached 0.66x only with the network output

", "votes": null, "replies": [] }, { "id": 659904, "author_name": "niuddd", "author_url": "", "post_date": "10/28/2019 12:24:38", "content": "

thx for the information, I also remove small masks, but it's not clear how much boost it got

", "votes": null, "replies": [] } ] }, { "id": 659556, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/27/2019 22:10:07", "content": "

I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+

", "votes": null, "replies": [ { "id": 659631, "author_name": "zhangeng", "author_url": "", "post_date": "10/28/2019 02:11:49", "content": "
\n

DimitreOliveira wrote:

\n \n

I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+

\n
\n\n

I have the same question

", "votes": null, "replies": [] }, { "id": 659823, "author_name": "valyukov", "author_url": "", "post_date": "10/28/2019 09:59:29", "content": "

I've got 0.605 with unet-effnet+bce, I'vent done yet any post processing and threshold tuning.

\n\n

upd:\nI also haven't use any TTA.

", "votes": null, "replies": [] }, { "id": 659914, "author_name": "sj626591833", "author_url": "", "post_date": "10/28/2019 12:36:06", "content": "

I got 0.600+ on validation score using dice_coef removing the smooth param as the eval metrics today with the references of here

", "votes": null, "replies": [] } ] }, { "id": 659607, "author_name": "xiejialun", "author_url": "", "post_date": "10/28/2019 01:13:34", "content": "

Thanks for the share. If you don't mind, did you use classifier to get this score? Or just segmentation model? Thanks in advanced

", "votes": null, "replies": [ { "id": 660201, "author_name": "naivelamb", "author_url": "", "post_date": "10/28/2019 21:29:02", "content": "

Only seg.

", "votes": null, "replies": [] }, { "id": 660241, "author_name": "xiejialun", "author_url": "", "post_date": "10/28/2019 23:22:54", "content": "

Thanks, that's impressive

", "votes": null, "replies": [] }, { "id": 660264, "author_name": "gogo827jz", "author_url": "", "post_date": "10/29/2019 00:13:12", "content": "

I got 0.660 with 5-fold segmentation models with thresholds (0.5, 20000) without classification, similar structure to my public kernel. I agree and think it is possible to reach higher (0.660+) without classification.

", "votes": null, "replies": [] }, { "id": 660286, "author_name": "xiejialun", "author_url": "", "post_date": "10/29/2019 00:44:30", "content": "

@gogo827jz Thanks for the information! I should take some serious thinking about my segmentation model... I only get this place with classifier, without it, I can only get 0.65LB lol

", "votes": null, "replies": [] }, { "id": 660298, "author_name": "gogo827jz", "author_url": "", "post_date": "10/29/2019 01:06:28", "content": "

I got an 0.006 boost on 5-fold compared with single fold. Very huge.

", "votes": null, "replies": [] }, { "id": 660303, "author_name": "xiejialun", "author_url": "", "post_date": "10/29/2019 01:13:45", "content": "

@gogo827jz Thanks for your kindly share, I will try that.

", "votes": null, "replies": [] }, { "id": 661544, "author_name": "phoenix9032", "author_url": "", "post_date": "10/30/2019 12:57:54", "content": "

After doing an EDA for the CV , I have seen that my Unet somehow has many False Negatives. Since the classification is used to remove the False Negatives , the public kernel of Classification Model Keras EfficientnetB2 gives me very less to no boost at all .

", "votes": null, "replies": [] }, { "id": 661567, "author_name": "xiejialun", "author_url": "", "post_date": "10/30/2019 13:19:12", "content": "

I thought classifier is for reducing the False Positive? But indeed, After I trained my segmentation with better setting, I can't get any benefit from classifier anymore. But still, I think classifier can be useful in this competition, since we only saw the score from 25% of testing data. Also in airbus competition, the winner sampled the data with large portion of labeled data to train the segmentation model and use classifier to clear the false positive. So I think classifier can still be crucial.

", "votes": null, "replies": [] }, { "id": 661663, "author_name": "phoenix9032", "author_url": "", "post_date": "10/30/2019 15:03:23", "content": "

Sorry .. I mean the Unet is not detecting masks for many of the Fish and Gravel . Therefore Classifier does not have much to remove .

", "votes": null, "replies": [] }, { "id": 662991, "author_name": "tmhung", "author_url": "", "post_date": "11/01/2019 09:46:09", "content": "

@phoenix9032 which encoder did you use with Unet?

", "votes": null, "replies": [] }, { "id": 663014, "author_name": "phoenix9032", "author_url": "", "post_date": "11/01/2019 10:47:08", "content": "

Couple of .. Resnet34, SERESNET50

", "votes": null, "replies": [] }, { "id": 663073, "author_name": "hustkevin1037", "author_url": "", "post_date": "11/01/2019 12:18:25", "content": "

@xiejialun what's your classifier performance, I trained a classifier,but the avg accuracy of 4 classes is only 0.77 in my validation data

", "votes": null, "replies": [] }, { "id": 663687, "author_name": "xiejialun", "author_url": "", "post_date": "11/02/2019 14:33:02", "content": "

@hustkevin1037 About 0.8 valid accuracy with regular binary crossentropy loss on densenet169. Maybe you can try some different augmentation, this should be helpful. But I don't use classifier now, I think in this moment, the classifier just make my public score worse. But I'm still working on training a better classifier with different training process and backbone.

", "votes": null, "replies": [] } ] }, { "id": 660965, "author_name": "dynamite2055", "author_url": "", "post_date": "10/29/2019 20:17:06", "content": "

What is post processing exactly here? Ensemble methods??

", "votes": null, "replies": [ { "id": 661068, "author_name": "xiejialun", "author_url": "", "post_date": "10/29/2019 23:29:33", "content": "
    \n
  1. Use classifier to filter out less confidence mask
    2. \n
  2. Use threshold and limit min size of mask to filter out less confidence mask
    3. \n
", "votes": null, "replies": [] }, { "id": 661085, "author_name": "cdeotte", "author_url": "", "post_date": "10/29/2019 23:49:02", "content": "

You can also alter the masks with post processing. For example, wherever the image is all black (between satellites), you can remove that from your mask. Additionally, you can reshape masks. You can turn them into polygon shapes, or you can analyze the density of blue and white in the images behind the masks and alter the masks accordingly.

", "votes": null, "replies": [] }, { "id": 661157, "author_name": "dimitreoliveira", "author_url": "", "post_date": "10/30/2019 00:52:30", "content": "

Fortunately the Unets are pretty good on not predicting on the black areas, but this blue/white pixel seems interesting.

", "votes": null, "replies": [] } ] }, { "id": 661268, "author_name": "luxianhao", "author_url": "", "post_date": "10/30/2019 04:15:29", "content": "

you mean you got the current position without threshold and minsize Post-preocess?

", "votes": null, "replies": [] }, { "id": 662204, "author_name": "khornlund", "author_url": "", "post_date": "10/31/2019 08:15:48", "content": "

Could you share some tips around augmentation and image size? I find I start overfitting quickly

", "votes": null, "replies": [ { "id": 662213, "author_name": "phoenix9032", "author_url": "", "post_date": "10/31/2019 08:30:36", "content": "

This has become a menace :( . Looks simple, but very tricky competition .

", "votes": null, "replies": [] }, { "id": 662222, "author_name": "garybios", "author_url": "", "post_date": "10/31/2019 09:00:47", "content": "

@khornlund Hah, don't worry.. Maybe one of your submissions is 1st on the private.

", "votes": null, "replies": [] }, { "id": 662582, "author_name": "naivelamb", "author_url": "", "post_date": "10/31/2019 18:04:29", "content": "

Augmentations are all taken from the public notebooks, I experimented to find the best parameters for my model.

\n\n

I am not sure whether image size matters here, the mask label is pretty noisy. Though I have not experimented, I doubt that large image size helps in this competition.

", "votes": null, "replies": [] }, { "id": 662788, "author_name": "rguo97", "author_url": "", "post_date": "11/01/2019 02:14:26", "content": "

What batch size did you use? I find that networks trained on Kaggle kernel with larger batchsize have a 0.015 boost compared with my offline models.

", "votes": null, "replies": [] } ] }, { "id": 662727, "author_name": "tugstugi", "author_url": "", "post_date": "10/31/2019 23:14:37", "content": "

@naivelamb

\n\n

do you have a problem with overfitting? My single fold segmentation with tta scores only 0.657. The network overfits already after 30 epochs...

", "votes": null, "replies": [ { "id": 662789, "author_name": "naivelamb", "author_url": "", "post_date": "11/01/2019 02:14:47", "content": "

It depends on model. Which model are you using?

", "votes": null, "replies": [] }, { "id": 662990, "author_name": "tugstugi", "author_url": "", "post_date": "11/01/2019 09:44:11", "content": "

With AdamW, even Resnet34 overfits. With SGD little better.

", "votes": null, "replies": [] }, { "id": 663071, "author_name": "lightnezzofbeing", "author_url": "", "post_date": "11/01/2019 12:16:43", "content": "

What scheduler are you using?

", "votes": null, "replies": [] }, { "id": 663077, "author_name": "tugstugi", "author_url": "", "post_date": "11/01/2019 12:26:13", "content": "

step. The bigger LR leads also more overfitting.

", "votes": null, "replies": [] }, { "id": 663083, "author_name": "hustkevin1037", "author_url": "", "post_date": "11/01/2019 12:35:37", "content": "

I also meet the same problem, I only train 20 epoch, but it seems also overfit😂

", "votes": null, "replies": [] }, { "id": 663120, "author_name": "robga", "author_url": "", "post_date": "11/01/2019 13:28:29", "content": "

Twenty is plenty.

", "votes": null, "replies": [] }, { "id": 663126, "author_name": "bamps53", "author_url": "", "post_date": "11/01/2019 13:36:11", "content": "

@tugstugi Why bigger LR leads overfitting??

", "votes": null, "replies": [] }, { "id": 663127, "author_name": "bamps53", "author_url": "", "post_date": "11/01/2019 13:37:11", "content": "

@robga I agree, my models also converge around 10~15 epochs usually.

", "votes": null, "replies": [] }, { "id": 663131, "author_name": "hustkevin1037", "author_url": "", "post_date": "11/01/2019 13:40:46", "content": "

what's more, bigger model seems didn't helpful, resnet34 is almost same lb with seresnet50 and b3

", "votes": null, "replies": [] }, { "id": 663139, "author_name": "phoenix9032", "author_url": "", "post_date": "11/01/2019 13:52:02", "content": "

@hustkevin1037 If you use deeper model , reduce the LR ... A lot depend on LR for convergence.. But , at the end , all it matters what CV score it gives in those epochs it run..

", "votes": null, "replies": [] }, { "id": 663163, "author_name": "tugstugi", "author_url": "", "post_date": "11/01/2019 14:30:14", "content": "

@bamps53 it seems bigger bigger LR destroyes the pretrained weights. I assume that @robga freezes some layers to prevent that... Or stops earlier.

", "votes": null, "replies": [] }, { "id": 664948, "author_name": "xujingzhao", "author_url": "", "post_date": "11/04/2019 12:59:17", "content": "

I'm also very curious about it that , seresnet50 performs the same as resnet34 , not overfitt , not underfitting .

", "votes": null, "replies": [] } ] }, { "id": 662839, "author_name": "zhaoguowang666", "author_url": "", "post_date": "11/01/2019 04:30:02", "content": "

How many epochs do you usually train one model? I trained 30 epochs, but wondering if it would overfit.

", "votes": null, "replies": [ { "id": 662846, "author_name": "naivelamb", "author_url": "", "post_date": "11/01/2019 04:48:22", "content": "

You can always run an experiment of more epochs to check whether 30 is enough or not.

", "votes": null, "replies": [] } ] }, { "id": 664743, "author_name": "mykttu", "author_url": "", "post_date": "11/04/2019 06:39:11", "content": "

Thanks for those hints. Really useful.

", "votes": null, "replies": [] }, { "id": 664853, "author_name": "jt120lz", "author_url": "", "post_date": "11/04/2019 10:02:37", "content": "

each true neg add 1, each true pos add 0.75? is that right?

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "659440": "I joined the competition around 3 days ago. Here I would like to share some hints from my journey to the current position (52nd, 0.661). \n\n1. You need to fully understanding the competition metrics and implement it correctly. Correct metrics + good split = stable CV. My CV and LB difference are all within +/- 0.003. \n\n2. Post-processing can bring huge benefit, but not necessarily needed. I got ~0.04 boost on LB on the first day. But my current position does not involve any post-processing.\n\n3. Be careful about where the boost comes from. The contributions from a true-negative and a true-positive are different. \n\nGood luck!", "659449": "0.661 without any post-processing is a custom model?", "659556": "I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+", "659583": "No.", "659607": "Thanks for the share. If you don't mind, did you use classifier to get this score? Or just segmentation model? Thanks in advanced", "659631": "> **DimitreOliveira wrote:**\n> \n> I'm also really curious about how some people can get scores that high without post-process, I don't think my raw model predictions can get 0.600+\n\nI have the same question", "659760": "when you say post-process, is it for example removing small size masks?", "659823": "I've got 0.605 with unet-effnet+bce, I'vent done yet any post processing and threshold tuning.\n\nupd:\nI also haven't use any TTA.", "659840": "niuddd yes it is\nit seems that some people reached 0.66x only with the network output", "659904": "thx for the information, I also remove small masks, but it's not clear how much boost it got", "659914": "I got 0.600+ on validation score using dice_coef removing the smooth param as the eval metrics today with the references of [here](https://www.kaggle.com/c/understanding_cloud_organization/discussion/114093#latest-659887)", "660201": "Only seg.", "660241": "Thanks, that's impressive", "660264": "I got 0.660 with 5-fold segmentation models with thresholds (0.5, 20000) without classification, similar structure to my public kernel. I agree and think it is possible to reach higher (0.660+) without classification.", "660286": "gogo827jz Thanks for the information! I should take some serious thinking about my segmentation model... I only get this place with classifier, without it, I can only get 0.65LB lol", "660298": "I got an 0.006 boost on 5-fold compared with single fold. Very huge.", "660303": "gogo827jz Thanks for your kindly share, I will try that.", "660965": "What is post processing exactly here? Ensemble methods??", "661068": "1. Use classifier to filter out less confidence mask\n2. Use threshold and limit min size of mask to filter out less confidence mask", "661085": "You can also alter the masks with post processing. For example, wherever the image is all black (between satellites), you can remove that from your mask. Additionally, you can reshape masks. You can turn them into polygon shapes, or you can analyze the density of blue and white in the images behind the masks and alter the masks accordingly.", "661157": "Fortunately the Unets are pretty good on not predicting on the black areas, but this blue/white pixel seems interesting.", "661268": "you mean you got the current position without threshold and minsize Post-preocess?", "661544": "After doing an EDA for the CV , I have seen that my Unet somehow has many False Negatives. Since the classification is used to remove the False Negatives , the public kernel of Classification Model Keras EfficientnetB2 gives me very less to no boost at all .", "661567": "I thought classifier is for reducing the False Positive? But indeed, After I trained my segmentation with better setting, I can't get any benefit from classifier anymore. But still, I think classifier can be useful in this competition, since we only saw the score from 25% of testing data. Also in airbus competition, the winner sampled the data with large portion of labeled data to train the segmentation model and use classifier to clear the false positive. So I think classifier can still be crucial.", "661663": "Sorry .. I mean the Unet is not detecting masks for many of the Fish and Gravel . Therefore Classifier does not have much to remove .", "662204": "Could you share some tips around augmentation and image size? I find I start overfitting quickly", "662213": "This has become a menace :( . Looks simple, but very tricky competition .", "662222": "khornlund Hah, don't worry.. Maybe one of your submissions is 1st on the private.", "662582": "Augmentations are all taken from the public notebooks, I experimented to find the best parameters for my model. \n\nI am not sure whether image size matters here, the mask label is pretty noisy. Though I have not experimented, I doubt that large image size helps in this competition.", "662727": "naivelamb \n\ndo you have a problem with overfitting? My single fold segmentation with tta scores only 0.657. The network overfits already after 30 epochs...", "662788": "What batch size did you use? I find that networks trained on Kaggle kernel with larger batchsize have a 0.015 boost compared with my offline models.", "662789": "It depends on model. Which model are you using?", "662839": "How many epochs do you usually train one model? I trained 30 epochs, but wondering if it would overfit.", "662846": "You can always run an experiment of more epochs to check whether 30 is enough or not.", "662990": "With AdamW, even Resnet34 overfits. With SGD little better.", "662991": "phoenix9032 which encoder did you use with Unet?", "663014": "Couple of .. Resnet34, SERESNET50", "663071": "What scheduler are you using?", "663073": "xiejialun what's your classifier performance, I trained a classifier,but the avg accuracy of 4 classes is only 0.77 in my validation data", "663077": "step. The bigger LR leads also more overfitting.", "663083": "I also meet the same problem, I only train 20 epoch, but it seems also overfit😂", "663120": "Twenty is plenty.", "663126": "tugstugi Why bigger LR leads overfitting??", "663127": "robga I agree, my models also converge around 10~15 epochs usually.", "663131": "what's more, bigger model seems didn't helpful, resnet34 is almost same lb with seresnet50 and b3", "663139": "hustkevin1037 If you use deeper model , reduce the LR ... A lot depend on LR for convergence.. But , at the end , all it matters what CV score it gives in those epochs it run..", "663163": "bamps53 it seems bigger bigger LR destroyes the pretrained weights. I assume that @robga freezes some layers to prevent that... Or stops earlier.", "663687": "hustkevin1037 About 0.8 valid accuracy with regular binary crossentropy loss on densenet169. Maybe you can try some different augmentation, this should be helpful. But I don't use classifier now, I think in this moment, the classifier just make my public score worse. But I'm still working on training a better classifier with different training process and backbone.", "664743": "Thanks for those hints. Really useful.", "664853": "each true neg add 1, each true pos add 0.75? is that right?", "664948": "I'm also very curious about it that , seresnet50 performs the same as resnet34 , not overfitt , not underfitting ." }, "source": "meta" }