![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5725749%2Fb9819315244312a47802b241517dc57e%2F4.PNG?generation=1686971322910074&alt=media\")
I'm very grateful for organizing very challenging and fantastic challenge.
\nI feel like there are a lot of really great people and I respect them all.
\nI would like to express my gratitude to both the organizers and participants of the competition.
\n
I would like to describe my solution in the order of contribution of performance improvement.
\nThe number of ★ in the description below means the degree of contribution to performance improvement.
\n
As you can see in the figure above, i thought that particular layers of fragment would not correspond to the same layers of other fragment. Using these points, i came up with an augmentation method that can give strong regularization to the model by reflecting the characteristics of the data.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5725749%2F8601353d50ccdc56721bfc1341395bff%2F8.PNG?generation=1686981811287731&alt=media\")
As you can see in the figure above, there are 3 steps.
\n1) temporal random crop
2) random paste
\n3) random cutout
\nThe above three processes can be seen simply by looking at the Pytorch Dataset code part below.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5725749%2F2cb3e87617e91bbff8df5dbdf213838e%2F9.PNG?generation=1686982042582252&alt=media\")
\n
As shown in the figure below, I saved the predicted mask for each epoch. The file name has the epoch, score, tp, fp, and fn, of which the fp value was important. Even if the score was similar, if the fp was large, it tended to be very bad in the public score. Therefore, I wanted to select a weight with a small fp value and a high score value, and I selected an appropriate fp value for each fold through several submissions. In addition, I tried to increase the generalization performance by ensemble models with a range of fp values.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5725749%2Faa46536cf349929607790d1c3a32a21c%2F55.PNG?generation=1687062720342906&alt=media\")
\n
I ensembles 3 models.
\n3D resnet152, 3D resnet200, 3D resnext101 with unet-like decoder.
Refered models was firstly inspired by JEBASTIN NADAR (https://www.kaggle.com/code/samfc10/vesuvius-challenge-3d-resnet-training) and applied. Thanks for your effort and respect you.
\n3D resnet152, 3D resnet200 is from https://github.com/kenshohara/3D-ResNets-PyTorch
\n3D resnext101 is from https://github.com/okankop/Efficient-3DCNNs
\nAll models have mit licenses and are not against the rules.
3D Resnet was pretrained on Kinetic 710, and 3D resnext101 was pretrained on Kinetic 600.
\nThere was a huge difference between being trained in kinetics and not being able to. Therefore, during the competition, I tried to find a model trained in the kinetic 600 or 700.
All models are combined unet-like simple decoder. Features from 3D encoder are upsampled and concatenated using decoder.
\ndetails are in code.
\n
\n
As shown in the table below, the two models are the ones I submitted finally.
\nEnsemble 2 had a lower public score than ensemble1, but performed better in cross validation. There were other models that could be adopted, but considering that it came out well in the cross validation, it was adopted, and the performance was improved more in the private, resulting in a better first place.
| models | \ncv score | \nthreshold | \npublic score | \nprivate score | \n
|---|---|---|---|---|
| Ensemble 1 | \n0.66 / 0.762/ 0.723 / 0.68 | \n0.5 | \n0.795835 | \n0.663542 | \n
| Ensemble 2 | \n0.67 / 0.764 / 0.724 / 0.69 | \n0.47 | \n0.789024 | \n0.674544 | \n
\n
I trained the model with 4fold
\nThe results are shown in the table below.
\nEnsemble means ensemble of 3 models with same weight.
| models | \ncv score | \nthreshold | \npublic score | \nprivate score | \n
|---|---|---|---|---|
| 3D Resnet152 | \n0.64 / 0.71 / 0.71 / 0.69 | \n0.5 | \n0.78 | \n- | \n
| 3D Resnet200 | \n0.66 / 0.71 / 0.69 / 0.66 | \n0.5 | \n0.77 | \n- | \n
| 3D Resnext101 | \n0.61 / 0.72 / 0.71 / 0.64 | \n0.5 | \n0.77 | \n- | \n
| Ensemble | \n0.66 / 0.762/ 0.723 / 0.68 | \n0.5 | \n0.795835 | \n0.663542 | \n
| Ensemble | \n0.67 / 0.764 / 0.724 / 0.69 | \n0.47 | \n0.789024 | \n0.674544 | \n
all probability set to 0.6
\n
mixup augmentation
temporal channel shuffle augmentation
3D transformer encoder(uniformerv2, video swin transformer) - cannot training… i don't know why loss didn't decrease.
spatial TTA
temporal TTA - because of my crop&paste method mentioned above, i tried temporal TTA. In other words, I tried tta by cropping 22, 20, and 18 layers in various ways and then averaging 3 predictions, sometimes performance increased and sometimes decreased. I didn't apply it because it took too much time.
other losses(tversky focal loss, fbeta loss)
\n
8.1. Github version(based on Docker)
\n\n8.2. Kaggle Notebook version
\nI'm very grateful for organizing very challenging and fantastic challenge.
\nI feel like there are a lot of really great people and I respect them all.
\nI would like to express my gratitude to both the organizers and participants of the competition.
\n
I would like to describe my solution in the order of contribution of performance improvement.
\nThe number of ★ in the description below means the degree of contribution to performance improvement.
\n
As you can see in the figure above, i thought that particular layers of fragment would not correspond to the same layers of other fragment. Using these points, i came up with an augmentation method that can give strong regularization to the model by reflecting the characteristics of the data.
\n
As you can see in the figure above, there are 3 steps.
\n1) temporal random crop
2) random paste
\n3) random cutout
\nThe above three processes can be seen simply by looking at the Pytorch Dataset code part below.
\n
\n
As shown in the figure below, I saved the predicted mask for each epoch. The file name has the epoch, score, tp, fp, and fn, of which the fp value was important. Even if the score was similar, if the fp was large, it tended to be very bad in the public score. Therefore, I wanted to select a weight with a small fp value and a high score value, and I selected an appropriate fp value for each fold through several submissions. In addition, I tried to increase the generalization performance by ensemble models with a range of fp values.
\n
\n
I ensembles 3 models.
\n3D resnet152, 3D resnet200, 3D resnext101 with unet-like decoder.
Refered models was firstly inspired by JEBASTIN NADAR (https://www.kaggle.com/code/samfc10/vesuvius-challenge-3d-resnet-training) and applied. Thanks for your effort and respect you.
\n3D resnet152, 3D resnet200 is from https://github.com/kenshohara/3D-ResNets-PyTorch
\n3D resnext101 is from https://github.com/okankop/Efficient-3DCNNs
\nAll models have mit licenses and are not against the rules.
3D Resnet was pretrained on Kinetic 710, and 3D resnext101 was pretrained on Kinetic 600.
\nThere was a huge difference between being trained in kinetics and not being able to. Therefore, during the competition, I tried to find a model trained in the kinetic 600 or 700.
All models are combined unet-like simple decoder. Features from 3D encoder are upsampled and concatenated using decoder.
\ndetails are in code.
\n
\n
As shown in the table below, the two models are the ones I submitted finally.
\nEnsemble 2 had a lower public score than ensemble1, but performed better in cross validation. There were other models that could be adopted, but considering that it came out well in the cross validation, it was adopted, and the performance was improved more in the private, resulting in a better first place.
| models | \ncv score | \nthreshold | \npublic score | \nprivate score | \n
|---|---|---|---|---|
| Ensemble 1 | \n0.66 / 0.762/ 0.723 / 0.68 | \n0.5 | \n0.795835 | \n0.663542 | \n
| Ensemble 2 | \n0.67 / 0.764 / 0.724 / 0.69 | \n0.47 | \n0.789024 | \n0.674544 | \n
\n
I trained the model with 4fold
\nThe results are shown in the table below.
\nEnsemble means ensemble of 3 models with same weight.
| models | \ncv score | \nthreshold | \npublic score | \nprivate score | \n
|---|---|---|---|---|
| 3D Resnet152 | \n0.64 / 0.71 / 0.71 / 0.69 | \n0.5 | \n0.78 | \n- | \n
| 3D Resnet200 | \n0.66 / 0.71 / 0.69 / 0.66 | \n0.5 | \n0.77 | \n- | \n
| 3D Resnext101 | \n0.61 / 0.72 / 0.71 / 0.64 | \n0.5 | \n0.77 | \n- | \n
| Ensemble | \n0.66 / 0.762/ 0.723 / 0.68 | \n0.5 | \n0.795835 | \n0.663542 | \n
| Ensemble | \n0.67 / 0.764 / 0.724 / 0.69 | \n0.47 | \n0.789024 | \n0.674544 | \n
all probability set to 0.6
\n
mixup augmentation
temporal channel shuffle augmentation
3D transformer encoder(uniformerv2, video swin transformer) - cannot training… i don't know why loss didn't decrease.
spatial TTA
temporal TTA - because of my crop&paste method mentioned above, i tried temporal TTA. In other words, I tried tta by cropping 22, 20, and 18 layers in various ways and then averaging 3 predictions, sometimes performance increased and sometimes decreased. I didn't apply it because it took too much time.
other losses(tversky focal loss, fbeta loss)
\n
8.1. Github version(based on Docker)
\n\n8.2. Kaggle Notebook version
\nThanks for sharing and explaining your project! it makes the difference.
", "rawMarkdown": "Thanks for sharing and explaining your project! it makes the difference.", "votes": null }, { "id": "2319408", "postDate": "06/27/2023 05:37:19", "content": "I publised the (docker based) code on my github.
\n(https://github.com/AhnHeeYoung/Competition/tree/master/kaggle)
Thank you!
", "rawMarkdown": "I publised the (docker based) code on my github.\n(https://github.com/AhnHeeYoung/Competition/tree/master/kaggle)\n\nThank you!", "votes": null } ], "comments": [ { "id": 2317415, "author_name": "guillermoperezg", "author_url": "", "post_date": "06/25/2023 17:31:46", "content": "Thanks for sharing and explaining your project! it makes the difference.
", "votes": null, "replies": [] }, { "id": 2319408, "author_name": "ahnheeyoung1", "author_url": "", "post_date": "06/27/2023 05:37:19", "content": "I publised the (docker based) code on my github.
\n(https://github.com/AhnHeeYoung/Competition/tree/master/kaggle)
Thank you!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2306168": "# Acknowledgement\nI'm very grateful for organizing very challenging and fantastic challenge.\nI feel like there are a lot of really great people and I respect them all.\nI would like to express my gratitude to both the organizers and participants of the competition.\n\n