First of all, thanks to all of my teammates, Venn, Tom and Alex.
\n\n- Overview\nAt the very beginning, we utilized softmax + fixed threshold to train the model but didn’t get a good result (<0.9). In order to use new_whale images in our network, we decided to do 2-class classification for each whale class. \nAfter several weeks’ experiments, senet154 performs the best and we’ve got a 0.96 (both public & private) result (single model). \nFor further improvements, we added some tricks (will discuss later) and gets 0.969, added 4 fold cross validation with class balance post processing to achieve 0.973.\nWe also tried to ensemble our se154 with other networks like seresnext101, dpn131 but didn’t get any boost.
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11474/network_.png\")
\n- Network input and training steps\ninput size is (512, 256)\nWe use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers.\n-Flip images (+0.006)\nThanks to Heng’s idea, we flip images and consider flipped id-whales as different whales and keep new whales as the same. \n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11466/hengck.png\")
- Pseudo labels (+ 0.001)\nWe added around 2000 test images (with confidence > 0.96) into our training set\n- Class balance (+0.001 ~ 0.002)\nDuring our continuous improvements (from 0.8+ to 0.96), we found that the number of labels are correlated with scores. Thus we use the follow strategy to further balance our predictions:\nFor top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, we switch class1 and class2’s positions.
\n\nFinally, congrats to all participants, especially Heng and Dene . Congrats to 3 new GM, SeuTao, David and Weimin!
\n\ncode of model\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11472/1551411492(1).png\")
--code\nhttps://github.com/earhian/Humpback-Whale-Identification-1st-
", "messages": [ { "id": 481042, "postDate": "2019-03-01T01:53:56.480Z", "content": "First of all, thanks to all of my teammates, Venn, Tom and Alex.
\n\n- Overview\nAt the very beginning, we utilized softmax + fixed threshold to train the model but didn’t get a good result (<0.9). In order to use new_whale images in our network, we decided to do 2-class classification for each whale class. \nAfter several weeks’ experiments, senet154 performs the best and we’ve got a 0.96 (both public & private) result (single model). \nFor further improvements, we added some tricks (will discuss later) and gets 0.969, added 4 fold cross validation with class balance post processing to achieve 0.973.\nWe also tried to ensemble our se154 with other networks like seresnext101, dpn131 but didn’t get any boost.
\n\n\n- Network input and training steps\ninput size is (512, 256)\nWe use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers.\n-Flip images (+0.006)\nThanks to Heng’s idea, we flip images and consider flipped id-whales as different whales and keep new whales as the same. \n
- Pseudo labels (+ 0.001)\nWe added around 2000 test images (with confidence > 0.96) into our training set\n- Class balance (+0.001 ~ 0.002)\nDuring our continuous improvements (from 0.8+ to 0.96), we found that the number of labels are correlated with scores. Thus we use the follow strategy to further balance our predictions:\nFor top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, we switch class1 and class2’s positions.
\n\nFinally, congrats to all participants, especially Heng and Dene . Congrats to 3 new GM, SeuTao, David and Weimin!
\n\ncode of model\n
--code\nhttps://github.com/earhian/Humpback-Whale-Identification-1st-
", "rawMarkdown": "First of all, thanks to all of my teammates, Venn, Tom and Alex.\n\n**- Overview**\nAt the very beginning, we utilized softmax + fixed threshold to train the model but didn’t get a good result (<0.9). In order to use new_whale images in our network, we decided to do 2-class classification for each whale class. \nAfter several weeks’ experiments, senet154 performs the best and we’ve got a 0.96 (both public & private) result (single model). \nFor further improvements, we added some tricks (will discuss later) and gets 0.969, added 4 fold cross validation with class balance post processing to achieve 0.973.\nWe also tried to ensemble our se154 with other networks like seresnext101, dpn131 but didn’t get any boost.\n\n![enter image description here][1]\n**- Network input and training steps**\ninput size is (512, 256)\nWe use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers.\n**-Flip images (+0.006)**\nThanks to Heng’s idea, we flip images and consider flipped id-whales as different whales and keep new whales as the same. \n![enter image description here][2]\n\n**- Pseudo labels (+ 0.001)**\nWe added around 2000 test images (with confidence > 0.96) into our training set\n**- Class balance (+0.001 ~ 0.002)**\nDuring our continuous improvements (from 0.8+ to 0.96), we found that the number of labels are correlated with scores. Thus we use the follow strategy to further balance our predictions:\nFor top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, we switch class1 and class2’s positions.\n\nFinally, congrats to all participants, especially Heng and Dene . Congrats to 3 new GM, SeuTao, David and Weimin!\n\ncode of model\n![enter image description here][3]\n\n--**code**\nhttps://github.com/earhian/Humpback-Whale-Identification-1st-\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11474/network_.png\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11466/hengck.png\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11472/1551411492(1).png", "votes": 230 }, { "id": 481084, "postDate": "2019-03-01T03:15:39.793Z", "content": "Very nice work!
\n\nGreat congratulation to @Earhian, @Venn, @Tomand @A.L. for getting first in this challenge!
\n\nIt is interesting to see how classification can be used in this challenge. In summary, i would the framework as end-to-end learning for:
\n\nglobal and local feature trained by metric learning
classification on these train features.
same observation is made by https://www.kaggle.com/c/humpback-whale-identification/discussion/82352, \"So out best single network is SE-ResNeXt-50, pre-trained with hard-mining triplet loss and then trained as usual by my pipeline: 0.955 lb\", however, @old-ufo did it in two steps
\n\nit will be interesting to see the results if we do it the other way round:\n- classification learned features\n- ranking by metric learning on these trained features
\n\nmore interesting if we do this in cascade (e.g. multi-task losses in different stage of the framework):
\n\n-->classify --> metric -->classify -->classify --> metric --> ....\n.
", "rawMarkdown": "Very nice work!\n\nGreat congratulation to @Earhian, @Venn, @Tomand @A.L. for getting first in this challenge!\n\nIt is interesting to see how classification can be used in this challenge. In summary, i would the framework as end-to-end learning for:\n\n- global and local feature trained by metric learning\n\n- classification on these train features.\n\nsame observation is made by https://www.kaggle.com/c/humpback-whale-identification/discussion/82352, \"So out best single network is SE-ResNeXt-50, pre-trained with hard-mining triplet loss and then trained as usual by my pipeline: 0.955 lb\", however, @old-ufo did it in two steps\n\n\nit will be interesting to see the results if we do it the other way round:\n- classification learned features\n- ranking by metric learning on these trained features\n\nmore interesting if we do this in cascade (e.g. multi-task losses in different stage of the framework):\n\n-->classify --> metric -->classify -->classify --> metric --> ....\n.\n", "votes": 10, "replies": [ { "id": 481091, "postDate": "2019-03-01T03:26:33.800Z", "content": "Cascade net looks good. Thank you a lot for your tips.
", "rawMarkdown": "Cascade net looks good. Thank you a lot for your tips.", "votes": 3 } ] }, { "id": 481903, "postDate": "2019-03-02T02:55:03.177Z", "content": "I don’t understand how flipping the images can work so well. One would assume that the patterns in the humpacks of the whales are horizontally symmetrical. By treating the flipped image as a different class aren’t you confusing the network?
", "rawMarkdown": "I don’t understand how flipping the images can work so well. One would assume that the patterns in the humpacks of the whales are horizontally symmetrical. By treating the flipped image as a different class aren’t you confusing the network? ", "votes": 7, "replies": [ { "id": 482603, "postDate": "2019-03-03T10:57:05.093Z", "content": "for most, patterns not horizontally symmetrical. :)
", "rawMarkdown": "for most, patterns not horizontally symmetrical. :)", "votes": 10 } ] }, { "id": 822214, "postDate": "2020-04-26T19:04:34.387Z", "content": "This looks good, I have been going through the solutions of some competitions lately and a lot of them are using triplet losses and arc face .
", "rawMarkdown": "This looks good, I have been going through the solutions of some competitions lately and a lot of them are using triplet losses and arc face . " }, { "id": 481817, "postDate": "2019-03-01T22:10:47.450Z", "content": "Amazing work and big win. Congrats!
", "rawMarkdown": "Amazing work and big win. Congrats!", "votes": 1 }, { "id": 481559, "postDate": "2019-03-01T14:59:45.023Z", "content": "Thanks for sharing \nHope all of you can get better grades in the future
", "rawMarkdown": "Thanks for sharing \nHope all of you can get better grades in the future", "votes": 1, "replies": [ { "id": 481588, "postDate": "2019-03-01T15:31:17.953Z", "content": "thx :)
", "rawMarkdown": "thx :)" } ] }, { "id": 481127, "postDate": "2019-03-01T04:29:42.533Z", "content": "Amazing work! I also made masks for all images that turned out pretty well but they didn't work out so well for me so I'm glad that strategy can work. I also thought about inverting the images and taking a channel from the inverted image as a 4th color channel, still thinking about that one.
", "rawMarkdown": "Amazing work! I also made masks for all images that turned out pretty well but they didn't work out so well for me so I'm glad that strategy can work. I also thought about inverting the images and taking a channel from the inverted image as a 4th color channel, still thinking about that one.", "votes": 1 }, { "id": 481050, "postDate": "2019-03-01T02:11:12.323Z", "content": "Congrats, Is it possible to share the source code ?
", "rawMarkdown": "Congrats, Is it possible to share the source code ?", "votes": 1, "replies": [ { "id": 481055, "postDate": "2019-03-01T02:18:23.563Z", "content": "yes， after clean up code.
", "rawMarkdown": "yes， after clean up code.", "votes": 12 } ] }, { "id": 943886, "postDate": "2020-07-24T16:52:01.540Z", "content": "Hello! How do you use the bounding boxes? And do you think using NetVLAD would work?
", "rawMarkdown": "Hello! How do you use the bounding boxes? And do you think using NetVLAD would work? " }, { "id": 771282, "postDate": "2020-03-14T00:54:30.680Z", "content": "👍 👍 👍
", "rawMarkdown": "👍 👍 👍 " }, { "id": 632984, "postDate": "2019-09-24T09:38:05.787Z", "content": "Hi, thx for your awesome work.I have a question about (pretrained='imagenet')which input channel is 4. loadstatic may cause problem?
", "rawMarkdown": "Hi, thx for your awesome work.I have a question about (pretrained='imagenet')which input channel is 4. loadstatic may cause problem?", "replies": [ { "id": 675405, "postDate": "2019-11-18T03:11:17.600Z", "content": "We use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nHe mentioned that he add a mask channel
Congratulation and thanks for the sharing. I have question on local feature. The default number of horizontal stripes is 6 in the PCB paper, but I did not found you explicit set the value in your code. Would you please explain more detail on how to create the local feat?
", "rawMarkdown": "Congratulation and thanks for the sharing. I have question on local feature. The default number of horizontal stripes is 6 in the PCB paper, but I did not found you explicit set the value in your code. Would you please explain more detail on how to create the local feat?", "replies": [ { "id": 507636, "postDate": "2019-04-05T02:16:33.807Z", "content": "My horizontal stripes is 8. I created local feature by horizontal avg pooling.
", "rawMarkdown": "My horizontal stripes is 8. I created local feature by horizontal avg pooling. " }, { "id": 518629, "postDate": "2019-04-17T14:38:59.580Z", "content": "What is horizontal stripes? What is PCB paper?
", "rawMarkdown": "What is *horizontal stripes*? What is *PCB paper*?", "votes": 1 }, { "id": 520227, "postDate": "2019-04-20T13:19:55.493Z", "content": "https://arxiv.org/pdf/1711.09349.pdf
", "rawMarkdown": "https://arxiv.org/pdf/1711.09349.pdf", "votes": 1 } ] }, { "id": 487708, "postDate": "2019-03-11T11:17:04.107Z", "content": "Hi, earhian:\nThanks for sharing !\nI saw your code, I find BCEWithLogitsLoss in your code is not considered when a whale belong to new_whale class. I'm not sure if it was your intention? and I'm not sure if the result will be better when considered?
```\nindexs_new = (labels != 5004 * 2).nonzero().view(-1)
\n\nif len(indexs_new) == 0:
\n return error_loss
results_nonew = results[torch.arange(0, len(results))[indexs_new], labels[indexs_new]].contiguous()
\n\ntarget_nonew = torch.ones_like(results_nonew).float().cuda()
\n\nnonew_loss = nn.BCEWithLogitsLoss(reduce=True)(results_nonew, target_nonew ）\n```
\n\nLooking forward to hearing from you.
", "rawMarkdown": "Hi, earhian:\nThanks for sharing !\nI saw your code, I find `BCEWithLogitsLoss` in your code is not considered when a whale belong to new_whale class. I'm not sure if it was your intention? and I'm not sure if the result will be better when considered?\n\n```\nindexs_new = (labels != 5004 * 2).nonzero().view(-1) \n\n if len(indexs_new) == 0: \n return error_loss \n\nresults_nonew = results[torch.arange(0, len(results))[indexs_new], labels[indexs_new]].contiguous() \n\ntarget_nonew = torch.ones_like(results_nonew).float().cuda() \n\nnonew_loss = nn.BCEWithLogitsLoss(reduce=True)(results_nonew, target_nonew ）\n```\n\nLooking forward to hearing from you.", "replies": [ { "id": 487754, "postDate": "2019-03-11T12:35:03.350Z", "content": "In error loss, target is always zero. I used nonew_loss to balance positive and negative samples
", "rawMarkdown": "In error loss, target is always zero. I used nonew_loss to balance positive and negative samples\n\n" }, { "id": 487798, "postDate": "2019-03-11T13:34:31.260Z", "content": "You bet! Thank you very much~
", "rawMarkdown": "You bet! Thank you very much~" } ] }, { "id": 485606, "postDate": "2019-03-07T16:25:32.740Z", "content": "Hi @qiaojian, congratulations for you and your team! Amazing work!
\n\nWhat is the intuition for freezing the bias for the last batch normalization?
\n\nBtw, I saw on your code that you use the playground's images. Aren't them a subset of this competition's training set?
\n\nThank you!
", "rawMarkdown": "Hi @qiaojian, congratulations for you and your team! Amazing work!\n\nWhat is the intuition for freezing the bias for the last batch normalization?\n\nBtw, I saw on your code that you use the playground's images. Aren't them a subset of this competition's training set?\n\nThank you!", "replies": [ { "id": 485893, "postDate": "2019-03-08T03:41:26.603Z", "content": "refer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias irefer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias is a common trick of reid.\nPlayground images are from https://www.kaggle.com/c/whale-categorization-playground
", "rawMarkdown": "refer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias irefer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias is a common trick of reid.\nPlayground images are from https://www.kaggle.com/c/whale-categorization-playground", "votes": 3 } ] }, { "id": 485254, "postDate": "2019-03-07T06:15:38.460Z", "content": "Congrats to your team @qiaojian !\nSimple and great model! I did not have the idea to solve this task as 5004 (10008)-way binary classification.
\n\nLet me know several details.
\n\n\n\n\ni didnt treat dataset as newwhale and not newwhale, i treat them as 5004 class and new_whale.
\n
https://github.com/earhian/Humpback-Whale-Identification-1st-/blob/master/train.py#L161\nThe output shape of the model is (10008,). For new_whale images, is the target a (10008,) tensor with all zero value?
\n\nEven if some new_whales seem to belong to known IDs, was it better to use new_whales?\nhttps://www.kaggle.com/c/humpback-whale-identification/discussion/76276
\n\nFor new_whale images, is the target a (10008,) tensor with all zero value?\n yes\nDo you think, without triplet loss, your model can achieve similar accuracy or severe degradation is expected?
\n\nI havent tried this, But i think it would get 0.96 at least.
", "rawMarkdown": "For new_whale images, is the target a (10008,) tensor with all zero value?\n yes\nDo you think, without triplet loss, your model can achieve similar accuracy or severe degradation is expected?\n\nI havent tried this, But i think it would get 0.96 at least.", "votes": 1 }, { "id": 485321, "postDate": "2019-03-07T08:32:34.493Z", "content": "Thank you for your immediate reply!
\n\n\n\n\nBut i think it would get 0.96 at least.
\n
Great!
", "rawMarkdown": "Thank you for your immediate reply!\n\n> But i think it would get 0.96 at least.\n\nGreat!" }, { "id": 507770, "postDate": "2019-04-05T07:14:59.487Z", "content": "hi earhian\ncould you give one example of how would Non New whale class encode look like in 10008 output layer\nSuppose we have an image whose Class Label index is 1 . how would its encoding look like.
", "rawMarkdown": "hi earhian\ncould you give one example of how would Non New whale class encode look like in 10008 output layer\nSuppose we have an image whose Class Label index is 1 . how would its encoding look like." } ] }, { "id": 485160, "postDate": "2019-03-07T02:34:49.950Z", "content": "Great job and congratulation!\nOne question, where can I get file model_50A_slim_ensemble.csv
", "rawMarkdown": "Great job and congratulation!\nOne question, where can I get file model_50A_slim_ensemble.csv\n", "replies": [ { "id": 485177, "postDate": "2019-03-07T03:11:39.463Z", "content": "https://drive.google.com/file/d/1hfOu3_JR0vWJkNlRhKwhqJDaF3ID2vRs/view?usp=sharing
", "rawMarkdown": " https://drive.google.com/file/d/1hfOu3_JR0vWJkNlRhKwhqJDaF3ID2vRs/view?usp=sharing " } ] }, { "id": 484485, "postDate": "2019-03-06T04:47:54.680Z", "content": "Thanks for sharing. I'm actually new to this kind of problems and I've learned a lot from public kernels and discussions.
", "rawMarkdown": "Thanks for sharing. I'm actually new to this kind of problems and I've learned a lot from public kernels and discussions. " }, { "id": 484370, "postDate": "2019-03-05T22:34:45.457Z", "content": "Awesome work! I'm not sure if you already shared it, but how do you do the masks?
", "rawMarkdown": "Awesome work! I'm not sure if you already shared it, but how do you do the masks?", "replies": [ { "id": 485296, "postDate": "2019-03-07T07:41:23.917Z", "content": "I trained a segmentation model with 450 images
", "rawMarkdown": "I trained a segmentation model with 450 images", "votes": 1 } ] }, { "id": 483794, "postDate": "2019-03-05T06:26:35.187Z", "content": "I have a question, what is the the output shape of your backbone model?
", "rawMarkdown": "I have a question, what is the the output shape of your backbone model?", "replies": [ { "id": 483811, "postDate": "2019-03-05T07:24:25.727Z", "content": "batchsize， 2048， 8, 16
", "rawMarkdown": "batchsize， 2048， 8, 16" }, { "id": 483836, "postDate": "2019-03-05T08:29:51.967Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 483487, "postDate": "2019-03-04T17:10:27.570Z", "content": "Congratulations! Nicely done!
", "rawMarkdown": "Congratulations! Nicely done!" }, { "id": 483173, "postDate": "2019-03-04T09:00:31.027Z", "content": "That's amazing ! 感谢分享，膜拜～太厉害了！
", "rawMarkdown": "That's amazing ! 感谢分享，膜拜～太厉害了！" }, { "id": 483145, "postDate": "2019-03-04T07:50:35.830Z", "content": "Very Impressive work...Thanks for sharing Learn interesting Ideas from your work...!!!
", "rawMarkdown": "Very Impressive work...Thanks for sharing Learn interesting Ideas from your work...!!!" }, { "id": 483094, "postDate": "2019-03-04T06:07:56.097Z", "content": "Great job!
", "rawMarkdown": "Great job!" }, { "id": 482932, "postDate": "2019-03-03T21:53:18Z", "content": "Congrats!
", "rawMarkdown": "Congrats!" }, { "id": 482819, "postDate": "2019-03-03T17:41:12.310Z", "content": "Congratulations!
", "rawMarkdown": "Congratulations!" }, { "id": 482803, "postDate": "2019-03-03T17:10:33.670Z", "content": "Congratulations.
", "rawMarkdown": "Congratulations." }, { "id": 482662, "postDate": "2019-03-03T13:09:38.610Z", "content": "Congrats! May I ask how you made masks? They looks so good! I tried to made them with the MaskRCNN, but my result looks not pretty well.
", "rawMarkdown": "Congrats! May I ask how you made masks? They looks so good! I tried to made them with the MaskRCNN, but my result looks not pretty well.", "replies": [ { "id": 482686, "postDate": "2019-03-03T14:12:53.760Z", "content": "simple U-net is enough.
", "rawMarkdown": "simple U-net is enough.", "votes": 2 } ] }, { "id": 482567, "postDate": "2019-03-03T08:49:36.613Z", "content": "Nice work!
", "rawMarkdown": "Nice work!" }, { "id": 482446, "postDate": "2019-03-03T01:43:20.463Z", "content": "Thanks for your contribution,which gave me a chance to start from scratch
", "rawMarkdown": "Thanks for your contribution,which gave me a chance to start from scratch" }, { "id": 482404, "postDate": "2019-03-02T23:20:57.107Z", "content": "Really cool approach! Congrats!
", "rawMarkdown": "Really cool approach! Congrats!" }, { "id": 482376, "postDate": "2019-03-02T21:48:29.943Z", "content": "Nice work!
", "rawMarkdown": "Nice work!" }, { "id": 482323, "postDate": "2019-03-02T18:36:39.420Z", "content": "congratulations and thank you for sharing your solution !
", "rawMarkdown": "congratulations and thank you for sharing your solution !" }, { "id": 482311, "postDate": "2019-03-02T17:33:16.013Z", "content": "Congratulations! And thank you for sharing this
", "rawMarkdown": "Congratulations! And thank you for sharing this" }, { "id": 482059, "postDate": "2019-03-02T09:00:13.980Z", "content": "cool!
", "rawMarkdown": "cool!" }, { "id": 482042, "postDate": "2019-03-02T08:31:00.210Z", "content": "Congratulations, and thank you for sharing!
", "rawMarkdown": "Congratulations, and thank you for sharing! " }, { "id": 481767, "postDate": "2019-03-01T20:23:49.990Z", "content": "I like the way you guys played with the loss. Congratulations!
", "rawMarkdown": "I like the way you guys played with the loss. Congratulations!" }, { "id": 481689, "postDate": "2019-03-01T18:05:08.350Z", "content": "Great Work.\nCongratulations on the win
", "rawMarkdown": "Great Work.\nCongratulations on the win" }, { "id": 481539, "postDate": "2019-03-01T14:31:03.650Z", "content": "Your local_feat code looks quite interesting.\nSo you average by width, do convolution and have descriptors for horizontal stripes. Have you compared with having descriptors for vertical stripes? I did verticals, because it looked more logical to me, but may be I was wrong.
", "rawMarkdown": "Your local_feat code looks quite interesting.\nSo you average by width, do convolution and have descriptors for horizontal stripes. Have you compared with having descriptors for vertical stripes? I did verticals, because it looked more logical to me, but may be I was wrong.", "replies": [ { "id": 481557, "postDate": "2019-03-01T14:58:09.810Z", "content": "I havent try vertical striples. I think horizontal stripes could make distance between fliplr image feature and raw image feature larger! Then we could get better \"TTA\" results.
", "rawMarkdown": "I havent try vertical striples. I think horizontal stripes could make distance between fliplr image feature and raw image feature larger! Then we could get better \"TTA\" results.", "votes": 1 } ] }, { "id": 481455, "postDate": "2019-03-01T12:27:28Z", "content": "amazing!
", "rawMarkdown": "amazing!" }, { "id": 481349, "postDate": "2019-03-01T09:42:41.250Z", "content": "Great congratulation to Earhian & Team for your win. Thanks for sharing the code and approach !!
", "rawMarkdown": "Great congratulation to Earhian & Team for your win. Thanks for sharing the code and approach !!" }, { "id": 481289, "postDate": "2019-03-01T08:14:00.113Z", "content": "Congrats @qiaojian and team. Thanks for sharing your solution and code.
\n\nBetween the Quora and the Elo competitions, I really did not have much time but joined this competition to try out a couple of simple solutions and learn a few things. In the end my result is the average result of @daisukelab's ProtoNet (Thank you) and a simple siamese network. But I got lots of reading material in terms of papers and tools posted, I am so glad I joined. Thanks everyone who posted and shared in the amazing discussions.
", "rawMarkdown": "Congrats @qiaojian and team. Thanks for sharing your solution and code.\n\nBetween the Quora and the Elo competitions, I really did not have much time but joined this competition to try out a couple of simple solutions and learn a few things. In the end my result is the average result of @daisukelab's [ProtoNet][1] (Thank you) and a simple siamese network. But I got lots of reading material in terms of papers and tools posted, I am so glad I joined. Thanks everyone who posted and shared in the amazing discussions.\n\n\n [1]: https://www.kaggle.com/c/humpback-whale-identification/discussion/81085" }, { "id": 481224, "postDate": "2019-03-01T07:15:51.507Z", "content": "Nice work and congratulation! Looking forward to the code.
", "rawMarkdown": "Nice work and congratulation! Looking forward to the code." }, { "id": 481143, "postDate": "2019-03-01T04:58:41.110Z", "content": "Congrats on first place and thanks for sharing. Is your solution pure classification ? or combine metric loss and classification loss ?
", "rawMarkdown": "Congrats on first place and thanks for sharing. Is your solution pure classification ? or combine metric loss and classification loss ?", "replies": [ { "id": 481144, "postDate": "2019-03-01T05:00:20.967Z", "content": "yes, classification with metric loss is right.
", "rawMarkdown": "yes, classification with metric loss is right.", "votes": 1 }, { "id": 481146, "postDate": "2019-03-01T05:02:09.853Z", "content": "great solution. waiting for you be the next GM
", "rawMarkdown": "great solution. waiting for you be the next GM" }, { "id": 481149, "postDate": "2019-03-01T05:03:00.947Z", "content": "hahah, thanks~
", "rawMarkdown": "hahah, thanks~" }, { "id": 481221, "postDate": "2019-03-01T07:14:21.240Z", "content": "Thanks for sharing solution. I have so many questions, here are several one:\n1. How did you choose validation strategy? Random images in Test or maybe ex. with more than 3 images?\n2. So in classificaftion (BCE), you have just two output (whale and new_whale), right?\n3. Do you have any special sampling for Tripeltes? I mean that as 'new-whales' class represent different classes inside, you cannot treat them as single one. So you cann't create a pair for them. So did you just sample them as 'negative-example'?\n4. Did you use detection? Or just RGB + Masks?
", "rawMarkdown": "Thanks for sharing solution. I have so many questions, here are several one:\n1. How did you choose validation strategy? Random images in Test or maybe ex. with more than 3 images?\n2. So in classificaftion (BCE), you have just two output (whale and new_whale), right?\n3. Do you have any special sampling for Tripeltes? I mean that as 'new-whales' class represent different classes inside, you cannot treat them as single one. So you cann't create a pair for them. So did you just sample them as 'negative-example'?\n4. Did you use detection? Or just RGB + Masks?", "votes": 3 }, { "id": 481239, "postDate": "2019-03-01T07:27:58.490Z", "content": "Very elegant solution with a single model backbone, impressive! I'm surprised that flipping images gave such a huge improvement (0.006), how many new classes did you generate using the flips?
", "rawMarkdown": "Very elegant solution with a single model backbone, impressive! I'm surprised that flipping images gave such a huge improvement (0.006), how many new classes did you generate using the flips?", "replies": [ { "id": 481130, "postDate": "2019-03-01T04:32:56.220Z", "content": "total 10008 class
", "rawMarkdown": "total 10008 class", "votes": 3 }, { "id": 481483, "postDate": "2019-03-01T13:13:13.670Z", "content": "That's really impressive! How then did you use those 10008 classes? did you just extend to your original 5005 classes as additional classes, wouldn't that be too many?
", "rawMarkdown": "That's really impressive! How then did you use those 10008 classes? did you just extend to your original 5005 classes as additional classes, wouldn't that be too many? ", "votes": 1 }, { "id": 481567, "postDate": "2019-03-01T15:08:56.443Z", "content": "for example , raw image output is output1 and size is [1, 10008], fliplr image output is output2 and size is [1, 10008].\nfinal_out = torch.sigmoid(output1)[0, :5004] + torch.sigmoid(output2)[0, 5004:]\nfinal_out = final_out / 2
", "rawMarkdown": "for example , raw image output is output1 and size is [1, 10008], fliplr image output is output2 and size is [1, 10008].\nfinal_out = torch.sigmoid(output1)[0, :5004] + torch.sigmoid(output2)[0, 5004:]\nfinal_out = final_out / 2\n", "votes": 3 } ] }, { "id": 481113, "postDate": "2019-03-01T03:55:35.670Z", "content": "Impressive work, can you explain what this hard triplet loss means?
", "rawMarkdown": "Impressive work, can you explain what this hard triplet loss means?", "replies": [ { "id": 481128, "postDate": "2019-03-01T04:32:36.493Z", "content": "hard triplet loss is metric learning loss and a variant of triplet loss.https://github.com/Yuol96/pytorch-triplet-loss
", "rawMarkdown": " hard triplet loss is metric learning loss and a variant of triplet loss.https://github.com/Yuol96/pytorch-triplet-loss " }, { "id": 481692, "postDate": "2019-03-01T18:10:13.227Z", "content": "hi earhian many congrats...\nin your github repo you have given multiple losses... which one actually got used and got\n1) Arcface ,triple or some thing else
", "rawMarkdown": "hi earhian many congrats...\nin your github repo you have given multiple losses... which one actually got used and got\n1) Arcface ,triple or some thing else" }, { "id": 481888, "postDate": "2019-03-02T02:11:30.193Z", "content": "triplet_loss and lovasz are used.By the way, why lovasz loss? Refer to https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109.\nI think method of @bestfitting is 2-class classification for each whale too.:)
", "rawMarkdown": "triplet_loss and lovasz are used.By the way, why lovasz loss? Refer to https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109.\nI think method of @bestfitting is 2-class classification for each whale too.:)" } ] }, { "id": 481105, "postDate": "2019-03-01T03:44:28.337Z", "content": "Congrats! Adding a mask channel is really a nice idea. May I ask how your team tackle the classes with only few examples(eg 1 or 2)? With heavy aug or class-balanced sampler or just train them as normal? thanks!
", "rawMarkdown": "Congrats! Adding a mask channel is really a nice idea. May I ask how your team tackle the classes with only few examples(eg 1 or 2)? With heavy aug or class-balanced sampler or just train them as normal? thanks!", "replies": [ { "id": 481107, "postDate": "2019-03-01T03:50:57.670Z", "content": "For few shot learning, my method is from https://arxiv.org/abs/1707.05574. I used heavy augment and class-balanced sampler.
", "rawMarkdown": "For few shot learning, my method is from https://arxiv.org/abs/1707.05574. I used heavy augment and class-balanced sampler.", "votes": 1 }, { "id": 481121, "postDate": "2019-03-01T04:09:33Z", "content": "thanks for the reference! Looking forward for your code :)
", "rawMarkdown": "thanks for the reference! Looking forward for your code :)" } ] }, { "id": 481097, "postDate": "2019-03-01T03:30:08.330Z", "content": "Congrats on winning the first place! I am really impressed classification model can get such high. Could you explain a bit more about your model architecture in details? like global and local features, as well as training procedures to reach 0.96? thanks a lot
", "rawMarkdown": "Congrats on winning the first place! I am really impressed classification model can get such high. Could you explain a bit more about your model architecture in details? like global and local features, as well as training procedures to reach 0.96? thanks a lot ", "replies": [ { "id": 481106, "postDate": "2019-03-01T03:49:13.223Z", "content": "training procedure is from https://arxiv.org/abs/1707.05574. For local and global feat, i have uploaded code screenshot. It is a common trick for Person Retrieval.
", "rawMarkdown": " training procedure is from https://arxiv.org/abs/1707.05574. For local and global feat, i have uploaded code screenshot. It is a common trick for Person Retrieval.", "votes": 4 }, { "id": 481108, "postDate": "2019-03-01T03:51:12.970Z", "content": "Thanks!
", "rawMarkdown": "Thanks!" } ] }, { "id": 481059, "postDate": "2019-03-01T02:29:21.837Z", "content": "cool
", "rawMarkdown": "cool" }, { "id": 481046, "postDate": "2019-03-01T02:01:12.837Z", "content": "Very elegant solution. Congratulations! \nHow long did you train SeNet154 and which batch size? I have tried it once, but it took a 6 days to do 200 epochs with bs = 8. \nWhat are SeNet local features?
", "rawMarkdown": "Very elegant solution. Congratulations! \nHow long did you train SeNet154 and which batch size? I have tried it once, but it took a 6 days to do 200 epochs with bs = 8. \nWhat are SeNet local features?", "replies": [ { "id": 481051, "postDate": "2019-03-01T02:15:01.357Z", "content": "batch size is 40 with 5 titan x pascal. i trained 8 hours and 40 epoch. local feature is part-level feature, like this http://link.zhihu.com/?target=https%3A//arxiv.org/abs/1711.09349
", "rawMarkdown": "batch size is 40 with 5 titan x pascal. i trained 8 hours and 40 epoch. local feature is part-level feature, like this http://link.zhihu.com/?target=https%3A//arxiv.org/abs/1711.09349" }, { "id": 481054, "postDate": "2019-03-01T02:17:03.563Z", "content": "40 epochs is very fast, wow.
", "rawMarkdown": "40 epochs is very fast, wow. " }, { "id": 507782, "postDate": "2019-04-05T07:34:25.300Z", "content": "Image size 512,256(h,w) would More in height but less in width... \nIf we look at the images they are more expanded along the width then height ... wouldnt that reduce image available for model to learn ?
\n\nImage frame view if height is more : |_|
", "rawMarkdown": "Image size 512,256(h,w) would More in height but less in width... \nIf we look at the images they are more expanded along the width then height ... wouldnt that reduce image available for model to learn ?\n\nImage frame view if height is more : |_|" } ] }, { "id": 481044, "postDate": "2019-03-01T01:58:57.883Z", "content": "cool
", "rawMarkdown": "cool", "replies": [ { "id": 481053, "postDate": "2019-03-01T02:15:33.453Z", "content": "thanks!
", "rawMarkdown": "thanks!" } ] }, { "id": 578447, "postDate": "2019-07-17T17:50:50.903Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 578691, "postDate": "2019-07-18T03:10:28.887Z", "content": "Global feature is avg of local feature.
", "rawMarkdown": "Global feature is avg of local feature. " }, { "id": 578789, "postDate": "2019-07-18T06:48:34.343Z", "rawMarkdown": "", "isDeleted": true }, { "id": 578822, "postDate": "2019-07-18T07:23:51.567Z", "content": "I trained tripletloss and BCE loss together.
\n\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers.
", "rawMarkdown": "I trained tripletloss and BCE loss together.\n\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers." } ] }, { "id": 518891, "postDate": "2019-04-18T02:50:06.567Z", "rawMarkdown": "", "isDeleted": true }, { "id": 523382, "postDate": "2019-04-26T05:54:48.873Z", "content": "thanks for sharing.
", "rawMarkdown": "thanks for sharing." }, { "id": 491096, "postDate": "2019-03-15T07:28:37.583Z", "content": "Cool,thanks for sharing
", "rawMarkdown": "Cool,thanks for sharing " }, { "id": 488828, "postDate": "2019-03-13T03:48:56.280Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" }, { "id": 481544, "postDate": "2019-03-01T14:38:54.257Z", "content": "Congrats!! Thanks for sharing!!!
", "rawMarkdown": "Congrats!! Thanks for sharing!!!" }, { "id": 481169, "postDate": "2019-03-01T05:32:58.410Z", "content": "Nice work, thanks for sharing
", "rawMarkdown": "Nice work, thanks for sharing" } ], "comments": [ { "id": 481084, "author_name": "hengck23", "author_url": "", "post_date": "2019-03-01T03:15:39.793000", "content": "Very nice work!
\n\nGreat congratulation to @Earhian, @Venn, @Tomand @A.L. for getting first in this challenge!
\n\nIt is interesting to see how classification can be used in this challenge. In summary, i would the framework as end-to-end learning for:
\n\nglobal and local feature trained by metric learning
classification on these train features.
same observation is made by https://www.kaggle.com/c/humpback-whale-identification/discussion/82352, \"So out best single network is SE-ResNeXt-50, pre-trained with hard-mining triplet loss and then trained as usual by my pipeline: 0.955 lb\", however, @old-ufo did it in two steps
\n\nit will be interesting to see the results if we do it the other way round:\n- classification learned features\n- ranking by metric learning on these trained features
\n\nmore interesting if we do this in cascade (e.g. multi-task losses in different stage of the framework):
\n\n-->classify --> metric -->classify -->classify --> metric --> ....\n.
", "votes": 10, "replies": [ { "id": 481091, "author_name": "earhian", "author_url": "", "post_date": "2019-03-01T03:26:33.800000", "content": "Cascade net looks good. Thank you a lot for your tips.
", "votes": 3, "replies": [] } ] }, { "id": 481903, "author_name": "FranciscoRubinCapalbo", "author_url": "", "post_date": "2019-03-02T02:55:03.177000", "content": "I don’t understand how flipping the images can work so well. One would assume that the patterns in the humpacks of the whales are horizontally symmetrical. By treating the flipped image as a different class aren’t you confusing the network?
", "votes": 7, "replies": [ { "id": 482603, "author_name": "earhian", "author_url": "", "post_date": "2019-03-03T10:57:05.093000", "content": "for most, patterns not horizontally symmetrical. :)
", "votes": 10, "replies": [] } ] }, { "id": 822214, "author_name": "Sanchit Tanwar", "author_url": "", "post_date": "2020-04-26T19:04:34.387000", "content": "This looks good, I have been going through the solutions of some competitions lately and a lot of them are using triplet losses and arc face .
", "votes": 0, "replies": [] }, { "id": 481817, "author_name": "Giba", "author_url": "", "post_date": "2019-03-01T22:10:47.450000", "content": "Amazing work and big win. Congrats!
", "votes": 1, "replies": [] }, { "id": 481559, "author_name": "", "author_url": "", "post_date": "2019-03-01T14:59:45.023000", "content": "Thanks for sharing \nHope all of you can get better grades in the future
", "votes": 1, "replies": [ { "id": 481588, "author_name": "earhian", "author_url": "", "post_date": "2019-03-01T15:31:17.953000", "content": "thx :)
", "votes": 0, "replies": [] } ] }, { "id": 481127, "author_name": "interneuron", "author_url": "", "post_date": "2019-03-01T04:29:42.533000", "content": "Amazing work! I also made masks for all images that turned out pretty well but they didn't work out so well for me so I'm glad that strategy can work. I also thought about inverting the images and taking a channel from the inverted image as a 4th color channel, still thinking about that one.
", "votes": 1, "replies": [] }, { "id": 481050, "author_name": "JM100", "author_url": "", "post_date": "2019-03-01T02:11:12.323000", "content": "Congrats, Is it possible to share the source code ?
", "votes": 1, "replies": [ { "id": 481055, "author_name": "earhian", "author_url": "", "post_date": "2019-03-01T02:18:23.563000", "content": "yes， after clean up code.
", "votes": 12, "replies": [] } ] }, { "id": 943886, "author_name": "Isabella Ramos", "author_url": "", "post_date": "2020-07-24T16:52:01.540000", "content": "Hello! How do you use the bounding boxes? And do you think using NetVLAD would work?
", "votes": 0, "replies": [] }, { "id": 771282, "author_name": "", "author_url": "", "post_date": "2020-03-14T00:54:30.680000", "content": "👍 👍 👍
", "votes": 0, "replies": [] }, { "id": 632984, "author_name": "", "author_url": "", "post_date": "2019-09-24T09:38:05.787000", "content": "Hi, thx for your awesome work.I have a question about (pretrained='imagenet')which input channel is 4. loadstatic may cause problem?
", "votes": 0, "replies": [ { "id": 675405, "author_name": "MadCoder", "author_url": "", "post_date": "2019-11-18T03:11:17.600000", "content": "We use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nHe mentioned that he add a mask channel
Congratulation and thanks for the sharing. I have question on local feature. The default number of horizontal stripes is 6 in the PCB paper, but I did not found you explicit set the value in your code. Would you please explain more detail on how to create the local feat?
", "votes": 0, "replies": [ { "id": 507636, "author_name": "earhian", "author_url": "", "post_date": "2019-04-05T02:16:33.807000", "content": "My horizontal stripes is 8. I created local feature by horizontal avg pooling.
", "votes": 0, "replies": [] }, { "id": 518629, "author_name": "", "author_url": "", "post_date": "2019-04-17T14:38:59.580000", "content": "What is horizontal stripes? What is PCB paper?
", "votes": 1, "replies": [] }, { "id": 520227, "author_name": "earhian", "author_url": "", "post_date": "2019-04-20T13:19:55.493000", "content": "https://arxiv.org/pdf/1711.09349.pdf
", "votes": 1, "replies": [] } ] }, { "id": 487708, "author_name": "WaterWood", "author_url": "", "post_date": "2019-03-11T11:17:04.107000", "content": "Hi, earhian:\nThanks for sharing !\nI saw your code, I find BCEWithLogitsLoss in your code is not considered when a whale belong to new_whale class. I'm not sure if it was your intention? and I'm not sure if the result will be better when considered?
```\nindexs_new = (labels != 5004 * 2).nonzero().view(-1)
\n\nif len(indexs_new) == 0:
\n return error_loss
results_nonew = results[torch.arange(0, len(results))[indexs_new], labels[indexs_new]].contiguous()
\n\ntarget_nonew = torch.ones_like(results_nonew).float().cuda()
\n\nnonew_loss = nn.BCEWithLogitsLoss(reduce=True)(results_nonew, target_nonew ）\n```
\n\nLooking forward to hearing from you.
", "votes": 0, "replies": [ { "id": 487754, "author_name": "earhian", "author_url": "", "post_date": "2019-03-11T12:35:03.350000", "content": "In error loss, target is always zero. I used nonew_loss to balance positive and negative samples
", "votes": 0, "replies": [] }, { "id": 487798, "author_name": "WaterWood", "author_url": "", "post_date": "2019-03-11T13:34:31.260000", "content": "You bet! Thank you very much~
", "votes": 0, "replies": [] } ] }, { "id": 485606, "author_name": "Eduardo Rocha de Andrade", "author_url": "", "post_date": "2019-03-07T16:25:32.740000", "content": "Hi @qiaojian, congratulations for you and your team! Amazing work!
\n\nWhat is the intuition for freezing the bias for the last batch normalization?
\n\nBtw, I saw on your code that you use the playground's images. Aren't them a subset of this competition's training set?
\n\nThank you!
", "votes": 0, "replies": [ { "id": 485893, "author_name": "earhian", "author_url": "", "post_date": "2019-03-08T03:41:26.603000", "content": "refer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias irefer to https://github.com/L1aoXingyu/reid_baseline/blob/master/modeling/baseline.py.\nBatchnorm without bias is a common trick of reid.\nPlayground images are from https://www.kaggle.com/c/whale-categorization-playground
", "votes": 3, "replies": [] } ] }, { "id": 485254, "author_name": "yu4u", "author_url": "", "post_date": "2019-03-07T06:15:38.460000", "content": "Congrats to your team @qiaojian !\nSimple and great model! I did not have the idea to solve this task as 5004 (10008)-way binary classification.
\n\nLet me know several details.
\n\n\n\n\ni didnt treat dataset as newwhale and not newwhale, i treat them as 5004 class and new_whale.
\n
https://github.com/earhian/Humpback-Whale-Identification-1st-/blob/master/train.py#L161\nThe output shape of the model is (10008,). For new_whale images, is the target a (10008,) tensor with all zero value?
\n\nEven if some new_whales seem to belong to known IDs, was it better to use new_whales?\nhttps://www.kaggle.com/c/humpback-whale-identification/discussion/76276
\n\nFor new_whale images, is the target a (10008,) tensor with all zero value?\n yes\nDo you think, without triplet loss, your model can achieve similar accuracy or severe degradation is expected?
\n\nI havent tried this, But i think it would get 0.96 at least.
", "votes": 1, "replies": [] }, { "id": 485321, "author_name": "yu4u", "author_url": "", "post_date": "2019-03-07T08:32:34.493000", "content": "Thank you for your immediate reply!
\n\n\n\n\nBut i think it would get 0.96 at least.
\n
Great!
", "votes": 0, "replies": [] }, { "id": 507770, "author_name": "Jaideep", "author_url": "", "post_date": "2019-04-05T07:14:59.487000", "content": "hi earhian\ncould you give one example of how would Non New whale class encode look like in 10008 output layer\nSuppose we have an image whose Class Label index is 1 . how would its encoding look like.
", "votes": 0, "replies": [] } ] }, { "id": 485160, "author_name": "RexHuang936", "author_url": "", "post_date": "2019-03-07T02:34:49.950000", "content": "Great job and congratulation!\nOne question, where can I get file model_50A_slim_ensemble.csv
", "votes": 0, "replies": [ { "id": 485177, "author_name": "earhian", "author_url": "", "post_date": "2019-03-07T03:11:39.463000", "content": "https://drive.google.com/file/d/1hfOu3_JR0vWJkNlRhKwhqJDaF3ID2vRs/view?usp=sharing
", "votes": 0, "replies": [] } ] }, { "id": 484485, "author_name": "syoya", "author_url": "", "post_date": "2019-03-06T04:47:54.680000", "content": "Thanks for sharing. I'm actually new to this kind of problems and I've learned a lot from public kernels and discussions.
", "votes": 0, "replies": [] }, { "id": 484370, "author_name": "Jimmy Z", "author_url": "", "post_date": "2019-03-05T22:34:45.457000", "content": "Awesome work! I'm not sure if you already shared it, but how do you do the masks?
", "votes": 0, "replies": [ { "id": 485296, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-03-07T07:41:23.917000", "content": "I trained a segmentation model with 450 images
", "votes": 1, "replies": [] } ] }, { "id": 483794, "author_name": "Enamor", "author_url": "", "post_date": "2019-03-05T06:26:35.187000", "content": "I have a question, what is the the output shape of your backbone model?
", "votes": 0, "replies": [ { "id": 483811, "author_name": "earhian", "author_url": "", "post_date": "2019-03-05T07:24:25.727000", "content": "batchsize， 2048， 8, 16
", "votes": 0, "replies": [] }, { "id": 483836, "author_name": "", "author_url": "", "post_date": "2019-03-05T08:29:51.967000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 483487, "author_name": "Eric", "author_url": "", "post_date": "2019-03-04T17:10:27.570000", "content": "Congratulations! Nicely done!
", "votes": 0, "replies": [] }, { "id": 483173, "author_name": "Jason Dong", "author_url": "", "post_date": "2019-03-04T09:00:31.027000", "content": "That's amazing ! 感谢分享，膜拜～太厉害了！
", "votes": 0, "replies": [] }, { "id": 483145, "author_name": "NowYSM", "author_url": "", "post_date": "2019-03-04T07:50:35.830000", "content": "Very Impressive work...Thanks for sharing Learn interesting Ideas from your work...!!!
", "votes": 0, "replies": [] }, { "id": 483094, "author_name": "Meghraj", "author_url": "", "post_date": "2019-03-04T06:07:56.097000", "content": "Great job!
", "votes": 0, "replies": [] }, { "id": 482932, "author_name": "Erica Zheng", "author_url": "", "post_date": "2019-03-03T21:53:18", "content": "Congrats!
", "votes": 0, "replies": [] }, { "id": 482819, "author_name": "Shun Kakinoki", "author_url": "", "post_date": "2019-03-03T17:41:12.310000", "content": "Congratulations!
", "votes": 0, "replies": [] }, { "id": 482803, "author_name": "Pedro Sousa", "author_url": "", "post_date": "2019-03-03T17:10:33.670000", "content": "Congratulations.
", "votes": 0, "replies": [] }, { "id": 482662, "author_name": "VladOS", "author_url": "", "post_date": "2019-03-03T13:09:38.610000", "content": "Congrats! May I ask how you made masks? They looks so good! I tried to made them with the MaskRCNN, but my result looks not pretty well.
", "votes": 0, "replies": [ { "id": 482686, "author_name": "earhian", "author_url": "", "post_date": "2019-03-03T14:12:53.760000", "content": "simple U-net is enough.
", "votes": 2, "replies": [] } ] }, { "id": 482567, "author_name": "beeyan", "author_url": "", "post_date": "2019-03-03T08:49:36.613000", "content": "Nice work!
", "votes": 0, "replies": [] }, { "id": 482446, "author_name": "sky007", "author_url": "", "post_date": "2019-03-03T01:43:20.463000", "content": "Thanks for your contribution,which gave me a chance to start from scratch
", "votes": 0, "replies": [] }, { "id": 482404, "author_name": "Fernando Henrique Fernandes", "author_url": "", "post_date": "2019-03-02T23:20:57.107000", "content": "Really cool approach! Congrats!
", "votes": 0, "replies": [] }, { "id": 482376, "author_name": "", "author_url": "", "post_date": "2019-03-02T21:48:29.943000", "content": "", "votes": 0, "replies": [] }, { "id": 482323, "author_name": "", "author_url": "", "post_date": "2019-03-02T18:36:39.420000", "content": "", "votes": 0, "replies": [] }, { "id": 482311, "author_name": "", "author_url": "", "post_date": "2019-03-02T17:33:16.013000", "content": "", "votes": 0, "replies": [] }, { "id": 482059, "author_name": "", "author_url": "", "post_date": "2019-03-02T09:00:13.980000", "content": "", "votes": 0, "replies": [] }, { "id": 482042, "author_name": "", "author_url": "", "post_date": "2019-03-02T08:31:00.210000", "content": "", "votes": 0, "replies": [] }, { "id": 481767, "author_name": "", "author_url": "", "post_date": "2019-03-01T20:23:49.990000", "content": "", "votes": 0, "replies": [] }, { "id": 481689, "author_name": "", "author_url": "", "post_date": "2019-03-01T18:05:08.350000", "content": "", "votes": 0, "replies": [] }, { "id": 481539, "author_name": "", "author_url": "", "post_date": "2019-03-01T14:31:03.650000", "content": "", "votes": 0, "replies": [ { "id": 481557, "author_name": "", "author_url": "", "post_date": "2019-03-01T14:58:09.810000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 481455, "author_name": "", "author_url": "", "post_date": "2019-03-01T12:27:28", "content": "", "votes": 0, "replies": [] }, { "id": 481349, "author_name": "", "author_url": "", "post_date": "2019-03-01T09:42:41.250000", "content": "", "votes": 0, "replies": [] }, { "id": 481289, "author_name": "", "author_url": "", "post_date": "2019-03-01T08:14:00.113000", "content": "", "votes": 0, "replies": [] }, { "id": 481224, "author_name": "", "author_url": "", "post_date": "2019-03-01T07:15:51.507000", "content": "", "votes": 0, "replies": [] }, { "id": 481143, "author_name": "", "author_url": "", "post_date": "2019-03-01T04:58:41.110000", "content": "", "votes": 0, "replies": [ { "id": 481144, "author_name": "", "author_url": "", "post_date": "2019-03-01T05:00:20.967000", "content": "", "votes": 1, "replies": [] }, { "id": 481146, "author_name": "", "author_url": "", "post_date": "2019-03-01T05:02:09.853000", "content": "", "votes": 0, "replies": [] }, { "id": 481149, "author_name": "", "author_url": "", "post_date": "2019-03-01T05:03:00.947000", "content": "", "votes": 0, "replies": [] }, { "id": 481221, "author_name": "", "author_url": "", "post_date": "2019-03-01T07:14:21.240000", "content": "", "votes": 3, "replies": [] }, { "id": 481239, "author_name": "", "author_url": "", "post_date": "2019-03-01T07:27:58.490000", "content": "", "votes": 5, "replies": [] } ] }, { "id": 481125, "author_name": "", "author_url": "", "post_date": "2019-03-01T04:25:48.407000", "content": "", "votes": 0, "replies": [ { "id": 481130, "author_name": "", "author_url": "", "post_date": "2019-03-01T04:32:56.220000", "content": "", "votes": 3, "replies": [] }, { "id": 481483, "author_name": "", "author_url": "", "post_date": "2019-03-01T13:13:13.670000", "content": "", "votes": 1, "replies": [] }, { "id": 481567, "author_name": "", "author_url": "", "post_date": "2019-03-01T15:08:56.443000", "content": "", "votes": 3, "replies": [] } ] }, { "id": 481113, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:55:35.670000", "content": "", "votes": 0, "replies": [ { "id": 481128, "author_name": "", "author_url": "", "post_date": "2019-03-01T04:32:36.493000", "content": "", "votes": 0, "replies": [] }, { "id": 481692, "author_name": "", "author_url": "", "post_date": "2019-03-01T18:10:13.227000", "content": "", "votes": 0, "replies": [] }, { "id": 481888, "author_name": "", "author_url": "", "post_date": "2019-03-02T02:11:30.193000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 481105, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:44:28.337000", "content": "", "votes": 0, "replies": [ { "id": 481107, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:50:57.670000", "content": "", "votes": 1, "replies": [] }, { "id": 481121, "author_name": "", "author_url": "", "post_date": "2019-03-01T04:09:33", "content": "", "votes": 0, "replies": [] } ] }, { "id": 481097, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:30:08.330000", "content": "", "votes": 0, "replies": [ { "id": 481106, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:49:13.223000", "content": "", "votes": 4, "replies": [] }, { "id": 481108, "author_name": "", "author_url": "", "post_date": "2019-03-01T03:51:12.970000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 481059, "author_name": "", "author_url": "", "post_date": "2019-03-01T02:29:21.837000", "content": "", "votes": 0, "replies": [] }, { "id": 481046, "author_name": "", "author_url": "", "post_date": "2019-03-01T02:01:12.837000", "content": "", "votes": 0, "replies": [ { "id": 481051, "author_name": "", "author_url": "", "post_date": "2019-03-01T02:15:01.357000", "content": "", "votes": 0, "replies": [] }, { "id": 481054, "author_name": "", "author_url": "", "post_date": "2019-03-01T02:17:03.563000", "content": "", "votes": 0, "replies": [] }, { "id": 507782, "author_name": "", "author_url": "", "post_date": "2019-04-05T07:34:25.300000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 481044, "author_name": "", "author_url": "", "post_date": "2019-03-01T01:58:57.883000", "content": "", "votes": 0, "replies": [ { "id": 481053, "author_name": "", "author_url": "", "post_date": "2019-03-01T02:15:33.453000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 578447, "author_name": "", "author_url": "", "post_date": "2019-07-17T17:50:50.903000", "content": "", "votes": 0, "replies": [ { "id": 578691, "author_name": "", "author_url": "", "post_date": "2019-07-18T03:10:28.887000", "content": "", "votes": 0, "replies": [] }, { "id": 578789, "author_name": "", "author_url": "", "post_date": "2019-07-18T06:48:34.343000", "content": "", "votes": 0, "replies": [] }, { "id": 578822, "author_name": "", "author_url": "", "post_date": "2019-07-18T07:23:51.567000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 518891, "author_name": "", "author_url": "", "post_date": "2019-04-18T02:50:06.567000", "content": "", "votes": 0, "replies": [] }, { "id": 523382, "author_name": "", "author_url": "", "post_date": "2019-04-26T05:54:48.873000", "content": "", "votes": 0, "replies": [] }, { "id": 491096, "author_name": "", "author_url": "", "post_date": "2019-03-15T07:28:37.583000", "content": "", "votes": 0, "replies": [] }, { "id": 488828, "author_name": "", "author_url": "", "post_date": "2019-03-13T03:48:56.280000", "content": "", "votes": 0, "replies": [] }, { "id": 481544, "author_name": "", "author_url": "", "post_date": "2019-03-01T14:38:54.257000", "content": "", "votes": 0, "replies": [] }, { "id": 481169, "author_name": "", "author_url": "", "post_date": "2019-03-01T05:32:58.410000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "481042": "First of all, thanks to all of my teammates, Venn, Tom and Alex.\n\n**- Overview**\nAt the very beginning, we utilized softmax + fixed threshold to train the model but didn’t get a good result (<0.9). In order to use new_whale images in our network, we decided to do 2-class classification for each whale class. \nAfter several weeks’ experiments, senet154 performs the best and we’ve got a 0.96 (both public & private) result (single model). \nFor further improvements, we added some tricks (will discuss later) and gets 0.969, added 4 fold cross validation with class balance post processing to achieve 0.973.\nWe also tried to ensemble our se154 with other networks like seresnext101, dpn131 but didn’t get any boost.\n\n![enter image description here][1]\n**- Network input and training steps**\ninput size is (512, 256)\nWe use 4 channels, RGB + masks (trained by 450 open source labels) as our input.\nStep 1: Training within all labels with >10 samples (this step helps to converge faster and easier)\nStep 2: Training with all samples, and fixed all of the networks except the last two layers.\n**-Flip images (+0.006)**\nThanks to Heng’s idea, we flip images and consider flipped id-whales as different whales and keep new whales as the same. \n![enter image description here][2]\n\n**- Pseudo labels (+ 0.001)**\nWe added around 2000 test images (with confidence > 0.96) into our training set\n**- Class balance (+0.001 ~ 0.002)**\nDuring our continuous improvements (from 0.8+ to 0.96), we found that the number of labels are correlated with scores. Thus we use the follow strategy to further balance our predictions:\nFor top 5 predictions class1 to class5, if: conf class1 – conf class 2 < 0.3, and class 2 is not used in all top 1 predictions, and class 1 has been used in top 2 predictions for many times, we switch class1 and class2’s positions.\n\nFinally, congrats to all participants, especially Heng and Dene . Congrats to 3 new GM, SeuTao, David and Weimin!\n\ncode of model\n![enter image description here][3]\n\n--**code**\nhttps://github.com/earhian/Humpback-Whale-Identification-1st-\n\n\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11474/network_.png\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11466/hengck.png\n [3]: https://storage.googleapis.com/kaggle-forum-message-attachments/481042/11472/1551411492(1).png", "481084": "Very nice work!\n\nGreat congratulation to @Earhian, @Venn, @Tomand @A.L. for getting first in this challenge!\n\nIt is interesting to see how classification can be used in this challenge. In summary, i would the framework as end-to-end learning for:\n\n- global and local feature trained by metric learning\n\n- classification on these train features.\n\nsame observation is made by https://www.kaggle.com/c/humpback-whale-identification/discussion/82352, \"So out best single network is SE-ResNeXt-50, pre-trained with hard-mining triplet loss and then trained as usual by my pipeline: 0.955 lb\", however, @old-ufo did it in two steps\n\n\nit will be interesting to see the results if we do it the other way round:\n- classification learned features\n- ranking by metric learning on these trained features\n\nmore interesting if we do this in cascade (e.g. multi-task losses in different stage of the framework):\n\n-->classify --> metric -->classify -->classify --> metric --> ....\n.\n", "481903": "I don’t understand how flipping the images can work so well. One would assume that the patterns in the humpacks of the whales are horizontally symmetrical. By treating the flipped image as a different class aren’t you confusing the network? ", "822214": "This looks good, I have been going through the solutions of some competitions lately and a lot of them are using triplet losses and arc face . ", "481817": "Amazing work and big win. Congrats!", "481559": "Thanks for sharing \nHope all of you can get better grades in the future", "481127": "Amazing work! I also made masks for all images that turned out pretty well but they didn't work out so well for me so I'm glad that strategy can work. I also thought about inverting the images and taking a channel from the inverted image as a 4th color channel, still thinking about that one.", "481050": "Congrats, Is it possible to share the source code ?", "943886": "Hello! How do you use the bounding boxes? And do you think using NetVLAD would work? ", "771282": "👍 👍 👍 ", "632984": "Hi, thx for your awesome work.I have a question about (pretrained='imagenet')which input channel is 4. loadstatic may cause problem?", "507575": "Congratulation and thanks for the sharing. I have question on local feature. The default number of horizontal stripes is 6 in the PCB paper, but I did not found you explicit set the value in your code. Would you please explain more detail on how to create the local feat?", "487708": "Hi, earhian:\nThanks for sharing !\nI saw your code, I find `BCEWithLogitsLoss` in your code is not considered when a whale belong to new_whale class. I'm not sure if it was your intention? and I'm not sure if the result will be better when considered?\n\n```\nindexs_new = (labels != 5004 * 2).nonzero().view(-1) \n\n if len(indexs_new) == 0: \n return error_loss \n\nresults_nonew = results[torch.arange(0, len(results))[indexs_new], labels[indexs_new]].contiguous() \n\ntarget_nonew = torch.ones_like(results_nonew).float().cuda() \n\nnonew_loss = nn.BCEWithLogitsLoss(reduce=True)(results_nonew, target_nonew ）\n```\n\nLooking forward to hearing from you.", "485606": "Hi @qiaojian, congratulations for you and your team! Amazing work!\n\nWhat is the intuition for freezing the bias for the last batch normalization?\n\nBtw, I saw on your code that you use the playground's images. Aren't them a subset of this competition's training set?\n\nThank you!", "485254": "Congrats to your team @qiaojian !\nSimple and great model! I did not have the idea to solve this task as 5004 (10008)-way binary classification.\n\nLet me know several details.\n\n- About top-k BCE loss\n\n> i didnt treat dataset as newwhale and not newwhale, i treat them as 5004 class and new_whale.\n\nhttps://github.com/earhian/Humpback-Whale-Identification-1st-/blob/master/train.py#L161\nThe output shape of the model is (10008,). For new_whale images, is the target a (10008,) tensor with all zero value?\n\nEven if some new_whales seem to belong to known IDs, was it better to use new_whales?\nhttps://www.kaggle.com/c/humpback-whale-identification/discussion/76276\n\n- Do you think, without triplet loss, your model can achieve similar accuracy or severe degradation is expected?", "485160": "Great job and congratulation!\nOne question, where can I get file model_50A_slim_ensemble.csv\n", "484485": "Thanks for sharing. I'm actually new to this kind of problems and I've learned a lot from public kernels and discussions. ", "484370": "Awesome work! I'm not sure if you already shared it, but how do you do the masks?", "483794": "I have a question, what is the the output shape of your backbone model?", "483487": "Congratulations! Nicely done!", "483173": "That's amazing ! 感谢分享，膜拜～太厉害了！", "483145": "Very Impressive work...Thanks for sharing Learn interesting Ideas from your work...!!!", "483094": "Great job!", "482932": "Congrats!", "482819": "Congratulations!", "482803": "Congratulations.", "482662": "Congrats! May I ask how you made masks? They looks so good! I tried to made them with the MaskRCNN, but my result looks not pretty well.", "482567": "Nice work!", "482446": "Thanks for your contribution,which gave me a chance to start from scratch", "482404": "Really cool approach! Congrats!", "482376": "Nice work!", "482323": "congratulations and thank you for sharing your solution !", "482311": "Congratulations! And thank you for sharing this", "482059": "cool!", "482042": "Congratulations, and thank you for sharing! ", "481767": "I like the way you guys played with the loss. Congratulations!", "481689": "Great Work.\nCongratulations on the win", "481539": "Your local_feat code looks quite interesting.\nSo you average by width, do convolution and have descriptors for horizontal stripes. Have you compared with having descriptors for vertical stripes? I did verticals, because it looked more logical to me, but may be I was wrong.", "481455": "amazing!", "481349": "Great congratulation to Earhian & Team for your win. Thanks for sharing the code and approach !!", "481289": "Congrats @qiaojian and team. Thanks for sharing your solution and code.\n\nBetween the Quora and the Elo competitions, I really did not have much time but joined this competition to try out a couple of simple solutions and learn a few things. In the end my result is the average result of @daisukelab's [ProtoNet][1] (Thank you) and a simple siamese network. But I got lots of reading material in terms of papers and tools posted, I am so glad I joined. Thanks everyone who posted and shared in the amazing discussions.\n\n\n [1]: https://www.kaggle.com/c/humpback-whale-identification/discussion/81085", "481224": "Nice work and congratulation! Looking forward to the code.", "481143": "Congrats on first place and thanks for sharing. Is your solution pure classification ? or combine metric loss and classification loss ?", "481125": "Very elegant solution with a single model backbone, impressive! I'm surprised that flipping images gave such a huge improvement (0.006), how many new classes did you generate using the flips?", "481113": "Impressive work, can you explain what this hard triplet loss means?", "481105": "Congrats! Adding a mask channel is really a nice idea. May I ask how your team tackle the classes with only few examples(eg 1 or 2)? With heavy aug or class-balanced sampler or just train them as normal? thanks!", "481097": "Congrats on winning the first place! I am really impressed classification model can get such high. Could you explain a bit more about your model architecture in details? like global and local features, as well as training procedures to reach 0.96? thanks a lot ", "481059": "cool", "481046": "Very elegant solution. Congratulations! \nHow long did you train SeNet154 and which batch size? I have tried it once, but it took a 6 days to do 200 epochs with bs = 8. \nWhat are SeNet local features?", "481044": "cool", "578447": "", "518891": "", "523382": "thanks for sharing.", "491096": "Cool,thanks for sharing ", "488828": "Thanks for sharing!", "481544": "Congrats!! Thanks for sharing!!!", "481169": "Nice work, thanks for sharing" } }