I use TensorFlow framework in this competition
\nI want to share which parts help improve your score.
Tensorflow
\nColab Tensorflow TPU is really fast. By using Tensorflow, You can really experiment with many things. I think learning tensorflow is better than using convenient PyTorch if you don't have many multiple GPUS.
\nI also have RTX 3090 but this isn't enough for this competition.
Gem pooling
\nIn my case, gem pooling was slightly better than average pooling
Arcface
\nI had tested other algorithms related to metric learning like Adacos, the Afcface is the best.
Data & Image size
\nWe can see a lot of datasets like original dataset of this competition, detic crop, backfin crop, full annotations, etc,,
\nI tested a lot and found an optimal one for me.
\nbut I have a plan to try backfin crop since the backfin crop hasn't been tried yet.
\nIt seems that the optimal size and algorithm are different for each data.
\nI would not tell what settings are best to be left for your try, but anyway, this part is the most crucial
Batch Size
\nSince I am not sure about which part of batch size helps in model performance in deeply, anyway large Batch size worked for me
Ensemble different architectures
\nFor example, when I ensembled efficientnet B5,B6, it boosted a score.
\nYou can concatenate embeddings from each model and run KNN algorithm to find matches.
Still the listed are all I have tried and nothing deviates from here. (2022-03-26)
\nNow I am interested in post processing but currently i don't know how to do it
I use TensorFlow framework in this competition
\nI want to share which parts help improve your score.
Tensorflow
\nColab Tensorflow TPU is really fast. By using Tensorflow, You can really experiment with many things. I think learning tensorflow is better than using convenient PyTorch if you don't have many multiple GPUS.
\nI also have RTX 3090 but this isn't enough for this competition.
Gem pooling
\nIn my case, gem pooling was slightly better than average pooling
Arcface
\nI had tested other algorithms related to metric learning like Adacos, the Afcface is the best.
Data & Image size
\nWe can see a lot of datasets like original dataset of this competition, detic crop, backfin crop, full annotations, etc,,
\nI tested a lot and found an optimal one for me.
\nbut I have a plan to try backfin crop since the backfin crop hasn't been tried yet.
\nIt seems that the optimal size and algorithm are different for each data.
\nI would not tell what settings are best to be left for your try, but anyway, this part is the most crucial
Batch Size
\nSince I am not sure about which part of batch size helps in model performance in deeply, anyway large Batch size worked for me
Ensemble different architectures
\nFor example, when I ensembled efficientnet B5,B6, it boosted a score.
\nYou can concatenate embeddings from each model and run KNN algorithm to find matches.
Still the listed are all I have tried and nothing deviates from here. (2022-03-26)
\nNow I am interested in post processing but currently i don't know how to do it
Thank you for sharing. Comments to your post.
\nPytorch … I am sure people from TOP uses Pytorch. I know Tensorflow in this competition play important part of the game but … I prefer Pytorch over TF for many reasons …
GemPooling - in my case we have better results using AvgPooling … but … you have better result so must check this again.
I completly agree - main boost/jump we almost everytime get when we changed something in data.
Can i ask you why many people prefer Pytorch over TF? I'm learning TF as my first framework and when it came to reading code, almost everyone use Pytorch. Is it about the accuracy or something.
", "rawMarkdown": "Can i ask you why many people prefer Pytorch over TF? I'm learning TF as my first framework and when it came to reading code, almost everyone use Pytorch. Is it about the accuracy or something. ", "votes": 2 }, { "id": 1735343, "postDate": "2022-03-26T06:13:23.613Z", "content": "Flexibility …. \"Easy to do common things …. and still easy to do uncommon things\". TF \"Easy to do common things … and hard to do uncommon things\".
\nAnd I feel that have control over everything ….
\nTF … error … messages can be frustrating ….
", "rawMarkdown": "Flexibility .... \"Easy to do common things .... and still easy to do uncommon things\". TF \"Easy to do common things ... and hard to do uncommon things\".\n\nAnd I feel that have control over everything ....\n\nTF ... error ... messages can be frustrating .... ", "votes": 9 }, { "id": 1735387, "postDate": "2022-03-26T07:23:48.750Z", "content": "In my case I also have better results using AvgPooling than Gem pooling….
", "rawMarkdown": "In my case I also have better results using AvgPooling than Gem pooling....", "votes": 2 }, { "id": 1735411, "postDate": "2022-03-26T08:05:43.367Z", "content": "@remekkinas I totally agree with you about your thinking about pytorch. but with using tensorflow, at least we can try many things to check what option leads to a boost
", "rawMarkdown": "@remekkinas I totally agree with you about your thinking about pytorch. but with using tensorflow, at least we can try many things to check what option leads to a boost" }, { "id": 1735420, "postDate": "2022-03-26T08:18:47.267Z", "content": "I was thinking this way until I started wasting my time solving puzzles in style “Segmentation fault #23339 error number #0098 at function blahblahnlahblah_impossible_to_check_blahh”
\nMoreover I think that Pytorch community doing great job to provide us way to fast prototyping - lighting, timm, Catalyst, yolo series for CV … etc. etc. Look on TF object detection framework it is outdated … and what it is??? Joke?
\nMy first choice is Pytorch and spend a lot of time learning it. I use TF/Keras as well. I think that the ideal situation is then you learn and use both but … first choice is Pytorch for me.
", "rawMarkdown": "I was thinking this way until I started wasting my time solving puzzles in style “Segmentation fault #23339 error number #0098 at function blahblahnlahblah_impossible_to_check_blahh” \n\nMoreover I think that Pytorch community doing great job to provide us way to fast prototyping - lighting, timm, Catalyst, yolo series for CV … etc. etc. Look on TF object detection framework it is outdated … and what it is??? Joke? \n\nMy first choice is Pytorch and spend a lot of time learning it. I use TF/Keras as well. I think that the ideal situation is then you learn and use both but … first choice is Pytorch for me.", "votes": 1 }, { "id": 1735425, "postDate": "2022-03-26T08:38:04.460Z", "content": "But sadly Pytorch + TPU is a nightmare
", "rawMarkdown": "But sadly Pytorch + TPU is a nightmare", "votes": 3 }, { "id": 1735442, "postDate": "2022-03-26T09:04:07.013Z", "content": "Another one news … from tomorrow - Pytorch - Avalanche: https://medium.com/pytorch/avalanche-and-end-to-end-library-for-continual-learning-based-on-pytorch-a99cf5661a0d
", "rawMarkdown": "Another one news ... from tomorrow - Pytorch - Avalanche: https://medium.com/pytorch/avalanche-and-end-to-end-library-for-continual-learning-based-on-pytorch-a99cf5661a0d", "votes": 1 }, { "id": 1735598, "postDate": "2022-03-26T12:36:39.810Z", "content": "@remekkinas i'm deeply interested in the catastrophic forgetting when finetuning models
\nthis can be a good source for me ! thanks!
Good post Pytorch vs TF …. https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/
", "rawMarkdown": "Good post Pytorch vs TF …. https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/", "votes": 2 }, { "id": 1737386, "postDate": "2022-03-28T11:50:39.383Z", "content": "77% of Deep Learning solutions used PyTorch (up from 72% last year)
\n", "rawMarkdown": "77% of Deep Learning solutions used PyTorch (up from 72% last year)\n\nhttps://mlcontests.com/\n", "votes": 1 }, { "id": 1738556, "postDate": "2022-03-29T11:18:03.977Z", "content": "Hello, how to use pytorch in TPU?
", "rawMarkdown": "Hello, how to use pytorch in TPU?" }, { "id": 1739705, "postDate": "2022-03-30T09:20:10.713Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1741764, "postDate": "2022-04-01T06:06:40.520Z", "content": "When I tried it, I found it isn't faster than GPU….
", "rawMarkdown": "When I tried it, I found it isn't faster than GPU....", "votes": 1 } ] }, { "id": 1744425, "postDate": "2022-04-04T01:13:31.517Z", "content": "Thank you for your sharing. Waiting for update in future!
", "rawMarkdown": "Thank you for your sharing. Waiting for update in future!", "votes": 1 }, { "id": 1737323, "postDate": "2022-03-28T10:54:54.897Z", "content": "Thanks for the post! I just wanna ask how you approached the ensembling of different architectures. Wanted to try it out myself for a different competition but not really sure as to where to start from
", "rawMarkdown": "Thanks for the post! I just wanna ask how you approached the ensembling of different architectures. Wanted to try it out myself for a different competition but not really sure as to where to start from ", "votes": 1, "replies": [ { "id": 1737614, "postDate": "2022-03-28T15:00:32.537Z", "content": "If you use arcface,
\nfor example, a network can be like this
\ninput -> Efficientnet -> Global Average pooling -> Dense(512) -> Arcface
\nYou can take the output of dense(512), then you have 512-dimensional embeddings
\nyou can train another model then again take 512-dimensional embeddings
\nthen you will have two 512-dimensional embeddings
\nyou can concatenate two embeddings, then embedding becomes 1024-dimensional embeddings
\nwith this, you can use KNN to find matches
I have been concatenating the embeddings on the WRONG axis for so long. Thanks for the pointer :). Now I can see the major boost from +0.001 (wrong concatenation) to +0.008 (correct concatenation).
", "rawMarkdown": "I have been concatenating the embeddings on the WRONG axis for so long. Thanks for the pointer :). Now I can see the major boost from +0.001 (wrong concatenation) to +0.008 (correct concatenation).", "votes": 3 }, { "id": 1738143, "postDate": "2022-03-29T04:14:34.387Z", "content": "Oh whao, utilizing KNN in this scenario? Is that common? Sorry for sounding very lost cause I just got started in deep learning and don't have much experience with image data.
", "rawMarkdown": "Oh whao, utilizing KNN in this scenario? Is that common? Sorry for sounding very lost cause I just got started in deep learning and don't have much experience with image data.", "votes": 1 }, { "id": 1738156, "postDate": "2022-03-29T04:28:35.680Z", "content": "@kimmik123 can use either the softmax outcome or KNN by using embeddings. In my case, KNN is better
\n@thakurudit nice improvement! keep going!
Thanks for sharing, in my case better results using AvgPooling too. I am also use the colab + TPU and it is very help in this competition.
", "rawMarkdown": "Thanks for sharing, in my case better results using AvgPooling too. I am also use the colab + TPU and it is very help in this competition.", "votes": 1 }, { "id": 1735440, "postDate": "2022-03-26T09:01:25.230Z", "content": "Thanks for sharing the information.
\nAccording to my experiment, Avg Pooling performed better than GeM Pooling.
\nI will give it another try!
I also want to know why the avg pooling doesn't work in my case!
\nGood Luck!
I experimented again and found that GeM is better in some conditions.
\nOnce again, thank you for sharing this great information!
@deepkim Is it possible to train a few models and concat their embeddings and then take those as our inputs for neural network with dropout and arcface to get better accuracy on validation data? I had this thought and want to give it a try.
\nIf it is possible can anyone confirm?
The other thing I thought was that an we make different models specific to the data for eg. dolphins and whales and the we generate embeddings specific to them and at the time of prediction we can rely on the threshold value to select the model to use for each specific test image so to get good prediction.
", "rawMarkdown": "@deepkim Is it possible to train a few models and concat their embeddings and then take those as our inputs for neural network with dropout and arcface to get better accuracy on validation data? I had this thought and want to give it a try.\nIf it is possible can anyone confirm?\n\nThe other thing I thought was that an we make different models specific to the data for eg. dolphins and whales and the we generate embeddings specific to them and at the time of prediction we can rely on the threshold value to select the model to use for each specific test image so to get good prediction.", "replies": [ { "id": 1745124, "postDate": "2022-04-04T16:11:43.387Z", "content": "@jainishsavalia the second thing seems smart. but for the first one, i have no idea about it
", "rawMarkdown": "@jainishsavalia the second thing seems smart. but for the first one, i have no idea about it" } ] }, { "id": 1737594, "postDate": "2022-03-28T14:51:32.463Z", "content": "I didn't see major improvements in ensembling embeddings from different models - only a low boost of around +0.001.
", "rawMarkdown": "I didn't see major improvements in ensembling embeddings from different models - only a low boost of around +0.001.", "replies": [ { "id": 1737619, "postDate": "2022-03-28T15:04:53.680Z", "content": "For me, it boosted my score a lot
", "rawMarkdown": "For me, it boosted my score a lot\n", "votes": 1 }, { "id": 1737622, "postDate": "2022-03-28T15:06:58.613Z", "content": "I tried two things:-
\nWill try the correct way.
", "rawMarkdown": "I tried two things:-\n1. Taking mean of all the embeddings np.mean(model 1+model 2): +0.001\n2. Concatenating both the embeddings: +0.001 (NVM was doing it wrong lmao)\n\nWill try the correct way.", "votes": 1 } ] }, { "id": 1737520, "postDate": "2022-03-28T13:56:58.213Z", "content": "Any reason why \"gem pooling was slightly better than average pooling\" ?
", "rawMarkdown": "Any reason why \"gem pooling was slightly better than average pooling\" ?", "replies": [ { "id": 1745122, "postDate": "2022-04-04T16:10:04.857Z", "content": "@mohandass sorry for the late reply,
\nI think gem pooling works when local features matter
\nIn the landmark competition held last year, many used gem pooling. so i adopted it!
Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": 1 }, { "id": 1736215, "postDate": "2022-03-27T05:33:27.543Z", "content": "Thanks for ideas
", "rawMarkdown": "Thanks for ideas", "votes": 1 }, { "id": 1735286, "postDate": "2022-03-26T04:54:06.460Z", "content": "Thanks for ideas👏
", "rawMarkdown": "Thanks for ideas👏", "votes": 1 }, { "id": 1751216, "postDate": "2022-04-10T14:07:49.263Z", "content": "Thanks a lot for sharing this!
", "rawMarkdown": "Thanks a lot for sharing this!" }, { "id": 1743013, "postDate": "2022-04-02T14:00:10.477Z", "content": "Thanks for your insight!
", "rawMarkdown": "Thanks for your insight!" } ], "comments": [ { "id": 1735331, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-03-26T05:57:19.917000", "content": "Thank you for sharing. Comments to your post.
\nPytorch … I am sure people from TOP uses Pytorch. I know Tensorflow in this competition play important part of the game but … I prefer Pytorch over TF for many reasons …
GemPooling - in my case we have better results using AvgPooling … but … you have better result so must check this again.
I completly agree - main boost/jump we almost everytime get when we changed something in data.
Can i ask you why many people prefer Pytorch over TF? I'm learning TF as my first framework and when it came to reading code, almost everyone use Pytorch. Is it about the accuracy or something.
", "votes": 2, "replies": [] }, { "id": 1735343, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-03-26T06:13:23.613000", "content": "Flexibility …. \"Easy to do common things …. and still easy to do uncommon things\". TF \"Easy to do common things … and hard to do uncommon things\".
\nAnd I feel that have control over everything ….
\nTF … error … messages can be frustrating ….
", "votes": 9, "replies": [] }, { "id": 1735387, "author_name": "老肥", "author_url": "", "post_date": "2022-03-26T07:23:48.750000", "content": "In my case I also have better results using AvgPooling than Gem pooling….
", "votes": 2, "replies": [] }, { "id": 1735411, "author_name": "kaggler", "author_url": "", "post_date": "2022-03-26T08:05:43.367000", "content": "@remekkinas I totally agree with you about your thinking about pytorch. but with using tensorflow, at least we can try many things to check what option leads to a boost
", "votes": 0, "replies": [] }, { "id": 1735420, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-03-26T08:18:47.267000", "content": "I was thinking this way until I started wasting my time solving puzzles in style “Segmentation fault #23339 error number #0098 at function blahblahnlahblah_impossible_to_check_blahh”
\nMoreover I think that Pytorch community doing great job to provide us way to fast prototyping - lighting, timm, Catalyst, yolo series for CV … etc. etc. Look on TF object detection framework it is outdated … and what it is??? Joke?
\nMy first choice is Pytorch and spend a lot of time learning it. I use TF/Keras as well. I think that the ideal situation is then you learn and use both but … first choice is Pytorch for me.
", "votes": 1, "replies": [] }, { "id": 1735425, "author_name": "Phat Tran", "author_url": "", "post_date": "2022-03-26T08:38:04.460000", "content": "But sadly Pytorch + TPU is a nightmare
", "votes": 3, "replies": [] }, { "id": 1735442, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-03-26T09:04:07.013000", "content": "Another one news … from tomorrow - Pytorch - Avalanche: https://medium.com/pytorch/avalanche-and-end-to-end-library-for-continual-learning-based-on-pytorch-a99cf5661a0d
", "votes": 1, "replies": [] }, { "id": 1735598, "author_name": "kaggler", "author_url": "", "post_date": "2022-03-26T12:36:39.810000", "content": "@remekkinas i'm deeply interested in the catastrophic forgetting when finetuning models
\nthis can be a good source for me ! thanks!
Good post Pytorch vs TF …. https://www.assemblyai.com/blog/pytorch-vs-tensorflow-in-2022/
", "votes": 2, "replies": [] }, { "id": 1737386, "author_name": "Yangranran", "author_url": "", "post_date": "2022-03-28T11:50:39.383000", "content": "77% of Deep Learning solutions used PyTorch (up from 72% last year)
\n", "votes": 1, "replies": [] }, { "id": 1738556, "author_name": "Zekun", "author_url": "", "post_date": "2022-03-29T11:18:03.977000", "content": "Hello, how to use pytorch in TPU?
", "votes": 0, "replies": [] }, { "id": 1739705, "author_name": "", "author_url": "", "post_date": "2022-03-30T09:20:10.713000", "content": "", "votes": 0, "replies": [] }, { "id": 1741764, "author_name": "Zekun", "author_url": "", "post_date": "2022-04-01T06:06:40.520000", "content": "When I tried it, I found it isn't faster than GPU….
", "votes": 1, "replies": [] } ] }, { "id": 1744425, "author_name": "Tan Phan", "author_url": "", "post_date": "2022-04-04T01:13:31.517000", "content": "Thank you for your sharing. Waiting for update in future!
", "votes": 1, "replies": [] }, { "id": 1737323, "author_name": "Kim Hyun Bin", "author_url": "", "post_date": "2022-03-28T10:54:54.897000", "content": "Thanks for the post! I just wanna ask how you approached the ensembling of different architectures. Wanted to try it out myself for a different competition but not really sure as to where to start from
", "votes": 1, "replies": [ { "id": 1737614, "author_name": "kaggler", "author_url": "", "post_date": "2022-03-28T15:00:32.537000", "content": "If you use arcface,
\nfor example, a network can be like this
\ninput -> Efficientnet -> Global Average pooling -> Dense(512) -> Arcface
\nYou can take the output of dense(512), then you have 512-dimensional embeddings
\nyou can train another model then again take 512-dimensional embeddings
\nthen you will have two 512-dimensional embeddings
\nyou can concatenate two embeddings, then embedding becomes 1024-dimensional embeddings
\nwith this, you can use KNN to find matches
I have been concatenating the embeddings on the WRONG axis for so long. Thanks for the pointer :). Now I can see the major boost from +0.001 (wrong concatenation) to +0.008 (correct concatenation).
", "votes": 3, "replies": [] }, { "id": 1738143, "author_name": "Kim Hyun Bin", "author_url": "", "post_date": "2022-03-29T04:14:34.387000", "content": "Oh whao, utilizing KNN in this scenario? Is that common? Sorry for sounding very lost cause I just got started in deep learning and don't have much experience with image data.
", "votes": 1, "replies": [] }, { "id": 1738156, "author_name": "kaggler", "author_url": "", "post_date": "2022-03-29T04:28:35.680000", "content": "@kimmik123 can use either the softmax outcome or KNN by using embeddings. In my case, KNN is better
\n@thakurudit nice improvement! keep going!
Thanks for sharing, in my case better results using AvgPooling too. I am also use the colab + TPU and it is very help in this competition.
", "votes": 1, "replies": [] }, { "id": 1735440, "author_name": "k_s", "author_url": "", "post_date": "2022-03-26T09:01:25.230000", "content": "Thanks for sharing the information.
\nAccording to my experiment, Avg Pooling performed better than GeM Pooling.
\nI will give it another try!
I also want to know why the avg pooling doesn't work in my case!
\nGood Luck!
I experimented again and found that GeM is better in some conditions.
\nOnce again, thank you for sharing this great information!
@deepkim Is it possible to train a few models and concat their embeddings and then take those as our inputs for neural network with dropout and arcface to get better accuracy on validation data? I had this thought and want to give it a try.
\nIf it is possible can anyone confirm?
The other thing I thought was that an we make different models specific to the data for eg. dolphins and whales and the we generate embeddings specific to them and at the time of prediction we can rely on the threshold value to select the model to use for each specific test image so to get good prediction.
", "votes": 0, "replies": [ { "id": 1745124, "author_name": "kaggler", "author_url": "", "post_date": "2022-04-04T16:11:43.387000", "content": "@jainishsavalia the second thing seems smart. but for the first one, i have no idea about it
", "votes": 0, "replies": [] } ] }, { "id": 1737594, "author_name": "ilovepotatoes", "author_url": "", "post_date": "2022-03-28T14:51:32.463000", "content": "I didn't see major improvements in ensembling embeddings from different models - only a low boost of around +0.001.
", "votes": 0, "replies": [ { "id": 1737619, "author_name": "kaggler", "author_url": "", "post_date": "2022-03-28T15:04:53.680000", "content": "For me, it boosted my score a lot
", "votes": 1, "replies": [] }, { "id": 1737622, "author_name": "ilovepotatoes", "author_url": "", "post_date": "2022-03-28T15:06:58.613000", "content": "I tried two things:-
\nWill try the correct way.
", "votes": 1, "replies": [] } ] }, { "id": 1737520, "author_name": "mohandass", "author_url": "", "post_date": "2022-03-28T13:56:58.213000", "content": "Any reason why \"gem pooling was slightly better than average pooling\" ?
", "votes": 0, "replies": [ { "id": 1745122, "author_name": "kaggler", "author_url": "", "post_date": "2022-04-04T16:10:04.857000", "content": "@mohandass sorry for the late reply,
\nI think gem pooling works when local features matter
\nIn the landmark competition held last year, many used gem pooling. so i adopted it!
Thanks for sharing.
", "votes": 1, "replies": [] }, { "id": 1736215, "author_name": "Fatma Gaber", "author_url": "", "post_date": "2022-03-27T05:33:27.543000", "content": "Thanks for ideas
", "votes": 1, "replies": [] }, { "id": 1735286, "author_name": "AlanHaBrony", "author_url": "", "post_date": "2022-03-26T04:54:06.460000", "content": "Thanks for ideas👏
", "votes": 1, "replies": [] }, { "id": 1751216, "author_name": "", "author_url": "", "post_date": "2022-04-10T14:07:49.263000", "content": "Thanks a lot for sharing this!
", "votes": 0, "replies": [] }, { "id": 1743013, "author_name": "ME2MLE", "author_url": "", "post_date": "2022-04-02T14:00:10.477000", "content": "Thanks for your insight!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1735284": "I use TensorFlow framework in this competition\nI want to share which parts help improve your score.\n\n0. Tensorflow\nColab Tensorflow TPU is really fast. By using Tensorflow, You can really experiment with many things. I think learning tensorflow is better than using convenient PyTorch if you don't have many multiple GPUS.\nI also have RTX 3090 but this isn't enough for this competition.\n\n1. Gem pooling\nIn my case, gem pooling was slightly better than average pooling\n2. Arcface\nI had tested other algorithms related to metric learning like Adacos, the Afcface is the best.\n3. Data & Image size\nWe can see a lot of datasets like original dataset of this competition, detic crop, backfin crop, full annotations, etc,,\nI tested a lot and found an optimal one for me.\nbut I have a plan to try backfin crop since the backfin crop hasn't been tried yet.\nIt seems that the optimal size and algorithm are different for each data.\nI would not tell what settings are best to be left for your try, but anyway, this part is the most crucial\n4. Batch Size\nSince I am not sure about which part of batch size helps in model performance in deeply, anyway large Batch size worked for me\n4. Ensemble different architectures\nFor example, when I ensembled efficientnet B5,B6, it boosted a score.\nYou can concatenate embeddings from each model and run KNN algorithm to find matches.\n\nStill the listed are all I have tried and nothing deviates from here. (2022-03-26)\nNow I am interested in post processing but currently i don't know how to do it \n", "1735331": "Thank you for sharing. Comments to your post.\n\n1. Pytorch ... I am sure people from TOP uses Pytorch. I know Tensorflow in this competition play important part of the game but ... I prefer Pytorch over TF for many reasons ...\n2. GemPooling - in my case we have better results using AvgPooling ... but ... you have better result so must check this again.\n\n4. I completly agree - main boost/jump we almost everytime get when we changed something in data.", "1744425": "Thank you for your sharing. Waiting for update in future!", "1737323": "Thanks for the post! I just wanna ask how you approached the ensembling of different architectures. Wanted to try it out myself for a different competition but not really sure as to where to start from ", "1736132": "Thanks for sharing, in my case better results using AvgPooling too. I am also use the colab + TPU and it is very help in this competition.", "1735440": "Thanks for sharing the information.\nAccording to my experiment, Avg Pooling performed better than GeM Pooling.\nI will give it another try!", "1743676": "@deepkim Is it possible to train a few models and concat their embeddings and then take those as our inputs for neural network with dropout and arcface to get better accuracy on validation data? I had this thought and want to give it a try.\nIf it is possible can anyone confirm?\n\nThe other thing I thought was that an we make different models specific to the data for eg. dolphins and whales and the we generate embeddings specific to them and at the time of prediction we can rely on the threshold value to select the model to use for each specific test image so to get good prediction.", "1737594": "I didn't see major improvements in ensembling embeddings from different models - only a low boost of around +0.001.", "1737520": "Any reason why \"gem pooling was slightly better than average pooling\" ?", "1736221": "", "1737378": "Thanks for sharing.", "1736215": "Thanks for ideas", "1735286": "Thanks for ideas👏", "1751216": "Thanks a lot for sharing this!", "1743013": "Thanks for your insight!" } }