\n
![\"enter](\"https://github.com/bestfitting/kaggle/blob/master/cdiscount/602.png?raw=true\" "\\"\\"")
\n![\"enter](\"https://raw.githubusercontent.com/bestfitting/kaggle/master/cdiscount/284.png\" "\\"\\"")
It is an interesting competition,and also a tough one.
\nI faced the following challenges:
\n1.Large Dataset with 15+ millions images and 5000+ categories.
\n2.Some products and 1-4 images
\n3.CD/BOOK are very hard to classify.
\n4.I estimated the best overall accuracy will be 0.8 also,there are a lot of methods to choose and there are a large space to improve,so it’s very hard to win.
\n\n
\n1.Making preparations for such a big dataset
\n2.Finetuning pretrained models.(0.759/0.757,inception-resnet-v2,0.757/0.756 resnet50).
\n3.Make full use of multi-images of a product. (0.772/0.771 inception-resnet-v2 and 0.769/0.768+ Resnet50 models).
\n4.Use OCR to add semantics to the models.\n
\nAt early stage of this competition,I spent 4 days on trying to feed the images to pytorch efficiently,I I found the best way is to buy 512M SSDs.
\nI also found that if I want to store prediction results into a file,I must use sparse matrix,I spend 1.5 days on writing codes to make sure I can keep probabilities of prediction in a small file(~40M)\n
\nWe can not win such a competition only by downloading pretrained models and finetuning them,we must understand the principles behind every models.
\nAfter got some statistics of this data set,I searched and read all kinds of papers related:network structures,the optimizers,the train plan．．．．．．
\nThen I started my experiments on Resnet34,I have following result:
\n1.The network structure of almost all pretrained model are for imagenet with 1000 labels ,but our dataset have 5270 labels.If we use then directly,there will be some representation bottleneck.
\n2.I found train with SGD need more epochs to converge than ADAM.
So I modified the head of resnet34,add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270.
And I used Adam with optim.Adam(net.parameters(), lr=lr) and lr plan is:
lr = 0.0003\n if epoch > 7:\n lr = 0.0001\n if epoch > 9:\n lr = 0.00005\n if epoch > 11:\n lr = 0.00001\n\n\n
When I trained 11.5 epochs with 160x160 patch random cropped from 180*180 images,I predicted and averaged predictions of the images of product,I could get a score >0.72 on public LB.
\nThen I thought,why not add a FC layer more?So I added a 5270x5270,and I found that it improved more than 0.5
Since I can train 1 epochs in 2.5 hours on a 4x 1080Ti machine,I could do a lot of experiments,I found if the result is not so good on first 1-2 epochs,the result not be good at end,this speed up my experiments.
\nI tried but not used in the next stage:
\n1.Multi-level categories as multi-task.
\n2.Hard Example,Focal Loss,
\n3.Dilation
\n4.Dropout
\n
After I got these conclusion,I turned to resnet50,I found resnet50 is better than resnet34: 0.756 on public.
Because I am busy with my everyday job ,I just let my GPUs keep running,and fine-tuned Resnet101,resnet152,and inceptionresnetv2 and inceptionV4.\nInceptionv4 is a little weak at this stage,others could easily get a score >0.755 on public LB.
\n\n
\nLast weeks of this competition,I had more time,and I started analyzing the dataset.
\nI trained my models using images of a product separately before,but when we have a look at a product,we will treat the images as a whole.So I concatenate the images into a image,and fine-tune the models.
That’s to say,We can split the trainset into 4 parts,products with 1,2,3,4 images,if we finetune products with 4 images ,we concatenate the 4 images into a image.if we finetune products with 3 images ,we concatenate the 3 images into a image. I also finetune the model with product with only 1 images.
\nBecause I found that there are some invalid or meaningless images for classification in images for multi-images products,that will be noise for product with only on product.It was easy to finetune,the acc improved a lot after 2-3 epochs,it took no so much time.
After this step,I got a cluster of model for resnet50,a model trained with all images,4 models for 1,2,3,4 images products.I predicted the product accordingly and ensembled them,I got a big improvement.Now,Almost all the models can get acc score near 0.77 .
I made several submissions just now,I found that if I ensemble resnet50 models with inception-resnet-v2 models,we can get 0.782/0.781 on private and public LB.
\n\n
\nI found that CD and BOOK are very hard to classify,because we need know the meaning of the cover!
\nAnd I want to evaluate the performance of all kinds of algorithms OCR related for my next project,so I spent more than a week on it,I use CTPN to extract the boxes containing text,the result was quite good.Then I used CRNN to extract the text from the boxes.But I found that the text are not so good,because the images are small sized and CRNN was trained for English and I have no French dataset to retrain it,I can generate it,but I can not found suitable books without copyright limitation to extract French words.Without good words extracted from CD and BOOK cover,I could not use Word2Vec,so I can not provide semantic meaning to the model. As a last resort,I extracted the features of boxes from CRNN and feed them into my multi-input CNN,the first input is Resnet50 FC features and the second is CRNN features.
\nAfter got the prediction from this OCR-Network,I ensembled with the predictions of Resnet50 ,I make a lot of submissions in last week to confirm I did not over-fit.
\nI found that OCR solution contributed ~0.14% to my final submissions.
\n[Edit],I found that if I use OCR on 0.782 model,the result can improve 0.35%,this means that the improvement got from OCR is covered by multi-model ensemble.\n
\nI also tried densenet161,densenet169,dpn92,they are much worse than resnet50,since I have trained them,so I just ensembled them to add some diversities.
\nAfter ensembled the models,the public LB can reach 0.79.
\nTo reach my final scored,I fine-tuned the 1-image products models with 224/299 sized images accordingly,to add more diversities,I also change the head to two 4096*4096 FC layer following VGG net and ensemble them.
if you want to reproduce my solution,and if you have 4-1080Ti,you can get 0.782/0.781 on private/public LB in 7-8 days with resnet50 and inception-resnet-v2.
\n\n
\n
It is an interesting competition,and also a tough one.
\nI faced the following challenges:
\n1.Large Dataset with 15+ millions images and 5000+ categories.
\n2.Some products and 1-4 images
\n3.CD/BOOK are very hard to classify.
\n4.I estimated the best overall accuracy will be 0.8 also,there are a lot of methods to choose and there are a large space to improve,so it’s very hard to win.
\n\n
\n1.Making preparations for such a big dataset
\n2.Finetuning pretrained models.(0.759/0.757,inception-resnet-v2,0.757/0.756 resnet50).
\n3.Make full use of multi-images of a product. (0.772/0.771 inception-resnet-v2 and 0.769/0.768+ Resnet50 models).
\n4.Use OCR to add semantics to the models.\n
\nAt early stage of this competition,I spent 4 days on trying to feed the images to pytorch efficiently,I I found the best way is to buy 512M SSDs.
\nI also found that if I want to store prediction results into a file,I must use sparse matrix,I spend 1.5 days on writing codes to make sure I can keep probabilities of prediction in a small file(~40M)\n
\nWe can not win such a competition only by downloading pretrained models and finetuning them,we must understand the principles behind every models.
\nAfter got some statistics of this data set,I searched and read all kinds of papers related:network structures,the optimizers,the train plan．．．．．．
\nThen I started my experiments on Resnet34,I have following result:
\n1.The network structure of almost all pretrained model are for imagenet with 1000 labels ,but our dataset have 5270 labels.If we use then directly,there will be some representation bottleneck.
\n2.I found train with SGD need more epochs to converge than ADAM.
So I modified the head of resnet34,add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270.
And I used Adam with optim.Adam(net.parameters(), lr=lr) and lr plan is:
lr = 0.0003\n if epoch > 7:\n lr = 0.0001\n if epoch > 9:\n lr = 0.00005\n if epoch > 11:\n lr = 0.00001\n\n\n
When I trained 11.5 epochs with 160x160 patch random cropped from 180*180 images,I predicted and averaged predictions of the images of product,I could get a score >0.72 on public LB.
\nThen I thought,why not add a FC layer more?So I added a 5270x5270,and I found that it improved more than 0.5
Since I can train 1 epochs in 2.5 hours on a 4x 1080Ti machine,I could do a lot of experiments,I found if the result is not so good on first 1-2 epochs,the result not be good at end,this speed up my experiments.
\nI tried but not used in the next stage:
\n1.Multi-level categories as multi-task.
\n2.Hard Example,Focal Loss,
\n3.Dilation
\n4.Dropout
\n
After I got these conclusion,I turned to resnet50,I found resnet50 is better than resnet34: 0.756 on public.
Because I am busy with my everyday job ,I just let my GPUs keep running,and fine-tuned Resnet101,resnet152,and inceptionresnetv2 and inceptionV4.\nInceptionv4 is a little weak at this stage,others could easily get a score >0.755 on public LB.
\n\n
\nLast weeks of this competition,I had more time,and I started analyzing the dataset.
\nI trained my models using images of a product separately before,but when we have a look at a product,we will treat the images as a whole.So I concatenate the images into a image,and fine-tune the models.
That’s to say,We can split the trainset into 4 parts,products with 1,2,3,4 images,if we finetune products with 4 images ,we concatenate the 4 images into a image.if we finetune products with 3 images ,we concatenate the 3 images into a image. I also finetune the model with product with only 1 images.
\nBecause I found that there are some invalid or meaningless images for classification in images for multi-images products,that will be noise for product with only on product.It was easy to finetune,the acc improved a lot after 2-3 epochs,it took no so much time.
After this step,I got a cluster of model for resnet50,a model trained with all images,4 models for 1,2,3,4 images products.I predicted the product accordingly and ensembled them,I got a big improvement.Now,Almost all the models can get acc score near 0.77 .
I made several submissions just now,I found that if I ensemble resnet50 models with inception-resnet-v2 models,we can get 0.782/0.781 on private and public LB.
\n\n
\nI found that CD and BOOK are very hard to classify,because we need know the meaning of the cover!
\nAnd I want to evaluate the performance of all kinds of algorithms OCR related for my next project,so I spent more than a week on it,I use CTPN to extract the boxes containing text,the result was quite good.Then I used CRNN to extract the text from the boxes.But I found that the text are not so good,because the images are small sized and CRNN was trained for English and I have no French dataset to retrain it,I can generate it,but I can not found suitable books without copyright limitation to extract French words.Without good words extracted from CD and BOOK cover,I could not use Word2Vec,so I can not provide semantic meaning to the model. As a last resort,I extracted the features of boxes from CRNN and feed them into my multi-input CNN,the first input is Resnet50 FC features and the second is CRNN features.
\nAfter got the prediction from this OCR-Network,I ensembled with the predictions of Resnet50 ,I make a lot of submissions in last week to confirm I did not over-fit.
\nI found that OCR solution contributed ~0.14% to my final submissions.
\n[Edit],I found that if I use OCR on 0.782 model,the result can improve 0.35%,this means that the improvement got from OCR is covered by multi-model ensemble.\n
\nI also tried densenet161,densenet169,dpn92,they are much worse than resnet50,since I have trained them,so I just ensembled them to add some diversities.
\nAfter ensembled the models,the public LB can reach 0.79.
\nTo reach my final scored,I fine-tuned the 1-image products models with 224/299 sized images accordingly,to add more diversities,I also change the head to two 4096*4096 FC layer following VGG net and ensemble them.
if you want to reproduce my solution,and if you have 4-1080Ti,you can get 0.782/0.781 on private/public LB in 7-8 days with resnet50 and inception-resnet-v2.
\n\n
\n
lr = 0.0003\n if epoch > 7:\n lr = 0.0001\n if epoch > 9:\n lr = 0.00005\n if epoch > 11:\n lr = 0.00001\n\nWhen I trained 11.5 epochs with 160x160 patch random cropped from 180*180 images,I predicted and averaged predictions of the images of product,I could get a score >0.72 on public LB.
Thanks bestfitting for sharing about your solution and giving some insights.
\n\nBeing such efficient and reaching the first place on kaggle global ranking in just 1 year are clearly outstanding achievements... Congrats !!!
", "rawMarkdown": "Thanks bestfitting for sharing about your solution and giving some insights. \n\nBeing such efficient and reaching the first place on kaggle global ranking in just 1 year are clearly outstanding achievements... Congrats !!!", "votes": null }, { "id": "258840", "postDate": "12/17/2017 05:36:11", "content": "Congrats, thanks for sharing.\nI also fine tune by image count and order but not concatenate them, ensemble 6 model trained from subset with 1 full set, public LB can reach 0.772.
", "rawMarkdown": "Congrats, thanks for sharing.\nI also fine tune by image count and order but not concatenate them, ensemble 6 model trained from subset with 1 full set, public LB can reach 0.772.", "votes": null }, { "id": "258856", "postDate": "12/17/2017 06:55:56", "content": "Congrats! Thanks for your sharing. \nI also tried Resnet50 and Inception_resnet_v2, but my best single model could just get the public LB 0.69. Can you share your code for us to learn?
", "rawMarkdown": "Congrats! Thanks for your sharing. \nI also tried Resnet50 and Inception_resnet_v2, but my best single model could just get the public LB 0.69. Can you share your code for us to learn?", "votes": null }, { "id": "258933", "postDate": "12/17/2017 11:37:07", "content": "Thanks for sharing. How do you concatenate 4 images into a image? concatenate with image channels?
", "rawMarkdown": "Thanks for sharing. How do you concatenate 4 images into a image? concatenate with image channels?", "votes": null }, { "id": "258943", "postDate": "12/17/2017 12:17:59", "content": "Congrats! Thanks for sharing. BTW, did you do any kind of augmentation at testing stage?
", "rawMarkdown": "Congrats! Thanks for sharing. BTW, did you do any kind of augmentation at testing stage?", "votes": null }, { "id": "259009", "postDate": "12/17/2017 14:24:03", "content": "Thanks for the answer. From your discussion and my experiments, i conclude the followings (which may not be correct). Key factors for top performance are:
\n\n[1. modification of imagenet model for larger class]
\n\nImagenet pretrained models sometimes do not get good results because they are designed for 1000 classes. To improve results we need to extend the last fc layers. Bestfitting did it by stacking 2 more fc layers to the imagenet model. For my case, i did it in additional post network on extracted imagenet features (coincidentally also using 2 additional fc layers).
\n\n[2. handling of multiple images]
\n\nWe need some way to handle multiple images. Bestfitting did it by training different models for different number of images. The models are ensembled together. For me, the post network is trained to find the best combinations.
\n\n[3. OCR]
\n\nBestfitting uses OCR to improve results for CD/BOOKS. We do not use OCR.
", "rawMarkdown": "Thanks for the answer. From your discussion and my experiments, i conclude the followings (which may not be correct). Key factors for top performance are:\n\n [1. modification of imagenet model for larger class] \n\nImagenet pretrained models sometimes do not get good results because they are designed for 1000 classes. To improve results we need to extend the last fc layers. Bestfitting did it by stacking 2 more fc layers to the imagenet model. For my case, i did it in additional post network on extracted imagenet features (coincidentally also using 2 additional fc layers). \n\n [2. handling of multiple images]\n\nWe need some way to handle multiple images. Bestfitting did it by training different models for different number of images. The models are ensembled together. For me, the post network is trained to find the best combinations.\n\n [3. OCR]\n\nBestfitting uses OCR to improve results for CD/BOOKS. We do not use OCR.", "votes": null }, { "id": "259355", "postDate": "12/18/2017 08:07:04", "content": "I cropped 5 160x160 patches from 180x180 images ,left-top,right-top,middle,left-bottom,right-bottom,the score could improve 0.5 also.
", "rawMarkdown": "I cropped 5 160x160 patches from 180x180 images ,left-top,right-top,middle,left-bottom,right-bottom,the score could improve 0.5 also.", "votes": null }, { "id": "259358", "postDate": "12/18/2017 08:10:27", "content": "Please refer to these two images,you can find that if we concatenate the images into a image,our models will get the information of a product in a whole.\n
\n\n
if self.img_cnt==4 and self.mmv==1:\n img_out = np.zeros([self.img_size*2, self.img_size * 2, 3])\n else:\n img_out = np.zeros([self.img_size, self.img_size * self.img_cnt, 3])\n.....
\n\nRead images of a product then:\n h, w = img.shape[0:2]\n if self.img_size != h or self.img_size != w:\n img = cv2.resize(img, (self.img_size, self.img_size))\n img_size=self.img_size\n if self.img_cnt == 4 and self.mmv == 1:\n if array_idx<2:\n img_out[ :img_size, array_idx * img_size:(array_idx + 1) * img_size,:] = img\n else:\n img_out[img_size:, (array_idx-2) * img_size:((array_idx-2) + 1) * img_size, :] = img\n else:\n img_out[ :, array_idx * img_size:(array_idx + 1) * img_size,:] = img\nYou can refer to this.please use Adam and scheduler in this post:
\nresnet for cdicount
I noticed that the order of images should be varied,so when I trained my model,I randomized the order of images of a product.
", "rawMarkdown": "I noticed that the order of images should be varied,so when I trained my model,I randomized the order of images of a product.", "votes": null }, { "id": "259367", "postDate": "12/18/2017 08:27:33", "content": "Thank you! I think the most important factors are the kaggle rank system and I am so lucky in several competitions including this one.
", "rawMarkdown": "Thank you! I think the most important factors are the kaggle rank system and I am so lucky in several competitions including this one.", "votes": null }, { "id": "259371", "postDate": "12/18/2017 08:40:03", "content": "Yes,when I read your post after competition,I had the similar feeling.we tried to solve this problem in different way,but the methods are similar.
", "rawMarkdown": "Yes,when I read your post after competition,I had the similar feeling.we tried to solve this problem in different way,but the methods are similar.", "votes": null }, { "id": "260013", "postDate": "12/19/2017 12:24:58", "content": "Thanks a lot! \nWhat's about hard examples and focal loss? Did it improve score or coverage speed? And what's was your final sampling strategy?
", "rawMarkdown": "Thanks a lot! \nWhat's about hard examples and focal loss? Did it improve score or coverage speed? And what's was your final sampling strategy?", "votes": null }, { "id": "260438", "postDate": "12/20/2017 07:39:34", "content": "I found that hard examples and focal loss could not improve score and coverage,I guest this is because of the eval metric is accuracy and the distribution of the dataset,we can not emphasize the hard examples .I just fed all the train images into the network and I found the distribution of test set prediction is almost the same with the trainset.
", "rawMarkdown": "I found that hard examples and focal loss could not improve score and coverage,I guest this is because of the eval metric is accuracy and the distribution of the dataset,we can not emphasize the hard examples .I just fed all the train images into the network and I found the distribution of test set prediction is almost the same with the trainset.", "votes": null }, { "id": "260991", "postDate": "12/21/2017 12:35:40", "content": "Thanks for your sharing. BTW, I'm very curious about how you did to store all probabilities in one ~40M file? I keep the top10 probabilities and also use sparse matrix but the total size of .h5 file is ~400M (just test images).
\n\nLook forward to your answer. Thank you.
", "rawMarkdown": "Thanks for your sharing. BTW, I'm very curious about how you did to store all probabilities in one ~40M file? I keep the top10 probabilities and also use sparse matrix but the total size of .h5 file is ~400M (just test images).\n\nLook forward to your answer. Thank you.", "votes": null }, { "id": "261269", "postDate": "12/22/2017 06:03:19", "content": "Hi bestfitting, congrats on your win and becoming Kaggle #1. You said that you used \"optim.Adam(net.parameters(), lr=lr)\" right ?, Does it mean that you finetuned all the weights at once ?, Usually the way I finetune is layer-by-layer or block-by-block. What was your intuition behind finetuning all parameters at once ?
", "rawMarkdown": "Hi bestfitting, congrats on your win and becoming Kaggle #1. You said that you used \"optim.Adam(net.parameters(), lr=lr)\" right ?, Does it mean that you finetuned all the weights at once ?, Usually the way I finetune is layer-by-layer or block-by-block. What was your intuition behind finetuning all parameters at once ?", "votes": null }, { "id": "261377", "postDate": "12/22/2017 13:40:52", "content": "Kapok,sorry for late reply,I guess you stored your probabilities as float64,I suggest you *255 and store them as uint8.And I use https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html#scipy.sparse.csr_matrix and save to .npz,you can have a try.
", "rawMarkdown": "Kapok,sorry for late reply,I guess you stored your probabilities as float64,I suggest you *255 and store them as uint8.And I use https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html#scipy.sparse.csr_matrix and save to .npz,you can have a try.", "votes": null }, { "id": "261379", "postDate": "12/22/2017 13:53:53", "content": "Thank you very much. You're very nice.
", "rawMarkdown": "Thank you very much. You're very nice.", "votes": null }, { "id": "261386", "postDate": "12/22/2017 14:22:54", "content": "@Ravi Teja Gutta,thanks!
\nYes,I finetuned all layers at once,if the dataset is large enough,we don't need finetune layer by layer,I this competition,we have so much images,we can even train our nets from scratch.
When we start learning a framework,we may find examples finetuneing a pretrained network by using a small dataset,that's reasonable.But when we have a lot of images,I think it's not necessary do so,in Planet Amazon competition,I finetuned all layers at once too.
\nYou may ask how many is enough?I think it's determined by the image number and images per label,if we have more than 20000 images and plenty of images to classify a label,we may finetune directly.
\nAll the results above are based on my experiments,I tried a lot of methods on different datasets such as different LR on differenet layers,fix BatchNorm after some epochs on Carvarna Competition,but I found that with a good LR scheduler,we can get good results without being too complex.
\nBut I can not say for sure these will still be valid on other datasets,all these can only be a reference.
I just trained a simple inception V3 which got me to 0.70LB. I'm new to challenges like these I'll definitely try to implement your approaches to an upcoming challenge. Thanks for the info!
", "rawMarkdown": "I just trained a simple inception V3 which got me to 0.70LB. I'm new to challenges like these I'll definitely try to implement your approaches to an upcoming challenge. Thanks for the info!", "votes": null }, { "id": "262060", "postDate": "12/25/2017 05:19:39", "content": "I guess I can't just download pre-trained model and finetune, as you said you need to understand why the architectures were designed that way in the first place. I still have a lot to learn\nThanks for sharing
", "rawMarkdown": "I guess I can't just download pre-trained model and finetune, as you said you need to understand why the architectures were designed that way in the first place. I still have a lot to learn\nThanks for sharing", "votes": null }, { "id": "264427", "postDate": "01/03/2018 02:31:51", "content": "Big Con. Thanks for sharing
", "rawMarkdown": "Big Con. Thanks for sharing", "votes": null }, { "id": "266646", "postDate": "01/09/2018 10:00:41", "content": "Some late question about text features.\nHow you agregate text features from image? Features per simbol or per words? Do you use spatial projection text features on feature grid or use some odering in 1d?
", "rawMarkdown": "Some late question about text features.\nHow you agregate text features from image? Features per simbol or per words? Do you use spatial projection text features on feature grid or use some odering in 1d?", "votes": null }, { "id": "267085", "postDate": "01/10/2018 15:00:40", "content": "As said in the solution:I extracted the features of boxes from CRNN and feed them into my multi-input CNN,the first input is Resnet50 FC features and the second is CRNN features.
\nIf you read the CRNN pager,you will find that you can extract features after LSTM,the dimension is 512x26 ,if a image have 5 boxes containing text,I averaged the features extracted to get a 512x26 array ,and convert to 1d vectors and then feed them into the network before last 2 fc layer and merged with the CNN features.
\nI hope I explained clearly,if you have question,please let me know.
Thank you.\nI worry that features from with several boxes is too blended after averaging.
", "rawMarkdown": "Thank you.\nI worry that features from with several boxes is too blended after averaging.", "votes": null }, { "id": "267539", "postDate": "01/11/2018 17:13:47", "content": "I did so for too reason:1.Time was limited2.Some NLP papers use average of the wordvecs of a sentence to classify sentences.In this kind of competition we must evaluate the ROI of every method,so I chose simple and this efficient way.
", "rawMarkdown": "I did so for too reason:1.Time was limited2.Some NLP papers use average of the wordvecs of a sentence to classify sentences.In this kind of competition we must evaluate the ROI of every method,so I chose simple and this efficient way.", "votes": null }, { "id": "269885", "postDate": "01/17/2018 13:23:26", "content": "Thanks for sharing bestfitting and congrats! When you concatenate product images, did you permute the order of the images? Does it matter?
", "rawMarkdown": "Thanks for sharing bestfitting and congrats! When you concatenate product images, did you permute the order of the images? Does it matter?", "votes": null }, { "id": "269932", "postDate": "01/17/2018 14:44:01", "content": "I permuted the order of the images,I guess it was a good option to add some more randomness.
\nWhen I trained my models for product with 3 images,I randomly selected 3 images from products with 4 images.
Thanks for the info!
", "rawMarkdown": "Thanks for the info!", "votes": null }, { "id": "270585", "postDate": "01/18/2018 14:58:23", "content": "Great job!! Thanks for sharing!!
", "rawMarkdown": "Great job!! Thanks for sharing!!", "votes": null }, { "id": "275465", "postDate": "01/29/2018 07:48:55", "content": "We all know that the CNN architectures can handle a variable amount of images per class/category, so what is the motivation behind this? To mitigate a class imbalance issue?
", "rawMarkdown": "We all know that the CNN architectures can handle a variable amount of images per class/category, so what is the motivation behind this? To mitigate a class imbalance issue?", "votes": null }, { "id": "279687", "postDate": "02/08/2018 14:34:32", "content": "@bestfitting,\nThank you for sharing your approach. I have now started learning pytorch. Since I am still a newbie and need to learn some basics. Can you please share just those lines of code whereby you did:\n\"add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270\"
", "rawMarkdown": "bestfitting,\nThank you for sharing your approach. I have now started learning pytorch. Since I am still a newbie and need to learn some basics. Can you please share just those lines of code whereby you did:\n\"add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270\"", "votes": null }, { "id": "279761", "postDate": "02/08/2018 16:58:17", "content": "Please refer to this file:\nhttps://github.com/bestfitting/kaggle/blob/master/cdiscount/resnet.py
", "rawMarkdown": "Please refer to this file:\nhttps://github.com/bestfitting/kaggle/blob/master/cdiscount/resnet.py", "votes": null }, { "id": "279777", "postDate": "02/08/2018 17:28:34", "content": "Thank you sir
", "rawMarkdown": "Thank you sir", "votes": null }, { "id": "283346", "postDate": "02/15/2018 10:59:24", "content": "Thanks a lot for sharing your method!
\n\nDid you give a try to Xception? If not, why did you try ResNet50? After all, Xception seems better than the later one in term of accuracy and amount of parameters.
\n\nDid you try to have a unique model taking a mosaic of 4 images as input, replacing missing images by some kind of empty color? Or some sequential model to handle multiple-cases?
", "rawMarkdown": "Thanks a lot for sharing your method!\n\nDid you give a try to Xception? If not, why did you try ResNet50? After all, Xception seems better than the later one in term of accuracy and amount of parameters.\n\nDid you try to have a unique model taking a mosaic of 4 images as input, replacing missing images by some kind of empty color? Or some sequential model to handle multiple-cases?", "votes": null }, { "id": "283368", "postDate": "02/15/2018 11:42:43", "content": "I did not try Xception and Inception V3,I guessed Inception V4 ,Inception-Resnet-V2 would be better.
\nAccuracy on imagenet sometimes can not keep on other dataset,I also tried DPN92,SE-Resnet,SE-Inception,they all had been reported have better accuracy on imagenet,but they can only get the accuracy as resnet34 and take much much more time to train.
\nI prefer to use resnet in my everyday job,simple and beautiful
Taking a mosaic of 4 images as input is also an option,but we need more memory and time to train and predict,so I trained a model with all images and finetuned it using products with different numbers of images.
Yes,I think sequence model would not get a improvement since there are no sequence patterns in these images.
", "rawMarkdown": "I did not try Xception and Inception V3,I guessed Inception V4 ,Inception-Resnet-V2 would be better.Thanks a lot
", "rawMarkdown": "Thanks a lot", "votes": null }, { "id": "313462", "postDate": "04/13/2018 09:47:27", "content": "Thanks for sharing. This might be late but I am wondering how do you ensemble your results? Do you take the majority vote? How to average the possibilities (softmax) output by different models, if they are needed for submission? Thanks.
", "rawMarkdown": "Thanks for sharing. This might be late but I am wondering how do you ensemble your results? Do you take the majority vote? How to average the possibilities (softmax) output by different models, if they are needed for submission? Thanks.", "votes": null }, { "id": "313474", "postDate": "04/13/2018 10:02:03", "content": "In this competition,the data size is huge,and the probs of eash images are huge too,more than 5000.\nSo the easiest and feasible way to ensemble is just weighted average of them by get feed back from local validation set.
If you want to try more complex method,please have a look at:https://www.kaggle.com/c/cdiscount-image-classification-challenge/discussion/45733
Thanks for your quick reply. I am still confuse with your weighted average method. Do your mean you manually set the weights (for averaging probabilities) by heuristics or clues from the result of validation set for that training?
", "rawMarkdown": "Thanks for your quick reply. I am still confuse with your weighted average method. Do your mean you manually set the weights (for averaging probabilities) by heuristics or clues from the result of validation set for that training?", "votes": null }, { "id": "313638", "postDate": "04/13/2018 16:02:40", "content": "Since our models have same validation set,we can adjust the weights and see the validation score,I found it's very stable in this competition with so large dateset.
", "rawMarkdown": "Since our models have same validation set,we can adjust the weights and see the validation score,I found it's very stable in this competition with so large dateset.", "votes": null }, { "id": "313648", "postDate": "04/13/2018 16:17:47", "content": "I see. Thanks a lot!
", "rawMarkdown": "I see. Thanks a lot!", "votes": null }, { "id": "2809257", "postDate": "05/12/2024 16:19:01", "content": "very Nice content
", "rawMarkdown": "very Nice content", "votes": null } ], "comments": [ { "id": 2809257, "author_name": "akshat110203", "author_url": "", "post_date": "05/12/2024 16:19:01", "content": "very Nice content
", "votes": null, "replies": [] }, { "id": 258697, "author_name": "serigne", "author_url": "", "post_date": "12/16/2017 20:18:14", "content": "Thanks bestfitting for sharing about your solution and giving some insights.
\n\nBeing such efficient and reaching the first place on kaggle global ranking in just 1 year are clearly outstanding achievements... Congrats !!!
", "votes": null, "replies": [ { "id": 259367, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:27:33", "content": "Thank you! I think the most important factors are the kaggle rank system and I am so lucky in several competitions including this one.
", "votes": null, "replies": [] } ] }, { "id": 258840, "author_name": "outrunner", "author_url": "", "post_date": "12/17/2017 05:36:11", "content": "Congrats, thanks for sharing.\nI also fine tune by image count and order but not concatenate them, ensemble 6 model trained from subset with 1 full set, public LB can reach 0.772.
", "votes": null, "replies": [ { "id": 259363, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:20:49", "content": "I noticed that the order of images should be varied,so when I trained my model,I randomized the order of images of a product.
", "votes": null, "replies": [] } ] }, { "id": 258856, "author_name": "youngkl", "author_url": "", "post_date": "12/17/2017 06:55:56", "content": "Congrats! Thanks for your sharing. \nI also tried Resnet50 and Inception_resnet_v2, but my best single model could just get the public LB 0.69. Can you share your code for us to learn?
", "votes": null, "replies": [ { "id": 259362, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:18:31", "content": "You can refer to this.please use Adam and scheduler in this post:
\nresnet for cdicount
Thanks for sharing. How do you concatenate 4 images into a image? concatenate with image channels?
", "votes": null, "replies": [ { "id": 259358, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:10:27", "content": "Please refer to these two images,you can find that if we concatenate the images into a image,our models will get the information of a product in a whole.\n
\n\n
if self.img_cnt==4 and self.mmv==1:\n img_out = np.zeros([self.img_size*2, self.img_size * 2, 3])\n else:\n img_out = np.zeros([self.img_size, self.img_size * self.img_cnt, 3])\n.....
\n\nRead images of a product then:\n h, w = img.shape[0:2]\n if self.img_size != h or self.img_size != w:\n img = cv2.resize(img, (self.img_size, self.img_size))\n img_size=self.img_size\n if self.img_cnt == 4 and self.mmv == 1:\n if array_idx<2:\n img_out[ :img_size, array_idx * img_size:(array_idx + 1) * img_size,:] = img\n else:\n img_out[img_size:, (array_idx-2) * img_size:((array_idx-2) + 1) * img_size, :] = img\n else:\n img_out[ :, array_idx * img_size:(array_idx + 1) * img_size,:] = img\nWe all know that the CNN architectures can handle a variable amount of images per class/category, so what is the motivation behind this? To mitigate a class imbalance issue?
", "votes": null, "replies": [] } ] }, { "id": 258943, "author_name": "dawnbreaker", "author_url": "", "post_date": "12/17/2017 12:17:59", "content": "Congrats! Thanks for sharing. BTW, did you do any kind of augmentation at testing stage?
", "votes": null, "replies": [ { "id": 259355, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:07:04", "content": "I cropped 5 160x160 patches from 180x180 images ,left-top,right-top,middle,left-bottom,right-bottom,the score could improve 0.5 also.
", "votes": null, "replies": [] } ] }, { "id": 259009, "author_name": "hengck23", "author_url": "", "post_date": "12/17/2017 14:24:03", "content": "Thanks for the answer. From your discussion and my experiments, i conclude the followings (which may not be correct). Key factors for top performance are:
\n\n[1. modification of imagenet model for larger class]
\n\nImagenet pretrained models sometimes do not get good results because they are designed for 1000 classes. To improve results we need to extend the last fc layers. Bestfitting did it by stacking 2 more fc layers to the imagenet model. For my case, i did it in additional post network on extracted imagenet features (coincidentally also using 2 additional fc layers).
\n\n[2. handling of multiple images]
\n\nWe need some way to handle multiple images. Bestfitting did it by training different models for different number of images. The models are ensembled together. For me, the post network is trained to find the best combinations.
\n\n[3. OCR]
\n\nBestfitting uses OCR to improve results for CD/BOOKS. We do not use OCR.
", "votes": null, "replies": [ { "id": 259371, "author_name": "bestfitting", "author_url": "", "post_date": "12/18/2017 08:40:03", "content": "Yes,when I read your post after competition,I had the similar feeling.we tried to solve this problem in different way,but the methods are similar.
", "votes": null, "replies": [] } ] }, { "id": 260013, "author_name": "drn01z3", "author_url": "", "post_date": "12/19/2017 12:24:58", "content": "Thanks a lot! \nWhat's about hard examples and focal loss? Did it improve score or coverage speed? And what's was your final sampling strategy?
", "votes": null, "replies": [ { "id": 260438, "author_name": "bestfitting", "author_url": "", "post_date": "12/20/2017 07:39:34", "content": "I found that hard examples and focal loss could not improve score and coverage,I guest this is because of the eval metric is accuracy and the distribution of the dataset,we can not emphasize the hard examples .I just fed all the train images into the network and I found the distribution of test set prediction is almost the same with the trainset.
", "votes": null, "replies": [] } ] }, { "id": 260991, "author_name": "plainsailing", "author_url": "", "post_date": "12/21/2017 12:35:40", "content": "Thanks for your sharing. BTW, I'm very curious about how you did to store all probabilities in one ~40M file? I keep the top10 probabilities and also use sparse matrix but the total size of .h5 file is ~400M (just test images).
\n\nLook forward to your answer. Thank you.
", "votes": null, "replies": [ { "id": 261377, "author_name": "bestfitting", "author_url": "", "post_date": "12/22/2017 13:40:52", "content": "Kapok,sorry for late reply,I guess you stored your probabilities as float64,I suggest you *255 and store them as uint8.And I use https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.csr_matrix.html#scipy.sparse.csr_matrix and save to .npz,you can have a try.
", "votes": null, "replies": [] }, { "id": 261379, "author_name": "plainsailing", "author_url": "", "post_date": "12/22/2017 13:53:53", "content": "Thank you very much. You're very nice.
", "votes": null, "replies": [] } ] }, { "id": 261269, "author_name": "rteja1113", "author_url": "", "post_date": "12/22/2017 06:03:19", "content": "Hi bestfitting, congrats on your win and becoming Kaggle #1. You said that you used \"optim.Adam(net.parameters(), lr=lr)\" right ?, Does it mean that you finetuned all the weights at once ?, Usually the way I finetune is layer-by-layer or block-by-block. What was your intuition behind finetuning all parameters at once ?
", "votes": null, "replies": [ { "id": 261386, "author_name": "bestfitting", "author_url": "", "post_date": "12/22/2017 14:22:54", "content": "@Ravi Teja Gutta,thanks!
\nYes,I finetuned all layers at once,if the dataset is large enough,we don't need finetune layer by layer,I this competition,we have so much images,we can even train our nets from scratch.
When we start learning a framework,we may find examples finetuneing a pretrained network by using a small dataset,that's reasonable.But when we have a lot of images,I think it's not necessary do so,in Planet Amazon competition,I finetuned all layers at once too.
\nYou may ask how many is enough?I think it's determined by the image number and images per label,if we have more than 20000 images and plenty of images to classify a label,we may finetune directly.
\nAll the results above are based on my experiments,I tried a lot of methods on different datasets such as different LR on differenet layers,fix BatchNorm after some epochs on Carvarna Competition,but I found that with a good LR scheduler,we can get good results without being too complex.
\nBut I can not say for sure these will still be valid on other datasets,all these can only be a reference.
I guess I can't just download pre-trained model and finetune, as you said you need to understand why the architectures were designed that way in the first place. I still have a lot to learn\nThanks for sharing
", "votes": null, "replies": [] } ] }, { "id": 261969, "author_name": "tejasshahpuri", "author_url": "", "post_date": "12/24/2017 17:40:26", "content": "I just trained a simple inception V3 which got me to 0.70LB. I'm new to challenges like these I'll definitely try to implement your approaches to an upcoming challenge. Thanks for the info!
", "votes": null, "replies": [] }, { "id": 264427, "author_name": "xfz666", "author_url": "", "post_date": "01/03/2018 02:31:51", "content": "Big Con. Thanks for sharing
", "votes": null, "replies": [] }, { "id": 266646, "author_name": "nicksergievskiy", "author_url": "", "post_date": "01/09/2018 10:00:41", "content": "Some late question about text features.\nHow you agregate text features from image? Features per simbol or per words? Do you use spatial projection text features on feature grid or use some odering in 1d?
", "votes": null, "replies": [ { "id": 267085, "author_name": "bestfitting", "author_url": "", "post_date": "01/10/2018 15:00:40", "content": "As said in the solution:I extracted the features of boxes from CRNN and feed them into my multi-input CNN,the first input is Resnet50 FC features and the second is CRNN features.
\nIf you read the CRNN pager,you will find that you can extract features after LSTM,the dimension is 512x26 ,if a image have 5 boxes containing text,I averaged the features extracted to get a 512x26 array ,and convert to 1d vectors and then feed them into the network before last 2 fc layer and merged with the CNN features.
\nI hope I explained clearly,if you have question,please let me know.
Thank you.\nI worry that features from with several boxes is too blended after averaging.
", "votes": null, "replies": [] }, { "id": 267539, "author_name": "bestfitting", "author_url": "", "post_date": "01/11/2018 17:13:47", "content": "I did so for too reason:1.Time was limited2.Some NLP papers use average of the wordvecs of a sentence to classify sentences.In this kind of competition we must evaluate the ROI of every method,so I chose simple and this efficient way.
", "votes": null, "replies": [] } ] }, { "id": 269885, "author_name": "princyralaivao", "author_url": "", "post_date": "01/17/2018 13:23:26", "content": "Thanks for sharing bestfitting and congrats! When you concatenate product images, did you permute the order of the images? Does it matter?
", "votes": null, "replies": [ { "id": 269932, "author_name": "bestfitting", "author_url": "", "post_date": "01/17/2018 14:44:01", "content": "I permuted the order of the images,I guess it was a good option to add some more randomness.
\nWhen I trained my models for product with 3 images,I randomly selected 3 images from products with 4 images.
Thanks for the info!
", "votes": null, "replies": [] } ] }, { "id": 270585, "author_name": "andrearapuzzi", "author_url": "", "post_date": "01/18/2018 14:58:23", "content": "Great job!! Thanks for sharing!!
", "votes": null, "replies": [] }, { "id": 279687, "author_name": "shirishr", "author_url": "", "post_date": "02/08/2018 14:34:32", "content": "@bestfitting,\nThank you for sharing your approach. I have now started learning pytorch. Since I am still a newbie and need to learn some basics. Can you please share just those lines of code whereby you did:\n\"add a 1x1 kernel convolution layer after layer4 of resnet,which increase the channel from 512 to 5270,and then a FC of 5270*5270\"
", "votes": null, "replies": [ { "id": 279761, "author_name": "bestfitting", "author_url": "", "post_date": "02/08/2018 16:58:17", "content": "Please refer to this file:\nhttps://github.com/bestfitting/kaggle/blob/master/cdiscount/resnet.py
", "votes": null, "replies": [] }, { "id": 279777, "author_name": "shirishr", "author_url": "", "post_date": "02/08/2018 17:28:34", "content": "Thank you sir
", "votes": null, "replies": [] } ] }, { "id": 283346, "author_name": "matthieudelaro", "author_url": "", "post_date": "02/15/2018 10:59:24", "content": "Thanks a lot for sharing your method!
\n\nDid you give a try to Xception? If not, why did you try ResNet50? After all, Xception seems better than the later one in term of accuracy and amount of parameters.
\n\nDid you try to have a unique model taking a mosaic of 4 images as input, replacing missing images by some kind of empty color? Or some sequential model to handle multiple-cases?
", "votes": null, "replies": [ { "id": 283368, "author_name": "bestfitting", "author_url": "", "post_date": "02/15/2018 11:42:43", "content": "I did not try Xception and Inception V3,I guessed Inception V4 ,Inception-Resnet-V2 would be better.
\nAccuracy on imagenet sometimes can not keep on other dataset,I also tried DPN92,SE-Resnet,SE-Inception,they all had been reported have better accuracy on imagenet,but they can only get the accuracy as resnet34 and take much much more time to train.
\nI prefer to use resnet in my everyday job,simple and beautiful
Taking a mosaic of 4 images as input is also an option,but we need more memory and time to train and predict,so I trained a model with all images and finetuned it using products with different numbers of images.
Yes,I think sequence model would not get a improvement since there are no sequence patterns in these images.
", "votes": null, "replies": [] }, { "id": 285185, "author_name": "matthieudelaro", "author_url": "", "post_date": "02/19/2018 14:05:25", "content": "Thanks a lot
", "votes": null, "replies": [] } ] }, { "id": 313462, "author_name": "tianwei", "author_url": "", "post_date": "04/13/2018 09:47:27", "content": "Thanks for sharing. This might be late but I am wondering how do you ensemble your results? Do you take the majority vote? How to average the possibilities (softmax) output by different models, if they are needed for submission? Thanks.
", "votes": null, "replies": [ { "id": 313474, "author_name": "bestfitting", "author_url": "", "post_date": "04/13/2018 10:02:03", "content": "In this competition,the data size is huge,and the probs of eash images are huge too,more than 5000.\nSo the easiest and feasible way to ensemble is just weighted average of them by get feed back from local validation set.
If you want to try more complex method,please have a look at:https://www.kaggle.com/c/cdiscount-image-classification-challenge/discussion/45733
Thanks for your quick reply. I am still confuse with your weighted average method. Do your mean you manually set the weights (for averaging probabilities) by heuristics or clues from the result of validation set for that training?
", "votes": null, "replies": [] }, { "id": 313638, "author_name": "bestfitting", "author_url": "", "post_date": "04/13/2018 16:02:40", "content": "Since our models have same validation set,we can adjust the weights and see the validation score,I found it's very stable in this competition with so large dateset.
", "votes": null, "replies": [] }, { "id": 313648, "author_name": "tianwei", "author_url": "", "post_date": "04/13/2018 16:17:47", "content": "I see. Thanks a lot!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "258683": "It is an interesting competition,and also a tough one.lr = 0.0003\n if epoch > 7:\n lr = 0.0001\n if epoch > 9:\n lr = 0.00005\n if epoch > 11:\n lr = 0.00001\n\nWhen I trained 11.5 epochs with 160x160 patch random cropped from 180*180 images,I predicted and averaged predictions of the images of product,I could get a score >0.72 on public LB.