![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F763007%2Fd95a2ac77c4d16c7541743a676c0c468%2Ftenor.gif?generation=1580860203011589&alt=media\")
I had another post asking people what kind of input size they use and a few people were surprised at the score I got with 64x64x1 inputs. So I decided to shared the strategy a little bit because using 64x64 is nice for people with low computing power (I only have 2x2080). \nThe strategy is as follows:\nSingle model of wide resnet with three dense layers on top\nDropout before each pooling layer\nCutout\nScores:\n0.97 with 0.15 validation split\n0.9718 without validation
", "messages": [ { "id": "733207", "postDate": "01/30/2020 20:27:18", "content": "I had another post asking people what kind of input size they use and a few people were surprised at the score I got with 64x64x1 inputs. So I decided to shared the strategy a little bit because using 64x64 is nice for people with low computing power (I only have 2x2080). \nThe strategy is as follows:\nSingle model of wide resnet with three dense layers on top\nDropout before each pooling layer\nCutout\nScores:\n0.97 with 0.15 validation split\n0.9718 without validation
", "rawMarkdown": "I had another post asking people what kind of input size they use and a few people were surprised at the score I got with 64x64x1 inputs. So I decided to shared the strategy a little bit because using 64x64 is nice for people with low computing power (I only have 2x2080). \nThe strategy is as follows:\nSingle model of wide resnet with three dense layers on top\nDropout before each pooling layer\nCutout\nScores:\n0.97 with 0.15 validation split\n0.9718 without validation", "votes": null }, { "id": "733345", "postDate": "01/31/2020 02:13:46", "content": "Congratulations, man. This is very strong result! If you don't mind disclosing, did you use cutmix / mixup in your training? If so, how many epochs did you train for?
", "rawMarkdown": "Congratulations, man. This is very strong result! If you don't mind disclosing, did you use cutmix / mixup in your training? If so, how many epochs did you train for?", "votes": null }, { "id": "733370", "postDate": "01/31/2020 03:09:07", "content": "Thanks! I used only cutout and trained for 100 epochs.
", "rawMarkdown": "Thanks! I used only cutout and trained for 100 epochs.", "votes": null }, { "id": "733489", "postDate": "01/31/2020 07:46:17", "content": "« I only have 2x2080 ». Dude, you have more vRAM on 1 single card than my laptop does; and I’m on a 2Gb mx150 dedicated GPU. That’s a hell of a lot of compute power compared to most people :P
\n\nAnyways, thanks for sharing your pipeline!
", "rawMarkdown": "« I only have 2x2080 ». Dude, you have more vRAM on 1 single card than my laptop does; and I’m on a 2Gb mx150 dedicated GPU. That’s a hell of a lot of compute power compared to most people :P\n\nAnyways, thanks for sharing your pipeline!", "votes": null }, { "id": "733493", "postDate": "01/31/2020 07:58:18", "content": "Seriously a very good result bro.. I too have very less compute capability. Lets create a thread specific for low compute guys.
\n\nCongrats and All the best @shujun717
", "rawMarkdown": "Seriously a very good result bro.. I too have very less compute capability. Lets create a thread specific for low compute guys.\n\nCongrats and All the best @shujun717", "votes": null }, { "id": "733582", "postDate": "01/31/2020 10:20:38", "content": "Dude 2x2080 is such a decent rig 😆. Great result bro!
", "rawMarkdown": "Dude 2x2080 is such a decent rig 😆. Great result bro!", "votes": null }, { "id": "733881", "postDate": "01/31/2020 16:28:02", "content": "@shujun717 Hi , even we had tried cutout using default parameters(size and max number of cutouts per image) but couldn't get promising results...maybe because we trained only for 40-45 epochs . Can you share how many cutouts u made per image and what their sizes were ? Thanks .
", "rawMarkdown": "shujun717 Hi , even we had tried cutout using default parameters(size and max number of cutouts per image) but couldn't get promising results...maybe because we trained only for 40-45 epochs . Can you share how many cutouts u made per image and what their sizes were ? Thanks .", "votes": null }, { "id": "733903", "postDate": "01/31/2020 16:54:14", "content": "40-50 epochs is not enough to converge for me. I train for 100. As for cutouts, you don't need more than 1 patch. Just use 1 patch and tune the size of it
", "rawMarkdown": "40-50 epochs is not enough to converge for me. I train for 100. As for cutouts, you don't need more than 1 patch. Just use 1 patch and tune the size of it", "votes": null }, { "id": "734148", "postDate": "02/01/2020 01:58:00", "content": "@shujun717 3 dense layers on top means 1 for each class or any other model?
", "rawMarkdown": "shujun717 3 dense layers on top means 1 for each class or any other model?", "votes": null }, { "id": "734179", "postDate": "02/01/2020 04:00:23", "content": "One for each class
", "rawMarkdown": "One for each class", "votes": null }, { "id": "734190", "postDate": "02/01/2020 04:19:06", "content": "Few months back when I saw this article https://www.kdnuggets.com/2018/09/dropout-convolutional-networks.html and a paper, I don't remember its title, I was convinced that dropout is not effective in or on CNN layers, but your results says otherwise. After that I searched many options for dropout and I found a paper https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf that suggests droupout on pooling can be very effective.\nyou gave me more ideas to experiment with... thanks.
", "rawMarkdown": "Few months back when I saw this article https://www.kdnuggets.com/2018/09/dropout-convolutional-networks.html and a paper, I don't remember its title, I was convinced that dropout is not effective in or on CNN layers, but your results says otherwise. After that I searched many options for dropout and I found a paper https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf that suggests droupout on pooling can be very effective.\nyou gave me more ideas to experiment with... thanks.", "votes": null }, { "id": "734840", "postDate": "02/02/2020 04:30:20", "content": "@shujun717 Thanks for your sharing, May I ask how did you get the 64 * 64 image, resize 236x137 to 64x64, is that right?
", "rawMarkdown": "shujun717 Thanks for your sharing, May I ask how did you get the 64 * 64 image, resize 236x137 to 64x64, is that right?", "votes": null }, { "id": "734850", "postDate": "02/02/2020 05:06:44", "content": "Yeah just resize with cv2
", "rawMarkdown": "Yeah just resize with cv2", "votes": null }, { "id": "734891", "postDate": "02/02/2020 06:31:50", "content": "Thanks
", "rawMarkdown": "Thanks", "votes": null }, { "id": "735201", "postDate": "02/02/2020 18:14:04", "content": "Congratulations ! Did you train for 100 epochs for each parquet ?
", "rawMarkdown": "Congratulations ! Did you train for 100 epochs for each parquet ?", "votes": null }, { "id": "735414", "postDate": "02/03/2020 02:07:47", "content": "Hi, Did you crop images before training? \nLooking forward to your reply.
", "rawMarkdown": "Hi, Did you crop images before training? \nLooking forward to your reply.", "votes": null }, { "id": "735508", "postDate": "02/03/2020 06:06:09", "content": "Nice result Shujun. 'Only 2•2080'.
", "rawMarkdown": "Nice result Shujun. 'Only 2•2080'.", "votes": null }, { "id": "736048", "postDate": "02/03/2020 18:45:18", "content": "Hey, im just curious what optimizer did you use and your strategy with lr? Personally ive had better results with SGD and OneCycleLr! \nI also have low computing power, (one gtx1080) so all my results/tests are on a resnet34, i might try wideresnet! thanks for your post :)
", "rawMarkdown": "Hey, im just curious what optimizer did you use and your strategy with lr? Personally ive had better results with SGD and OneCycleLr! \nI also have low computing power, (one gtx1080) so all my results/tests are on a resnet34, i might try wideresnet! thanks for your post :)", "votes": null }, { "id": "736237", "postDate": "02/04/2020 01:23:57", "content": "I can't really say you have low computing power if you own a gtx1080 😂
", "rawMarkdown": "I can't really say you have low computing power if you own a gtx1080 😂", "votes": null }, { "id": "736322", "postDate": "02/04/2020 04:14:08", "content": "Thanks for your info @shujun717 . I have tried image size 64x46 and optimize the network arch. The cv score can reach to 0.972 but lb score is lower than that. May I ask what's your split method and your local cv score?
", "rawMarkdown": "Thanks for your info @shujun717 . I have tried image size 64x46 and optimize the network arch. The cv score can reach to 0.972 but lb score is lower than that. May I ask what's your split method and your local cv score?", "votes": null }, { "id": "736991", "postDate": "02/04/2020 19:26:15", "content": "I used Adam with 1-e4 initial learning rate and decay by sqrt(0.1) at 75 and 90 epochs
", "rawMarkdown": "I used Adam with 1-e4 initial learning rate and decay by sqrt(0.1) at 75 and 90 epochs", "votes": null }, { "id": "736993", "postDate": "02/04/2020 19:28:19", "content": "Split is just random with fixed seed. I assume CV means cross validation? In that case, I only did single fold and local score is around 0.982
", "rawMarkdown": "Split is just random with fixed seed. I assume CV means cross validation? In that case, I only did single fold and local score is around 0.982", "votes": null }, { "id": "737051", "postDate": "02/04/2020 21:20:12", "content": "amazing result
", "rawMarkdown": "amazing result", "votes": null }, { "id": "737119", "postDate": "02/04/2020 23:50:15", "content": "\n\n\n... low computing power (I only have 2x2080).
\n
Thanks! Really impressive result.
", "rawMarkdown": "Thanks! Really impressive result.", "votes": null }, { "id": "738099", "postDate": "02/06/2020 06:07:44", "content": "So impressive! Congrats!
", "rawMarkdown": "So impressive! Congrats!", "votes": null }, { "id": "739100", "postDate": "02/07/2020 12:00:41", "content": "There are many varieties of wide resnet with different depth and widening factor. Can you please tell me, which one did you use?
", "rawMarkdown": "There are many varieties of wide resnet with different depth and widening factor. Can you please tell me, which one did you use?", "votes": null }, { "id": "742500", "postDate": "02/11/2020 10:25:30", "content": "that is higher computing power than most people include me(I only have 1x2070).
", "rawMarkdown": "that is higher computing power than most people include me(I only have 1x2070).", "votes": null }, { "id": "742549", "postDate": "02/11/2020 11:19:51", "content": "I found three dense layers are not as effective as just global pooling followed by output layers. At least that's what I concluded from my experiments.
", "rawMarkdown": "I found three dense layers are not as effective as just global pooling followed by output layers. At least that's what I concluded from my experiments.", "votes": null }, { "id": "751443", "postDate": "02/20/2020 07:43:02", "content": "How to use ResNet with 1 channel image?
", "rawMarkdown": "How to use ResNet with 1 channel image?", "votes": null }, { "id": "751926", "postDate": "02/20/2020 16:41:15", "content": "There are many ways. When you build your model using pretrained ResNet which wants 3 channels, just put a new bottom on it. For example
\n\n x = tf.keras.Input(shape=(64,64,1))\n x = tf.keras.layers.Concatenate()([x, x, x])\n ResNet34(weights='imagenet',include_top=False)(x)\ngreat
", "rawMarkdown": "great", "votes": null }, { "id": "761304", "postDate": "03/02/2020 11:25:24", "content": "@suvalex : Either use what @cdeotte mentioned, or you can just sum up the first layer weights and replace that in a new layer that uses 1 channel (shown here for efficient-net):
\n\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2308010%2F6261e91fa64434c4e2148f089973b4b3%2FCapture.PNG?generation=1583148300942594&alt=media\")
Hi @shujun717 , as mentioned in this paper which talks about using dropout before pooling layer: https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf , did u use probabilistic weighted pooling during test time? If yes, can u pls share the code?
", "rawMarkdown": "Hi @shujun717 , as mentioned in this paper which talks about using dropout before pooling layer: https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf , did u use probabilistic weighted pooling during test time? If yes, can u pls share the code?", "votes": null }, { "id": "770689", "postDate": "03/13/2020 09:08:54", "content": "@shujun717 I hope you will make your kernel public after the competition. Even after applying your approach I'm nowhere near your score. Thanks in advance.
", "rawMarkdown": "shujun717 I hope you will make your kernel public after the competition. Even after applying your approach I'm nowhere near your score. Thanks in advance.", "votes": null } ], "comments": [ { "id": 733345, "author_name": "roguekk007", "author_url": "", "post_date": "01/31/2020 02:13:46", "content": "Congratulations, man. This is very strong result! If you don't mind disclosing, did you use cutmix / mixup in your training? If so, how many epochs did you train for?
", "votes": null, "replies": [ { "id": 733370, "author_name": "shujun717", "author_url": "", "post_date": "01/31/2020 03:09:07", "content": "Thanks! I used only cutout and trained for 100 epochs.
", "votes": null, "replies": [] } ] }, { "id": 733489, "author_name": "maxlenormand", "author_url": "", "post_date": "01/31/2020 07:46:17", "content": "« I only have 2x2080 ». Dude, you have more vRAM on 1 single card than my laptop does; and I’m on a 2Gb mx150 dedicated GPU. That’s a hell of a lot of compute power compared to most people :P
\n\nAnyways, thanks for sharing your pipeline!
", "votes": null, "replies": [] }, { "id": 733493, "author_name": "dhakshiin1601", "author_url": "", "post_date": "01/31/2020 07:58:18", "content": "Seriously a very good result bro.. I too have very less compute capability. Lets create a thread specific for low compute guys.
\n\nCongrats and All the best @shujun717
", "votes": null, "replies": [] }, { "id": 733582, "author_name": "quandapro", "author_url": "", "post_date": "01/31/2020 10:20:38", "content": "Dude 2x2080 is such a decent rig 😆. Great result bro!
", "votes": null, "replies": [] }, { "id": 733881, "author_name": "virajbagal", "author_url": "", "post_date": "01/31/2020 16:28:02", "content": "@shujun717 Hi , even we had tried cutout using default parameters(size and max number of cutouts per image) but couldn't get promising results...maybe because we trained only for 40-45 epochs . Can you share how many cutouts u made per image and what their sizes were ? Thanks .
", "votes": null, "replies": [ { "id": 733903, "author_name": "shujun717", "author_url": "", "post_date": "01/31/2020 16:54:14", "content": "40-50 epochs is not enough to converge for me. I train for 100. As for cutouts, you don't need more than 1 patch. Just use 1 patch and tune the size of it
", "votes": null, "replies": [] } ] }, { "id": 734148, "author_name": "raghaw", "author_url": "", "post_date": "02/01/2020 01:58:00", "content": "@shujun717 3 dense layers on top means 1 for each class or any other model?
", "votes": null, "replies": [ { "id": 734179, "author_name": "shujun717", "author_url": "", "post_date": "02/01/2020 04:00:23", "content": "One for each class
", "votes": null, "replies": [] } ] }, { "id": 734190, "author_name": "raghaw", "author_url": "", "post_date": "02/01/2020 04:19:06", "content": "Few months back when I saw this article https://www.kdnuggets.com/2018/09/dropout-convolutional-networks.html and a paper, I don't remember its title, I was convinced that dropout is not effective in or on CNN layers, but your results says otherwise. After that I searched many options for dropout and I found a paper https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf that suggests droupout on pooling can be very effective.\nyou gave me more ideas to experiment with... thanks.
", "votes": null, "replies": [] }, { "id": 734840, "author_name": "hesene", "author_url": "", "post_date": "02/02/2020 04:30:20", "content": "@shujun717 Thanks for your sharing, May I ask how did you get the 64 * 64 image, resize 236x137 to 64x64, is that right?
", "votes": null, "replies": [ { "id": 734850, "author_name": "shujun717", "author_url": "", "post_date": "02/02/2020 05:06:44", "content": "Yeah just resize with cv2
", "votes": null, "replies": [] }, { "id": 734891, "author_name": "hesene", "author_url": "", "post_date": "02/02/2020 06:31:50", "content": "Thanks
", "votes": null, "replies": [] } ] }, { "id": 735201, "author_name": "namansingh2803", "author_url": "", "post_date": "02/02/2020 18:14:04", "content": "Congratulations ! Did you train for 100 epochs for each parquet ?
", "votes": null, "replies": [] }, { "id": 735414, "author_name": "kasim0226", "author_url": "", "post_date": "02/03/2020 02:07:47", "content": "Hi, Did you crop images before training? \nLooking forward to your reply.
", "votes": null, "replies": [] }, { "id": 735508, "author_name": "ganeshravisankar", "author_url": "", "post_date": "02/03/2020 06:06:09", "content": "Nice result Shujun. 'Only 2•2080'.
", "votes": null, "replies": [] }, { "id": 736048, "author_name": "yannmajewski", "author_url": "", "post_date": "02/03/2020 18:45:18", "content": "Hey, im just curious what optimizer did you use and your strategy with lr? Personally ive had better results with SGD and OneCycleLr! \nI also have low computing power, (one gtx1080) so all my results/tests are on a resnet34, i might try wideresnet! thanks for your post :)
", "votes": null, "replies": [ { "id": 736237, "author_name": "quandapro", "author_url": "", "post_date": "02/04/2020 01:23:57", "content": "I can't really say you have low computing power if you own a gtx1080 😂
", "votes": null, "replies": [] }, { "id": 736991, "author_name": "shujun717", "author_url": "", "post_date": "02/04/2020 19:26:15", "content": "I used Adam with 1-e4 initial learning rate and decay by sqrt(0.1) at 75 and 90 epochs
", "votes": null, "replies": [] } ] }, { "id": 736322, "author_name": "syoya1997", "author_url": "", "post_date": "02/04/2020 04:14:08", "content": "Thanks for your info @shujun717 . I have tried image size 64x46 and optimize the network arch. The cv score can reach to 0.972 but lb score is lower than that. May I ask what's your split method and your local cv score?
", "votes": null, "replies": [ { "id": 736993, "author_name": "shujun717", "author_url": "", "post_date": "02/04/2020 19:28:19", "content": "Split is just random with fixed seed. I assume CV means cross validation? In that case, I only did single fold and local score is around 0.982
", "votes": null, "replies": [] }, { "id": 737221, "author_name": "syoya1997", "author_url": "", "post_date": "02/05/2020 04:14:00", "content": "Thanks! Really impressive result.
", "votes": null, "replies": [] } ] }, { "id": 737051, "author_name": "mohamedabdelshafi", "author_url": "", "post_date": "02/04/2020 21:20:12", "content": "amazing result
", "votes": null, "replies": [] }, { "id": 737119, "author_name": "rohitagarwal", "author_url": "", "post_date": "02/04/2020 23:50:15", "content": "\n\n\n... low computing power (I only have 2x2080).
\n
that is higher computing power than most people include me(I only have 1x2070).
", "votes": null, "replies": [] } ] }, { "id": 738099, "author_name": "yuanlin08", "author_url": "", "post_date": "02/06/2020 06:07:44", "content": "So impressive! Congrats!
", "votes": null, "replies": [] }, { "id": 739100, "author_name": "zaber666", "author_url": "", "post_date": "02/07/2020 12:00:41", "content": "There are many varieties of wide resnet with different depth and widening factor. Can you please tell me, which one did you use?
", "votes": null, "replies": [] }, { "id": 742549, "author_name": "quandapro", "author_url": "", "post_date": "02/11/2020 11:19:51", "content": "I found three dense layers are not as effective as just global pooling followed by output layers. At least that's what I concluded from my experiments.
", "votes": null, "replies": [] }, { "id": 751443, "author_name": "suvalex", "author_url": "", "post_date": "02/20/2020 07:43:02", "content": "How to use ResNet with 1 channel image?
", "votes": null, "replies": [ { "id": 751926, "author_name": "cdeotte", "author_url": "", "post_date": "02/20/2020 16:41:15", "content": "There are many ways. When you build your model using pretrained ResNet which wants 3 channels, just put a new bottom on it. For example
\n\n x = tf.keras.Input(shape=(64,64,1))\n x = tf.keras.layers.Concatenate()([x, x, x])\n ResNet34(weights='imagenet',include_top=False)(x)\n@suvalex : Either use what @cdeotte mentioned, or you can just sum up the first layer weights and replace that in a new layer that uses 1 channel (shown here for efficient-net):
\n\n
great
", "votes": null, "replies": [] }, { "id": 761362, "author_name": "virajbagal", "author_url": "", "post_date": "03/02/2020 12:38:11", "content": "Hi @shujun717 , as mentioned in this paper which talks about using dropout before pooling layer: https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf , did u use probabilistic weighted pooling during test time? If yes, can u pls share the code?
", "votes": null, "replies": [] }, { "id": 770689, "author_name": "gaur128", "author_url": "", "post_date": "03/13/2020 09:08:54", "content": "@shujun717 I hope you will make your kernel public after the competition. Even after applying your approach I'm nowhere near your score. Thanks in advance.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "733207": "I had another post asking people what kind of input size they use and a few people were surprised at the score I got with 64x64x1 inputs. So I decided to shared the strategy a little bit because using 64x64 is nice for people with low computing power (I only have 2x2080). \nThe strategy is as follows:\nSingle model of wide resnet with three dense layers on top\nDropout before each pooling layer\nCutout\nScores:\n0.97 with 0.15 validation split\n0.9718 without validation", "733345": "Congratulations, man. This is very strong result! If you don't mind disclosing, did you use cutmix / mixup in your training? If so, how many epochs did you train for?", "733370": "Thanks! I used only cutout and trained for 100 epochs.", "733489": "« I only have 2x2080 ». Dude, you have more vRAM on 1 single card than my laptop does; and I’m on a 2Gb mx150 dedicated GPU. That’s a hell of a lot of compute power compared to most people :P\n\nAnyways, thanks for sharing your pipeline!", "733493": "Seriously a very good result bro.. I too have very less compute capability. Lets create a thread specific for low compute guys.\n\nCongrats and All the best @shujun717", "733582": "Dude 2x2080 is such a decent rig 😆. Great result bro!", "733881": "shujun717 Hi , even we had tried cutout using default parameters(size and max number of cutouts per image) but couldn't get promising results...maybe because we trained only for 40-45 epochs . Can you share how many cutouts u made per image and what their sizes were ? Thanks .", "733903": "40-50 epochs is not enough to converge for me. I train for 100. As for cutouts, you don't need more than 1 patch. Just use 1 patch and tune the size of it", "734148": "shujun717 3 dense layers on top means 1 for each class or any other model?", "734179": "One for each class", "734190": "Few months back when I saw this article https://www.kdnuggets.com/2018/09/dropout-convolutional-networks.html and a paper, I don't remember its title, I was convinced that dropout is not effective in or on CNN layers, but your results says otherwise. After that I searched many options for dropout and I found a paper https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf that suggests droupout on pooling can be very effective.\nyou gave me more ideas to experiment with... thanks.", "734840": "shujun717 Thanks for your sharing, May I ask how did you get the 64 * 64 image, resize 236x137 to 64x64, is that right?", "734850": "Yeah just resize with cv2", "734891": "Thanks", "735201": "Congratulations ! Did you train for 100 epochs for each parquet ?", "735414": "Hi, Did you crop images before training? \nLooking forward to your reply.", "735508": "Nice result Shujun. 'Only 2•2080'.", "736048": "Hey, im just curious what optimizer did you use and your strategy with lr? Personally ive had better results with SGD and OneCycleLr! \nI also have low computing power, (one gtx1080) so all my results/tests are on a resnet34, i might try wideresnet! thanks for your post :)", "736237": "I can't really say you have low computing power if you own a gtx1080 😂", "736322": "Thanks for your info @shujun717 . I have tried image size 64x46 and optimize the network arch. The cv score can reach to 0.972 but lb score is lower than that. May I ask what's your split method and your local cv score?", "736991": "I used Adam with 1-e4 initial learning rate and decay by sqrt(0.1) at 75 and 90 epochs", "736993": "Split is just random with fixed seed. I assume CV means cross validation? In that case, I only did single fold and local score is around 0.982", "737051": "amazing result", "737119": ">... low computing power (I only have 2x2080).\n\n", "737221": "Thanks! Really impressive result.", "738099": "So impressive! Congrats!", "739100": "There are many varieties of wide resnet with different depth and widening factor. Can you please tell me, which one did you use?", "742500": "that is higher computing power than most people include me(I only have 1x2070).", "742549": "I found three dense layers are not as effective as just global pooling followed by output layers. At least that's what I concluded from my experiments.", "751443": "How to use ResNet with 1 channel image?", "751926": "There are many ways. When you build your model using pretrained ResNet which wants 3 channels, just put a new bottom on it. For example \n\n x = tf.keras.Input(shape=(64,64,1))\n x = tf.keras.layers.Concatenate()([x, x, x])\n ResNet34(weights='imagenet',include_top=False)(x)", "752139": "great", "761304": "suvalex : Either use what @cdeotte mentioned, or you can just sum up the first layer weights and replace that in a new layer that uses 1 channel (shown here for efficient-net):\n\n", "761362": "Hi @shujun717 , as mentioned in this paper which talks about using dropout before pooling layer: https://arxiv.org/ftp/arxiv/papers/1512/1512.00242.pdf , did u use probabilistic weighted pooling during test time? If yes, can u pls share the code?", "770689": "shujun717 I hope you will make your kernel public after the competition. Even after applying your approach I'm nowhere near your score. Thanks in advance." }, "source": "meta" }