{ "id": 110246, "title": "Public LB 0.988 solution", "url": "/competitions/recursion-cellular-image-classification/discussion/110246", "author_name": "", "post_date": "2019-09-26T08:46:26.685365400Z", "votes": 18, "comment_count": 15, "views": 0, "content": "

While waiting for other top teams to share their solutions, here is our:

\n\n

Ensemble of ensemble with ensemble of pseudo of ensemble

\n\n

\"Schema\"

", "messages": [ { "id": "634407", "postDate": "09/26/2019 08:46:26", "content": "

While waiting for other top teams to share their solutions, here is our:

\n\n

Ensemble of ensemble with ensemble of pseudo of ensemble

\n\n

\"Schema\"

", "rawMarkdown": "While waiting for other top teams to share their solutions, here is our:\n\nEnsemble of ensemble with ensemble of pseudo of ensemble\n\n![Schema](http://stonestacking.co.uk/wp-content/uploads/2016/11/13087354_10204857975884114_7473766630052528712_n.jpg)", "votes": null }, { "id": "634419", "postDate": "09/26/2019 09:00:23", "content": "
\n

Ensemble of ensemble with ensemble of pseudo of ensemble

\n
\n\n

it's what I call: ultra instinct ensemble 😄 😄

", "rawMarkdown": "> Ensemble of ensemble with ensemble of pseudo of ensemble\n\nit's what I call: ultra instinct ensemble 😄 😄", "votes": null }, { "id": "634453", "postDate": "09/26/2019 10:12:57", "content": "

The picture you chose looks more of a stacking than ensembles. A close enough pictorial representation of ensembling could be this \n\"\"

", "rawMarkdown": "The picture you chose looks more of a stacking than ensembles. A close enough pictorial representation of ensembling could be this \n![](https://media0.giphy.com/media/cIn9zl1ZaasZFXsjf7/200w.gif?cid=790b7611a84f884b4fd0181cbf3db64a68eaf86c666dbddf&rid=200w.gif)", "votes": null }, { "id": "634473", "postDate": "09/26/2019 10:35:22", "content": "

LOL, I know these guys :)

\n\n

I hope they show us their ensembling power in the world championship the next week.

", "rawMarkdown": "LOL, [I know these guys](https://lol.gamepedia.com/G2_Esports) :) \n\nI hope they show us their ensembling power in the world championship the next week.", "votes": null }, { "id": "634513", "postDate": "09/26/2019 11:52:00", "content": "

Thanks for the last minute hint~

", "rawMarkdown": "Thanks for the last minute hint~", "votes": null }, { "id": "634674", "postDate": "09/26/2019 15:38:42", "content": "

I see some leakage seeping into your ensemble.

", "rawMarkdown": "I see some leakage seeping into your ensemble.", "votes": null }, { "id": "634713", "postDate": "09/26/2019 16:48:45", "content": "

My experience in a nutshell

\n\n

\"\"

", "rawMarkdown": "My experience in a nutshell\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1827990%2Faf0af585d3833ecdac712bce32e5ebbe%2Fmeme.jpg?generation=1569516505444675&alt=media)", "votes": null }, { "id": "634744", "postDate": "09/26/2019 17:34:23", "content": "

Your score suggests that the natual of the experiment setup is still greatly underutilized by your method.

\n\n

As of right now, my preprocessing, augmentation, model archtecure a and training procedure are the same as 2 months ago when I scored .397. Every bit of difference between my .4 and .8 solution is due to how to use the knowledge about the experiment setup. And I think I am still not exploiting them enough.

", "rawMarkdown": "Your score suggests that the natual of the experiment setup is still greatly underutilized by your method. \n\nAs of right now, my preprocessing, augmentation, model archtecure a and training procedure are the same as 2 months ago when I scored .397. Every bit of difference between my .4 and .8 solution is due to how to use the knowledge about the experiment setup. And I think I am still not exploiting them enough.", "votes": null }, { "id": "634751", "postDate": "09/26/2019 18:01:26", "content": "

That's precisely what I'm feeling right now. In 20 days since I joined this competition, I basically use one single efficientnetB4 with the same preprocessing. I've improved my score from 0.3 to 0.7 just by making data split is consistent. Tried a couple of augments and experimented with control set, got to 0.77. Unfortunately. I had no time to try metric learning. Your recent climb up the LB is quite impressive, I wonder what experiment setup tricks can make this.

", "rawMarkdown": "That's precisely what I'm feeling right now. In 20 days since I joined this competition, I basically use one single efficientnetB4 with the same preprocessing. I've improved my score from 0.3 to 0.7 just by making data split is consistent. Tried a couple of augments and experimented with control set, got to 0.77. Unfortunately. I had no time to try metric learning. Your recent climb up the LB is quite impressive, I wonder what experiment setup tricks can make this.", "votes": null }, { "id": "634759", "postDate": "09/26/2019 18:13:31", "content": "

It would be interesting to learn about the things in experiment setup that made such a big difference for you after the competition is over!

", "rawMarkdown": "It would be interesting to learn about the things in experiment setup that made such a big difference for you after the competition is over!", "votes": null }, { "id": "634900", "postDate": "09/26/2019 23:15:51", "content": "

our ensemble performs similar as the highest single model, pretty hard to select the submissions. Good luck to all!

", "rawMarkdown": "our ensemble performs similar as the highest single model, pretty hard to select the submissions. Good luck to all!", "votes": null }, { "id": "634908", "postDate": "09/27/2019 00:03:34", "content": "

The 4 groups of 277 brought me to .7 range, and to go to .8 range I used prediction balancing method. See https://github.com/PavelOstyakov/predictions_balancing

", "rawMarkdown": "The 4 groups of 277 brought me to .7 range, and to go to .8 range I used prediction balancing method. See https://github.com/PavelOstyakov/predictions_balancing", "votes": null }, { "id": "634914", "postDate": "09/27/2019 00:10:05", "content": "

I got above 0.7 the same way. Never seen this method, will take a look. Thank you!

", "rawMarkdown": "I got above 0.7 the same way. Never seen this method, will take a look. Thank you!", "votes": null }, { "id": "634956", "postDate": "09/27/2019 02:06:11", "content": "

yeah, so true, but we are the tiny pile on the corner...

", "rawMarkdown": "yeah, so true, but we are the tiny pile on the corner...", "votes": null }, { "id": "635131", "postDate": "09/27/2019 07:27:05", "content": "

@ryanzhang What do you mean by \"4 groups of 277\"? Plates leak? I used it the same way as in public kernel, although thought about training with it

", "rawMarkdown": "ryanzhang What do you mean by \"4 groups of 277\"? Plates leak? I used it the same way as in public kernel, although thought about training with it", "votes": null }, { "id": "635388", "postDate": "09/27/2019 13:16:33", "content": "

I added the group as an additional input to the model.

\n\n

```\nclass MyModel(nn.Module):\n def init(self, base, num_cnn_features, embedding_dim=4):\n super(MyModel, self).init()\n self.base = base\n self.base.classifier = Identity()\n self.embedding = nn.Embedding(4, embedding_dim)\n self.classifier = nn.Linear(num_cnn_features + embedding_dim, 1108)

\n\n
def forward(self, image, group):\n    embedding = self.embedding(group)\n    features = torch.cat([self.base(image), embedding], 1)\n    return self.classifier(features)\n
\n\n

```

\n\n

I guess as the model becomes better, this additional information may not be crucial, but it did speed up convergence a lot for me. My model took about 12 epochs to reach convergence.

", "rawMarkdown": "I added the group as an additional input to the model. \n\n```\nclass MyModel(nn.Module):\n def __init__(self, base, num_cnn_features, embedding_dim=4):\n super(MyModel, self).__init__()\n self.base = base\n self.base.classifier = Identity()\n self.embedding = nn.Embedding(4, embedding_dim)\n self.classifier = nn.Linear(num_cnn_features + embedding_dim, 1108)\n\n def forward(self, image, group):\n embedding = self.embedding(group)\n features = torch.cat([self.base(image), embedding], 1)\n return self.classifier(features)\n```\n\nI guess as the model becomes better, this additional information may not be crucial, but it did speed up convergence a lot for me. My model took about 12 epochs to reach convergence.", "votes": null } ], "comments": [ { "id": 634419, "author_name": "jesucristo", "author_url": "", "post_date": "09/26/2019 09:00:23", "content": "
\n

Ensemble of ensemble with ensemble of pseudo of ensemble

\n
\n\n

it's what I call: ultra instinct ensemble 😄 😄

", "votes": null, "replies": [] }, { "id": 634453, "author_name": "cyberia", "author_url": "", "post_date": "09/26/2019 10:12:57", "content": "

The picture you chose looks more of a stacking than ensembles. A close enough pictorial representation of ensembling could be this \n\"\"

", "votes": null, "replies": [ { "id": 634473, "author_name": "zaharch", "author_url": "", "post_date": "09/26/2019 10:35:22", "content": "

LOL, I know these guys :)

\n\n

I hope they show us their ensembling power in the world championship the next week.

", "votes": null, "replies": [] } ] }, { "id": 634513, "author_name": "ryanzhang", "author_url": "", "post_date": "09/26/2019 11:52:00", "content": "

Thanks for the last minute hint~

", "votes": null, "replies": [] }, { "id": 634674, "author_name": "inversion", "author_url": "", "post_date": "09/26/2019 15:38:42", "content": "

I see some leakage seeping into your ensemble.

", "votes": null, "replies": [] }, { "id": 634713, "author_name": "devvindan", "author_url": "", "post_date": "09/26/2019 16:48:45", "content": "

My experience in a nutshell

\n\n

\"\"

", "votes": null, "replies": [ { "id": 634744, "author_name": "ryanzhang", "author_url": "", "post_date": "09/26/2019 17:34:23", "content": "

Your score suggests that the natual of the experiment setup is still greatly underutilized by your method.

\n\n

As of right now, my preprocessing, augmentation, model archtecure a and training procedure are the same as 2 months ago when I scored .397. Every bit of difference between my .4 and .8 solution is due to how to use the knowledge about the experiment setup. And I think I am still not exploiting them enough.

", "votes": null, "replies": [] }, { "id": 634751, "author_name": "cateek", "author_url": "", "post_date": "09/26/2019 18:01:26", "content": "

That's precisely what I'm feeling right now. In 20 days since I joined this competition, I basically use one single efficientnetB4 with the same preprocessing. I've improved my score from 0.3 to 0.7 just by making data split is consistent. Tried a couple of augments and experimented with control set, got to 0.77. Unfortunately. I had no time to try metric learning. Your recent climb up the LB is quite impressive, I wonder what experiment setup tricks can make this.

", "votes": null, "replies": [] }, { "id": 634759, "author_name": "devvindan", "author_url": "", "post_date": "09/26/2019 18:13:31", "content": "

It would be interesting to learn about the things in experiment setup that made such a big difference for you after the competition is over!

", "votes": null, "replies": [] }, { "id": 634908, "author_name": "ryanzhang", "author_url": "", "post_date": "09/27/2019 00:03:34", "content": "

The 4 groups of 277 brought me to .7 range, and to go to .8 range I used prediction balancing method. See https://github.com/PavelOstyakov/predictions_balancing

", "votes": null, "replies": [] }, { "id": 634914, "author_name": "cateek", "author_url": "", "post_date": "09/27/2019 00:10:05", "content": "

I got above 0.7 the same way. Never seen this method, will take a look. Thank you!

", "votes": null, "replies": [] }, { "id": 635131, "author_name": "devvindan", "author_url": "", "post_date": "09/27/2019 07:27:05", "content": "

@ryanzhang What do you mean by \"4 groups of 277\"? Plates leak? I used it the same way as in public kernel, although thought about training with it

", "votes": null, "replies": [] }, { "id": 635388, "author_name": "ryanzhang", "author_url": "", "post_date": "09/27/2019 13:16:33", "content": "

I added the group as an additional input to the model.

\n\n

```\nclass MyModel(nn.Module):\n def init(self, base, num_cnn_features, embedding_dim=4):\n super(MyModel, self).init()\n self.base = base\n self.base.classifier = Identity()\n self.embedding = nn.Embedding(4, embedding_dim)\n self.classifier = nn.Linear(num_cnn_features + embedding_dim, 1108)

\n\n
def forward(self, image, group):\n    embedding = self.embedding(group)\n    features = torch.cat([self.base(image), embedding], 1)\n    return self.classifier(features)\n
\n\n

```

\n\n

I guess as the model becomes better, this additional information may not be crucial, but it did speed up convergence a lot for me. My model took about 12 epochs to reach convergence.

", "votes": null, "replies": [] } ] }, { "id": 634900, "author_name": "yiheng", "author_url": "", "post_date": "09/26/2019 23:15:51", "content": "

our ensemble performs similar as the highest single model, pretty hard to select the submissions. Good luck to all!

", "votes": null, "replies": [] }, { "id": 634956, "author_name": "raynardj", "author_url": "", "post_date": "09/27/2019 02:06:11", "content": "

yeah, so true, but we are the tiny pile on the corner...

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "634407": "While waiting for other top teams to share their solutions, here is our:\n\nEnsemble of ensemble with ensemble of pseudo of ensemble\n\n![Schema](http://stonestacking.co.uk/wp-content/uploads/2016/11/13087354_10204857975884114_7473766630052528712_n.jpg)", "634419": "> Ensemble of ensemble with ensemble of pseudo of ensemble\n\nit's what I call: ultra instinct ensemble 😄 😄", "634453": "The picture you chose looks more of a stacking than ensembles. A close enough pictorial representation of ensembling could be this \n![](https://media0.giphy.com/media/cIn9zl1ZaasZFXsjf7/200w.gif?cid=790b7611a84f884b4fd0181cbf3db64a68eaf86c666dbddf&rid=200w.gif)", "634473": "LOL, [I know these guys](https://lol.gamepedia.com/G2_Esports) :) \n\nI hope they show us their ensembling power in the world championship the next week.", "634513": "Thanks for the last minute hint~", "634674": "I see some leakage seeping into your ensemble.", "634713": "My experience in a nutshell\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1827990%2Faf0af585d3833ecdac712bce32e5ebbe%2Fmeme.jpg?generation=1569516505444675&alt=media)", "634744": "Your score suggests that the natual of the experiment setup is still greatly underutilized by your method. \n\nAs of right now, my preprocessing, augmentation, model archtecure a and training procedure are the same as 2 months ago when I scored .397. Every bit of difference between my .4 and .8 solution is due to how to use the knowledge about the experiment setup. And I think I am still not exploiting them enough.", "634751": "That's precisely what I'm feeling right now. In 20 days since I joined this competition, I basically use one single efficientnetB4 with the same preprocessing. I've improved my score from 0.3 to 0.7 just by making data split is consistent. Tried a couple of augments and experimented with control set, got to 0.77. Unfortunately. I had no time to try metric learning. Your recent climb up the LB is quite impressive, I wonder what experiment setup tricks can make this.", "634759": "It would be interesting to learn about the things in experiment setup that made such a big difference for you after the competition is over!", "634900": "our ensemble performs similar as the highest single model, pretty hard to select the submissions. Good luck to all!", "634908": "The 4 groups of 277 brought me to .7 range, and to go to .8 range I used prediction balancing method. See https://github.com/PavelOstyakov/predictions_balancing", "634914": "I got above 0.7 the same way. Never seen this method, will take a look. Thank you!", "634956": "yeah, so true, but we are the tiny pile on the corner...", "635131": "ryanzhang What do you mean by \"4 groups of 277\"? Plates leak? I used it the same way as in public kernel, although thought about training with it", "635388": "I added the group as an additional input to the model. \n\n```\nclass MyModel(nn.Module):\n def __init__(self, base, num_cnn_features, embedding_dim=4):\n super(MyModel, self).__init__()\n self.base = base\n self.base.classifier = Identity()\n self.embedding = nn.Embedding(4, embedding_dim)\n self.classifier = nn.Linear(num_cnn_features + embedding_dim, 1108)\n\n def forward(self, image, group):\n embedding = self.embedding(group)\n features = torch.cat([self.base(image), embedding], 1)\n return self.classifier(features)\n```\n\nI guess as the model becomes better, this additional information may not be crucial, but it did speed up convergence a lot for me. My model took about 12 epochs to reach convergence." }, "source": "meta" }