![\"Image\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2070357%2F87339eeccaeb1c21ff1e02d50787a66b%2FScreenshot_2020-08-18%20Way%20to%20Heaven%20drawio%20-%20diagrams%20net.png?generation=1597733085472260&alt=media\")
Hey everyone, hope you are all doing good and \"shaking\" a lot. After seeing Mobassir's post, I have finally made my mind to put what I did in this competition here. Since it was my first competition, I could not implement a lot of things which were in my mind, but yeah, I have learned a lot at the same time.
\nFirst of all, I would like to tell that please experiment what you want to do, what happened with me was that after working for about 2 months with Pytorch, when the time came to work with TPU, I simply could not implement it, I have although finally implemented it with the help of @aliabdin1 After that, I was inactive for about a week before starting my work with Tensorflow and with the notebook shared by @cdeotte I trained a lot of models with different image sizes and some changes on the way.
\nAfter training all the models, what I did can be understood by the following:
\n
In this setup, all I did while ensembling was getting highest score for CV and setting the weights accordingly with the help of Weighted Power ensembling with Ranking. I finally got 0.9352 score for my CV, and I submitted it, which got me 0.9479 on public lb and 0.9321 on private lb.
\nAlthough when I checked my submissions, I saw that some of my single models were giving me about 0.9375 score on private lb, which makes me consider the fact that I had done something really wrong with the CV ensemble itself.
\nWith Pytorch, my model architecture was like this, as @nroman put it in his notebook, but with some modifictions:
\nclasses = 2\nclass Net(nn.Module):\n def __init__(self, arch, n_meta_features: int):\n super(Net, self).__init__()\n self.arch = arch\n if \"EfficientNet\" in str(arch.__class__):\n arch._fc = nn.Linear(in_features = 2304, out_features = 500, bias = True)\n self.classifier = nn.Linear(500 + 250, 2)\n self.meta = nn.Sequential(nn.Linear(n_meta_features, 500), \n nn.BatchNorm1d(500), \n nn.ReLU(), \n nn.Dropout(p = 0.25), \n nn.Linear(500, 250), \n nn.BatchNorm1d(250), \n nn.ReLU(), \n nn.Dropout(p = 0.2))\n\ndef forward(self, inputs):\n x, meta = inputs\n image_feats = self.arch(x)\n meta_feats = self.meta(meta)\n feats = torch.cat((image_feats, meta_feats), dim = 1)\n classifier = self.classifier(feats)\n return classifier\nI am planning on working on this architecture more and will see how it works since I have implemented the TPU now because the first place solution has the very much the same base I guess of course with some major changes.
\nThere are a lot of things which I have learned from the community, thanks everyone for the support, Now, with all the lesson learned, I would like to meet you all in the next competition!
\nThanks again.
", "messages": [ { "id": 975125, "postDate": "2020-08-18T06:56:30.510Z", "content": "Hey everyone, hope you are all doing good and \"shaking\" a lot. After seeing Mobassir's post, I have finally made my mind to put what I did in this competition here. Since it was my first competition, I could not implement a lot of things which were in my mind, but yeah, I have learned a lot at the same time.
\nFirst of all, I would like to tell that please experiment what you want to do, what happened with me was that after working for about 2 months with Pytorch, when the time came to work with TPU, I simply could not implement it, I have although finally implemented it with the help of @aliabdin1 After that, I was inactive for about a week before starting my work with Tensorflow and with the notebook shared by @cdeotte I trained a lot of models with different image sizes and some changes on the way.
\nAfter training all the models, what I did can be understood by the following:
\n
In this setup, all I did while ensembling was getting highest score for CV and setting the weights accordingly with the help of Weighted Power ensembling with Ranking. I finally got 0.9352 score for my CV, and I submitted it, which got me 0.9479 on public lb and 0.9321 on private lb.
\nAlthough when I checked my submissions, I saw that some of my single models were giving me about 0.9375 score on private lb, which makes me consider the fact that I had done something really wrong with the CV ensemble itself.
\nWith Pytorch, my model architecture was like this, as @nroman put it in his notebook, but with some modifictions:
\nclasses = 2\nclass Net(nn.Module):\n def __init__(self, arch, n_meta_features: int):\n super(Net, self).__init__()\n self.arch = arch\n if \"EfficientNet\" in str(arch.__class__):\n arch._fc = nn.Linear(in_features = 2304, out_features = 500, bias = True)\n self.classifier = nn.Linear(500 + 250, 2)\n self.meta = nn.Sequential(nn.Linear(n_meta_features, 500), \n nn.BatchNorm1d(500), \n nn.ReLU(), \n nn.Dropout(p = 0.25), \n nn.Linear(500, 250), \n nn.BatchNorm1d(250), \n nn.ReLU(), \n nn.Dropout(p = 0.2))\n\ndef forward(self, inputs):\n x, meta = inputs\n image_feats = self.arch(x)\n meta_feats = self.meta(meta)\n feats = torch.cat((image_feats, meta_feats), dim = 1)\n classifier = self.classifier(feats)\n return classifier\nI am planning on working on this architecture more and will see how it works since I have implemented the TPU now because the first place solution has the very much the same base I guess of course with some major changes.
\nThere are a lot of things which I have learned from the community, thanks everyone for the support, Now, with all the lesson learned, I would like to meet you all in the next competition!
\nThanks again.
", "rawMarkdown": "Hey everyone, hope you are all doing good and \"shaking\" a lot. After seeing Mobassir's post, I have finally made my mind to put what I did in this competition here. Since it was my first competition, I could not implement a lot of things which were in my mind, but yeah, I have learned a lot at the same time.\n\nFirst of all, I would like to tell that please experiment what you want to do, what happened with me was that after working for about 2 months with Pytorch, when the time came to work with TPU, I simply could not implement it, I have although finally implemented it with the help of @aliabdin1 After that, I was inactive for about a week before starting my work with Tensorflow and with the notebook shared by @cdeotte I trained a lot of models with different image sizes and some changes on the way.\n\nAfter training all the models, what I did can be understood by the following:\n\n\n\nIn this setup, all I did while ensembling was getting highest score for CV and setting the weights accordingly with the help of Weighted Power ensembling with Ranking. I finally got 0.9352 score for my CV, and I submitted it, which got me 0.9479 on public lb and 0.9321 on private lb.\n\nAlthough when I checked my submissions, I saw that some of my single models were giving me about 0.9375 score on private lb, which makes me consider the fact that I had done something really wrong with the CV ensemble itself.\n\nWith Pytorch, my model architecture was like this, as @nroman put it in his notebook, but with some modifictions:\n\n classes = 2\n class Net(nn.Module):\n def __init__(self, arch, n_meta_features: int):\n super(Net, self).__init__()\n self.arch = arch\n if \"EfficientNet\" in str(arch.__class__):\n arch._fc = nn.Linear(in_features = 2304, out_features = 500, bias = True)\n self.classifier = nn.Linear(500 + 250, 2)\n self.meta = nn.Sequential(nn.Linear(n_meta_features, 500), \n nn.BatchNorm1d(500), \n nn.ReLU(), \n nn.Dropout(p = 0.25), \n nn.Linear(500, 250), \n nn.BatchNorm1d(250), \n nn.ReLU(), \n nn.Dropout(p = 0.2))\n \n def forward(self, inputs):\n x, meta = inputs\n image_feats = self.arch(x)\n meta_feats = self.meta(meta)\n feats = torch.cat((image_feats, meta_feats), dim = 1)\n classifier = self.classifier(feats)\n return classifier\n\nI am planning on working on this architecture more and will see how it works since I have implemented the TPU now because the first place solution has the very much the same base I guess of course with some major changes.\n\nThere are a lot of things which I have learned from the community, thanks everyone for the support, Now, with all the lesson learned, I would like to meet you all in the next competition!\n\nThanks again.", "votes": 7 } ], "comments": [], "raw_markdown_by_id": { "975125": "Hey everyone, hope you are all doing good and \"shaking\" a lot. After seeing Mobassir's post, I have finally made my mind to put what I did in this competition here. Since it was my first competition, I could not implement a lot of things which were in my mind, but yeah, I have learned a lot at the same time.\n\nFirst of all, I would like to tell that please experiment what you want to do, what happened with me was that after working for about 2 months with Pytorch, when the time came to work with TPU, I simply could not implement it, I have although finally implemented it with the help of @aliabdin1 After that, I was inactive for about a week before starting my work with Tensorflow and with the notebook shared by @cdeotte I trained a lot of models with different image sizes and some changes on the way.\n\nAfter training all the models, what I did can be understood by the following:\n\n\n\nIn this setup, all I did while ensembling was getting highest score for CV and setting the weights accordingly with the help of Weighted Power ensembling with Ranking. I finally got 0.9352 score for my CV, and I submitted it, which got me 0.9479 on public lb and 0.9321 on private lb.\n\nAlthough when I checked my submissions, I saw that some of my single models were giving me about 0.9375 score on private lb, which makes me consider the fact that I had done something really wrong with the CV ensemble itself.\n\nWith Pytorch, my model architecture was like this, as @nroman put it in his notebook, but with some modifictions:\n\n classes = 2\n class Net(nn.Module):\n def __init__(self, arch, n_meta_features: int):\n super(Net, self).__init__()\n self.arch = arch\n if \"EfficientNet\" in str(arch.__class__):\n arch._fc = nn.Linear(in_features = 2304, out_features = 500, bias = True)\n self.classifier = nn.Linear(500 + 250, 2)\n self.meta = nn.Sequential(nn.Linear(n_meta_features, 500), \n nn.BatchNorm1d(500), \n nn.ReLU(), \n nn.Dropout(p = 0.25), \n nn.Linear(500, 250), \n nn.BatchNorm1d(250), \n nn.ReLU(), \n nn.Dropout(p = 0.2))\n \n def forward(self, inputs):\n x, meta = inputs\n image_feats = self.arch(x)\n meta_feats = self.meta(meta)\n feats = torch.cat((image_feats, meta_feats), dim = 1)\n classifier = self.classifier(feats)\n return classifier\n\nI am planning on working on this architecture more and will see how it works since I have implemented the TPU now because the first place solution has the very much the same base I guess of course with some major changes.\n\nThere are a lot of things which I have learned from the community, thanks everyone for the support, Now, with all the lesson learned, I would like to meet you all in the next competition!\n\nThanks again." } }