So I am new to competing in kaggle and with my knowledge of the basics I decided to make a quick simple CNN model. But I am having trouble picking the out channels of my CNN. Currently my CNN looks like this:
\nclass LeafModel(nn.Module):\n def __init__(self):\n super(LeafModel, self).__init__()\n self.cnn1 = nn.Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=2, padding=1)\n self.cnn2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=3, stride=2, padding=1)\n self.lin1 = nn.Linear(90000, 5)\n\n def forward(self, inp):\n inp = F.relu(self.cnn1(inp))\n inp = F.relu(self.cnn2(inp))\n out = self.lin1(inp.view(-1, 90000))\n return out\nThe model is training and the loss is decreasing but the problem that I am facing is that my validation metric is going down with each epoch with is an indicator of overfitting. I know that I should decrease the number of parameters in my model and make the architecture simpler. But I am having trouble picking new out_channel count for the CNNs. Is there any rule of thumb? Should I keep my model same and go for regularization techniques?
\nUPD: Found something useful about this topic here
", "messages": [ { "id": "1090225", "postDate": "11/25/2020 07:16:30", "content": "So I am new to competing in kaggle and with my knowledge of the basics I decided to make a quick simple CNN model. But I am having trouble picking the out channels of my CNN. Currently my CNN looks like this:
\nclass LeafModel(nn.Module):\n def __init__(self):\n super(LeafModel, self).__init__()\n self.cnn1 = nn.Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=2, padding=1)\n self.cnn2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=3, stride=2, padding=1)\n self.lin1 = nn.Linear(90000, 5)\n\n def forward(self, inp):\n inp = F.relu(self.cnn1(inp))\n inp = F.relu(self.cnn2(inp))\n out = self.lin1(inp.view(-1, 90000))\n return out\nThe model is training and the loss is decreasing but the problem that I am facing is that my validation metric is going down with each epoch with is an indicator of overfitting. I know that I should decrease the number of parameters in my model and make the architecture simpler. But I am having trouble picking new out_channel count for the CNNs. Is there any rule of thumb? Should I keep my model same and go for regularization techniques?
\nUPD: Found something useful about this topic here
", "rawMarkdown": "So I am new to competing in kaggle and with my knowledge of the basics I decided to make a quick simple CNN model. But I am having trouble picking the out channels of my CNN. Currently my CNN looks like this:\n```\nclass LeafModel(nn.Module):\n def __init__(self):\n super(LeafModel, self).__init__()\n self.cnn1 = nn.Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=2, padding=1)\n self.cnn2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=3, stride=2, padding=1)\n self.lin1 = nn.Linear(90000, 5)\n \n def forward(self, inp):\n inp = F.relu(self.cnn1(inp))\n inp = F.relu(self.cnn2(inp))\n out = self.lin1(inp.view(-1, 90000))\n return out\n```\nThe model is training and the loss is decreasing but the problem that I am facing is that my validation metric is going down with each epoch with is an indicator of overfitting. I know that I should decrease the number of parameters in my model and make the architecture simpler. But I am having trouble picking new out_channel count for the CNNs. Is there any rule of thumb? Should I keep my model same and go for regularization techniques?\n\nUPD: Found something useful about this topic [here](https://towardsdatascience.com/a-guide-to-an-efficient-way-to-build-neural-network-architectures-part-ii-hyper-parameter-42efca01e5d7)", "votes": null }, { "id": "1090360", "postDate": "11/25/2020 09:34:56", "content": "If your validation loss is decreasing enough with training loss then it's not overfitting. out_channel is nothing but the number of filters/kernels for your Conv layer. Intuitively, each of the filter/kernel learns different patterns in your image, for example, if you choose 3 filters, one of them can learn horizontal edges, other two can focus on vertical edges and angled edges etc. So you can set them to any number of your choice. Reduce the count if you think your model is complex.
My validation loss is increasing as well, just the training loss is decreasing.
", "rawMarkdown": "My validation loss is increasing as well, just the training loss is decreasing.", "votes": null }, { "id": "1091030", "postDate": "11/25/2020 18:28:11", "content": "If you are facing the same problem then this might help
", "rawMarkdown": "If you are facing the same problem then [this](https://towardsdatascience.com/a-guide-to-an-efficient-way-to-build-neural-network-architectures-part-ii-hyper-parameter-42efca01e5d7) might help", "votes": null } ], "comments": [ { "id": 1090360, "author_name": "kaushal2896", "author_url": "", "post_date": "11/25/2020 09:34:56", "content": "If your validation loss is decreasing enough with training loss then it's not overfitting. out_channel is nothing but the number of filters/kernels for your Conv layer. Intuitively, each of the filter/kernel learns different patterns in your image, for example, if you choose 3 filters, one of them can learn horizontal edges, other two can focus on vertical edges and angled edges etc. So you can set them to any number of your choice. Reduce the count if you think your model is complex.
My validation loss is increasing as well, just the training loss is decreasing.
", "votes": null, "replies": [] } ] }, { "id": 1091030, "author_name": "koushiksahu", "author_url": "", "post_date": "11/25/2020 18:28:11", "content": "If you are facing the same problem then this might help
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1090225": "So I am new to competing in kaggle and with my knowledge of the basics I decided to make a quick simple CNN model. But I am having trouble picking the out channels of my CNN. Currently my CNN looks like this:\n```\nclass LeafModel(nn.Module):\n def __init__(self):\n super(LeafModel, self).__init__()\n self.cnn1 = nn.Conv2d(in_channels=3, out_channels=6, kernel_size=3, stride=2, padding=1)\n self.cnn2 = nn.Conv2d(in_channels=6, out_channels=12, kernel_size=3, stride=2, padding=1)\n self.lin1 = nn.Linear(90000, 5)\n \n def forward(self, inp):\n inp = F.relu(self.cnn1(inp))\n inp = F.relu(self.cnn2(inp))\n out = self.lin1(inp.view(-1, 90000))\n return out\n```\nThe model is training and the loss is decreasing but the problem that I am facing is that my validation metric is going down with each epoch with is an indicator of overfitting. I know that I should decrease the number of parameters in my model and make the architecture simpler. But I am having trouble picking new out_channel count for the CNNs. Is there any rule of thumb? Should I keep my model same and go for regularization techniques?\n\nUPD: Found something useful about this topic [here](https://towardsdatascience.com/a-guide-to-an-efficient-way-to-build-neural-network-architectures-part-ii-hyper-parameter-42efca01e5d7)", "1090360": "If your validation loss is decreasing enough with training loss then it's not overfitting. `out_channel` is nothing but the number of filters/kernels for your Conv layer. Intuitively, each of the filter/kernel learns different patterns in your image, for example, if you choose 3 filters, one of them can learn horizontal edges, other two can focus on vertical edges and angled edges etc. So you can set them to any number of your choice. Reduce the count if you think your model is complex.", "1090576": "My validation loss is increasing as well, just the training loss is decreasing.", "1091030": "If you are facing the same problem then [this](https://towardsdatascience.com/a-guide-to-an-efficient-way-to-build-neural-network-architectures-part-ii-hyper-parameter-42efca01e5d7) might help" }, "source": "meta" }