I am using centre net public kernel with backbone effnet b3 or resnet 34 ,what i find that the loss is not coming down beyond 20 -15 and keeps fluctuating.
\n\nWhat might be the issue.
\n\nAny help appreciated!
", "messages": [ { "id": 702466, "postDate": "2019-12-24T17:47:18.910Z", "content": "I am using centre net public kernel with backbone effnet b3 or resnet 34 ,what i find that the loss is not coming down beyond 20 -15 and keeps fluctuating.
\n\nWhat might be the issue.
\n\nAny help appreciated!
", "rawMarkdown": "I am using centre net public kernel with backbone effnet b3 or resnet 34 ,what i find that the loss is not coming down beyond 20 -15 and keeps fluctuating. \n\nWhat might be the issue.\n\nAny help appreciated!", "votes": 6 }, { "id": 702475, "postDate": "2019-12-24T18:08:04.543Z", "content": "Looks like your encoding of input data has issues. Check that your heatmap and the way you're encoding train data are the way they should look!
", "rawMarkdown": "Looks like your encoding of input data has issues. Check that your heatmap and the way you're encoding train data are the way they should look!", "replies": [ { "id": 702789, "postDate": "2019-12-25T06:48:02.747Z", "rawMarkdown": "", "isDeleted": true }, { "id": 702828, "postDate": "2019-12-25T08:20:18.307Z", "content": "@chroteus thanks for your insight..\nhttps://www.kaggle.com/hocop1/centernet-baseline
\n\ncould u help checking i dont think any heat map used over here..\nany pointers on how to make one. i use above kernel as reference and its label encoding...
", "rawMarkdown": "@chroteus thanks for your insight..\nhttps://www.kaggle.com/hocop1/centernet-baseline\n\ncould u help checking i dont think any heat map used over here..\nany pointers on how to make one. i use above kernel as reference and its label encoding..." }, { "id": 702840, "postDate": "2019-12-25T08:38:23.143Z", "content": "I think what @chroteus mean is :
\n\nWith pretty much the same input data format and proper optimizer as public kernel shouldn't have convergence issue. So if your loss couldn't go down below 15-20, then the input data or the loss or even the model architecture might have some bug.
", "rawMarkdown": "I think what @chroteus mean is : \n\nWith pretty much the same input data format and proper optimizer as public kernel shouldn't have convergence issue. So if your loss couldn't go down below 15-20, then the input data or the loss or even the model architecture might have some bug. " }, { "id": 702893, "postDate": "2019-12-25T10:00:59.470Z", "content": "For example, plot your heatmap and (downscaled) image.\nIf points don't overlap with cars, there's nothing for neural network to learn.
\n\nAnother issue, pointed by @xiejialun, if you have a bug in loss function (if you used a custom one), then it might also prevent learning.
", "rawMarkdown": "For example, plot your heatmap and (downscaled) image.\nIf points don't overlap with cars, there's nothing for neural network to learn.\n\nAnother issue, pointed by @xiejialun, if you have a bug in loss function (if you used a custom one), then it might also prevent learning.", "votes": 1 }, { "id": 703021, "postDate": "2019-12-25T13:36:06.843Z", "content": "1) @chroteus how to generate the heat map is there any utility function. I generated only the mask such mask(2d x,y)=1 for labels along with regr points.Is it sufficient ?
\n\n2)I tried using both F.bce with logits whichgives mean of loss based on no of elements but it gets stuck after .01 and dsnt makes any predictions not sure why , while custom one just does mean at batch level and sums it up at element level and is able to do meaningful predictions.\nMask loss=mask*log(sigmoid(pred))+(1-mask) log(1-pred)\nmask_loss.mean(0).sum()
", "rawMarkdown": "\n1) @chroteus how to generate the heat map is there any utility function. I generated only the mask such mask(2d x,y)=1 for labels along with regr points.Is it sufficient ?\n\n2)I tried using both F.bce with logits whichgives mean of loss based on no of elements but it gets stuck after .01 and dsnt makes any predictions not sure why , while custom one just does mean at batch level and sums it up at element level and is able to do meaningful predictions.\nMask loss=mask*log(sigmoid(pred))+(1-mask) log(1-pred)\nmask_loss.mean(0).sum() " }, { "id": 703160, "postDate": "2019-12-25T17:41:30.493Z", "content": "You can use this one, i took it from CenterNet paper. Draws a gaussian.\n```
\n\ndef draw_msra_gaussian(heatmap, center, sigma):\n tmp_size = sigma * 3\n mu_x = int(center[0] + 0.5)\n mu_y = int(center[1] + 0.5)\n w, h = heatmap.shape[0], heatmap.shape[1]\n ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]\n br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]\n if ul[0] >= h or ul[1] >= w or br[0] < 0 or br[1] < 0:\n return heatmap\n size = 2 * tmp_size + 1\n x = np.arange(0, size, 1, np.float32)\n y = x[:, np.newaxis]\n x0 = y0 = size // 2\n g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))\n g_x = max(0, -ul[0]), min(br[0], h) - ul[0]\n g_y = max(0, -ul[1]), min(br[1], w) - ul[1]\n img_x = max(0, ul[0]), min(br[0], h)\n img_y = max(0, ul[1]), min(br[1], w)\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]] = np.maximum(\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]],\n g[g_y[0]:g_y[1], g_x[0]:g_x[1]])\n return heatmap\n```
\n\nTo draw the heatmap, you use matplotlib. The imshow function.
\n\nGood luck!
", "rawMarkdown": "You can use this one, i took it from CenterNet paper. Draws a gaussian.\n```\n\ndef draw_msra_gaussian(heatmap, center, sigma):\n tmp_size = sigma * 3\n mu_x = int(center[0] + 0.5)\n mu_y = int(center[1] + 0.5)\n w, h = heatmap.shape[0], heatmap.shape[1]\n ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]\n br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]\n if ul[0] >= h or ul[1] >= w or br[0] < 0 or br[1] < 0:\n return heatmap\n size = 2 * tmp_size + 1\n x = np.arange(0, size, 1, np.float32)\n y = x[:, np.newaxis]\n x0 = y0 = size // 2\n g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))\n g_x = max(0, -ul[0]), min(br[0], h) - ul[0]\n g_y = max(0, -ul[1]), min(br[1], w) - ul[1]\n img_x = max(0, ul[0]), min(br[0], h)\n img_y = max(0, ul[1]), min(br[1], w)\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]] = np.maximum(\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]],\n g[g_y[0]:g_y[1], g_x[0]:g_x[1]])\n return heatmap\n```\n\nTo draw the heatmap, you use matplotlib. The imshow function.\n\nGood luck!", "votes": 4 } ] } ], "comments": [ { "id": 702475, "author_name": "Askar Bozcan", "author_url": "", "post_date": "2019-12-24T18:08:04.543000", "content": "Looks like your encoding of input data has issues. Check that your heatmap and the way you're encoding train data are the way they should look!
", "votes": 0, "replies": [ { "id": 702789, "author_name": "", "author_url": "", "post_date": "2019-12-25T06:48:02.747000", "content": "", "votes": 0, "replies": [] }, { "id": 702828, "author_name": "Jaideep", "author_url": "", "post_date": "2019-12-25T08:20:18.307000", "content": "@chroteus thanks for your insight..\nhttps://www.kaggle.com/hocop1/centernet-baseline
\n\ncould u help checking i dont think any heat map used over here..\nany pointers on how to make one. i use above kernel as reference and its label encoding...
", "votes": 0, "replies": [] }, { "id": 702840, "author_name": "Tsai29", "author_url": "", "post_date": "2019-12-25T08:38:23.143000", "content": "I think what @chroteus mean is :
\n\nWith pretty much the same input data format and proper optimizer as public kernel shouldn't have convergence issue. So if your loss couldn't go down below 15-20, then the input data or the loss or even the model architecture might have some bug.
", "votes": 0, "replies": [] }, { "id": 702893, "author_name": "Askar Bozcan", "author_url": "", "post_date": "2019-12-25T10:00:59.470000", "content": "For example, plot your heatmap and (downscaled) image.\nIf points don't overlap with cars, there's nothing for neural network to learn.
\n\nAnother issue, pointed by @xiejialun, if you have a bug in loss function (if you used a custom one), then it might also prevent learning.
", "votes": 1, "replies": [] }, { "id": 703021, "author_name": "Jaideep", "author_url": "", "post_date": "2019-12-25T13:36:06.843000", "content": "1) @chroteus how to generate the heat map is there any utility function. I generated only the mask such mask(2d x,y)=1 for labels along with regr points.Is it sufficient ?
\n\n2)I tried using both F.bce with logits whichgives mean of loss based on no of elements but it gets stuck after .01 and dsnt makes any predictions not sure why , while custom one just does mean at batch level and sums it up at element level and is able to do meaningful predictions.\nMask loss=mask*log(sigmoid(pred))+(1-mask) log(1-pred)\nmask_loss.mean(0).sum()
", "votes": 0, "replies": [] }, { "id": 703160, "author_name": "Askar Bozcan", "author_url": "", "post_date": "2019-12-25T17:41:30.493000", "content": "You can use this one, i took it from CenterNet paper. Draws a gaussian.\n```
\n\ndef draw_msra_gaussian(heatmap, center, sigma):\n tmp_size = sigma * 3\n mu_x = int(center[0] + 0.5)\n mu_y = int(center[1] + 0.5)\n w, h = heatmap.shape[0], heatmap.shape[1]\n ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]\n br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]\n if ul[0] >= h or ul[1] >= w or br[0] < 0 or br[1] < 0:\n return heatmap\n size = 2 * tmp_size + 1\n x = np.arange(0, size, 1, np.float32)\n y = x[:, np.newaxis]\n x0 = y0 = size // 2\n g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * sigma ** 2))\n g_x = max(0, -ul[0]), min(br[0], h) - ul[0]\n g_y = max(0, -ul[1]), min(br[1], w) - ul[1]\n img_x = max(0, ul[0]), min(br[0], h)\n img_y = max(0, ul[1]), min(br[1], w)\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]] = np.maximum(\n heatmap[img_y[0]:img_y[1], img_x[0]:img_x[1]],\n g[g_y[0]:g_y[1], g_x[0]:g_x[1]])\n return heatmap\n```
\n\nTo draw the heatmap, you use matplotlib. The imshow function.
\n\nGood luck!
", "votes": 4, "replies": [] } ] } ], "raw_markdown_by_id": { "702466": "I am using centre net public kernel with backbone effnet b3 or resnet 34 ,what i find that the loss is not coming down beyond 20 -15 and keeps fluctuating. \n\nWhat might be the issue.\n\nAny help appreciated!", "702475": "Looks like your encoding of input data has issues. Check that your heatmap and the way you're encoding train data are the way they should look!" } }