{
  "id": 205424,
  "title": "Gambler's loss for noisy label in pytorch ",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/205424",
  "author_name": "",
  "post_date": "2020-12-20T05:50:04.125643200Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I read some discussing and it was recommended to use gambler's loss function. so </p>\n<p>step - </p>\n<ul>\n<li>Train the CNN on normal loss function for 10 epochs </li>\n<li>Train that CNN with Gambler's loss function for 10 epochs </li>\n</ul>\n<p>After step 1  i was able to get 85% of accuracy but when i add </p>\n<pre><code>reward = 0.6\ndef custom_loss(outputs,target):\n        outputs  = F.softmax(outputs,dim=1)\n        outputs, reservation = outputs[:,:-1], outputs[:,-1]\n        gain = torch.gather(outputs, dim=1, \n                                         index=target.unsqueeze(1)).squeeze()\n        doubling_rate = (gain.add(reservation.div(reward))).log()\n\n        loss = -outputs.mean()\n        return loss\n</code></pre>\n<p>This loss function which is GAMBLER'S LOSS function CNN accuracy start declining. And now it is in range of 20%. I tried reward = {2.2,3,0.6}</p>\n<p>What is wrong ?</p>",
  "messages": [
    {
      "id": "1119498",
      "postDate": "12/20/2020 05:50:04",
      "content": "<p>I read some discussing and it was recommended to use gambler's loss function. so </p>\n<p>step - </p>\n<ul>\n<li>Train the CNN on normal loss function for 10 epochs </li>\n<li>Train that CNN with Gambler's loss function for 10 epochs </li>\n</ul>\n<p>After step 1  i was able to get 85% of accuracy but when i add </p>\n<pre><code>reward = 0.6\ndef custom_loss(outputs,target):\n        outputs  = F.softmax(outputs,dim=1)\n        outputs, reservation = outputs[:,:-1], outputs[:,-1]\n        gain = torch.gather(outputs, dim=1, \n                                         index=target.unsqueeze(1)).squeeze()\n        doubling_rate = (gain.add(reservation.div(reward))).log()\n\n        loss = -outputs.mean()\n        return loss\n</code></pre>\n<p>This loss function which is GAMBLER'S LOSS function CNN accuracy start declining. And now it is in range of 20%. I tried reward = {2.2,3,0.6}</p>\n<p>What is wrong ?</p>",
      "rawMarkdown": "I read some discussing and it was recommended to use gambler's loss function. so \n\nstep - \n- Train the CNN on normal loss function for 10 epochs \n- Train that CNN with Gambler's loss function for 10 epochs \n\nAfter step 1  i was able to get 85% of accuracy but when i add \n\n```\nreward = 0.6\ndef custom_loss(outputs,target):\n        outputs  = F.softmax(outputs,dim=1)\n        outputs, reservation = outputs[:,:-1], outputs[:,-1]\n        gain = torch.gather(outputs, dim=1, \n                                         index=target.unsqueeze(1)).squeeze()\n        doubling_rate = (gain.add(reservation.div(reward))).log()\n        \n        loss = -outputs.mean()\n        return loss\n```\nThis loss function which is GAMBLER'S LOSS function CNN accuracy start declining. And now it is in range of 20%. I tried reward = {2.2,3,0.6}\n\nWhat is wrong ?",
      "votes": null
    },
    {
      "id": "1119502",
      "postDate": "12/20/2020 05:54:26",
      "content": "<pre><code>    for epoch in tqdm(range(num_epochs)):\n\n        #Evaluation and training on training dataset\n        model.train()\n        for i,(image,label) in enumerate(train_loader):\n            images= image['image'].to(device)\n            labels= label.to(device)\n\n            outputs=model(images.float())\n            loss = custom_loss(outputs,labels)\n            loss.backward()\n            optimizer.step()\n            train_loss+= loss.cpu().data*images.size(0)\n            _,prediction=torch.max(outputs.data,1)\n        scheduler.step()\n</code></pre>\n<p>This is part of my code using <code>custom_loss</code></p>",
      "rawMarkdown": "```\n    for epoch in tqdm(range(num_epochs)):\n\n        #Evaluation and training on training dataset\n        model.train()\n        for i,(image,label) in enumerate(train_loader):\n            images= image['image'].to(device)\n            labels= label.to(device)\n\n            outputs=model(images.float())\n            loss = custom_loss(outputs,labels)\n            loss.backward()\n            optimizer.step()\n            train_loss+= loss.cpu().data*images.size(0)\n            _,prediction=torch.max(outputs.data,1)\n        scheduler.step()\n```\n\nThis is part of my code using `custom_loss`",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1119502,
      "author_name": "rajanlagah",
      "author_url": "",
      "post_date": "12/20/2020 05:54:26",
      "content": "<pre><code>    for epoch in tqdm(range(num_epochs)):\n\n        #Evaluation and training on training dataset\n        model.train()\n        for i,(image,label) in enumerate(train_loader):\n            images= image['image'].to(device)\n            labels= label.to(device)\n\n            outputs=model(images.float())\n            loss = custom_loss(outputs,labels)\n            loss.backward()\n            optimizer.step()\n            train_loss+= loss.cpu().data*images.size(0)\n            _,prediction=torch.max(outputs.data,1)\n        scheduler.step()\n</code></pre>\n<p>This is part of my code using <code>custom_loss</code></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1119498": "I read some discussing and it was recommended to use gambler's loss function. so \n\nstep - \n- Train the CNN on normal loss function for 10 epochs \n- Train that CNN with Gambler's loss function for 10 epochs \n\nAfter step 1  i was able to get 85% of accuracy but when i add \n\n```\nreward = 0.6\ndef custom_loss(outputs,target):\n        outputs  = F.softmax(outputs,dim=1)\n        outputs, reservation = outputs[:,:-1], outputs[:,-1]\n        gain = torch.gather(outputs, dim=1, \n                                         index=target.unsqueeze(1)).squeeze()\n        doubling_rate = (gain.add(reservation.div(reward))).log()\n        \n        loss = -outputs.mean()\n        return loss\n```\nThis loss function which is GAMBLER'S LOSS function CNN accuracy start declining. And now it is in range of 20%. I tried reward = {2.2,3,0.6}\n\nWhat is wrong ?",
    "1119502": "```\n    for epoch in tqdm(range(num_epochs)):\n\n        #Evaluation and training on training dataset\n        model.train()\n        for i,(image,label) in enumerate(train_loader):\n            images= image['image'].to(device)\n            labels= label.to(device)\n\n            outputs=model(images.float())\n            loss = custom_loss(outputs,labels)\n            loss.backward()\n            optimizer.step()\n            train_loss+= loss.cpu().data*images.size(0)\n            _,prediction=torch.max(outputs.data,1)\n        scheduler.step()\n```\n\nThis is part of my code using `custom_loss`"
  },
  "source": "meta"
}