{
  "id": 204682,
  "title": "Test loss not decreasing beyond ~0.4, with accuracy on LB 85.9",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/204682",
  "author_name": "",
  "post_date": "2020-12-16T10:39:19.505684900Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Using resnet18/50 variants not freezing BN layers, added one/two linear layers for fine-tuning. <br>\nRelu activations, with dropouts.<br>\nUsing Resize, RandomCrop, RandomRotation, RandomHorizontalFlip, ColorJitter for image augmentation at train and test time.<br>\nUsed ReduceLROnPlateau</p>\n<p>Still the test loss is not reducing beyond 0.43, and train loss is around 0.39 after 20 epochs leaderboard accuracy with this model is 85.9, <br>\nIf i train it for some more epochs its starting to overfit.<br>\nHow should i further proceed to reduce my test loss and increase accuracy?</p>",
  "messages": [
    {
      "id": "1115515",
      "postDate": "12/16/2020 10:39:19",
      "content": "<p>Using resnet18/50 variants not freezing BN layers, added one/two linear layers for fine-tuning. <br>\nRelu activations, with dropouts.<br>\nUsing Resize, RandomCrop, RandomRotation, RandomHorizontalFlip, ColorJitter for image augmentation at train and test time.<br>\nUsed ReduceLROnPlateau</p>\n<p>Still the test loss is not reducing beyond 0.43, and train loss is around 0.39 after 20 epochs leaderboard accuracy with this model is 85.9, <br>\nIf i train it for some more epochs its starting to overfit.<br>\nHow should i further proceed to reduce my test loss and increase accuracy?</p>",
      "rawMarkdown": "Using resnet18/50 variants not freezing BN layers, added one/two linear layers for fine-tuning. \nRelu activations, with dropouts.\nUsing Resize, RandomCrop, RandomRotation, RandomHorizontalFlip, ColorJitter for image augmentation at train and test time.\nUsed ReduceLROnPlateau\n\nStill the test loss is not reducing beyond 0.43, and train loss is around 0.39 after 20 epochs leaderboard accuracy with this model is 85.9, \nIf i train it for some more epochs its starting to overfit.\nHow should i further proceed to reduce my test loss and increase accuracy?",
      "votes": null
    },
    {
      "id": "1115549",
      "postDate": "12/16/2020 11:06:28",
      "content": "<p>What is the loss function you are using? <code>CrossEntropyLoss</code> ?</p>",
      "rawMarkdown": "What is the loss function you are using? `CrossEntropyLoss` ?",
      "votes": null
    },
    {
      "id": "1115554",
      "postDate": "12/16/2020 11:13:18",
      "content": "<p>loss function labelsmoothing loss</p>",
      "rawMarkdown": "loss function labelsmoothing loss",
      "votes": null
    },
    {
      "id": "1115561",
      "postDate": "12/16/2020 11:25:46",
      "content": "<p>Did you try different loss functions? With <code>CrossEntropyLoss</code>, without TTA and a \"simple\" model like EF-B1, I was able to achieve 0.8720 CV and 0.869 LB</p>\n<blockquote>\n  <p>If i train it for some more epochs its starting to overfit.</p>\n</blockquote>\n<p>Are you using Early-Stopping, Regularisations (e.g. weight-decay), smaller model e.g. EF-B0 etc.</p>",
      "rawMarkdown": "Did you try different loss functions? With `CrossEntropyLoss`, without TTA and a \"simple\" model like EF-B1, I was able to achieve 0.8720 CV and 0.869 LB\n\n\n> If i train it for some more epochs its starting to overfit.\n\nAre you using Early-Stopping, Regularisations (e.g. weight-decay), smaller model e.g. EF-B0 etc.",
      "votes": null
    },
    {
      "id": "1116712",
      "postDate": "12/17/2020 12:15:35",
      "content": "<p>Simple resnet50 is giving me 0.87. First try to make baseline then improve with different augmentations etc</p>",
      "rawMarkdown": "Simple resnet50 is giving me 0.87. First try to make baseline then improve with different augmentations etc",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1115549,
      "author_name": "aliabdin1",
      "author_url": "",
      "post_date": "12/16/2020 11:06:28",
      "content": "<p>What is the loss function you are using? <code>CrossEntropyLoss</code> ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1115554,
      "author_name": "saikumargv",
      "author_url": "",
      "post_date": "12/16/2020 11:13:18",
      "content": "<p>loss function labelsmoothing loss</p>",
      "votes": null,
      "replies": [
        {
          "id": 1115561,
          "author_name": "aliabdin1",
          "author_url": "",
          "post_date": "12/16/2020 11:25:46",
          "content": "<p>Did you try different loss functions? With <code>CrossEntropyLoss</code>, without TTA and a \"simple\" model like EF-B1, I was able to achieve 0.8720 CV and 0.869 LB</p>\n<blockquote>\n  <p>If i train it for some more epochs its starting to overfit.</p>\n</blockquote>\n<p>Are you using Early-Stopping, Regularisations (e.g. weight-decay), smaller model e.g. EF-B0 etc.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1116712,
      "author_name": "deepdreamx",
      "author_url": "",
      "post_date": "12/17/2020 12:15:35",
      "content": "<p>Simple resnet50 is giving me 0.87. First try to make baseline then improve with different augmentations etc</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1115515": "Using resnet18/50 variants not freezing BN layers, added one/two linear layers for fine-tuning. \nRelu activations, with dropouts.\nUsing Resize, RandomCrop, RandomRotation, RandomHorizontalFlip, ColorJitter for image augmentation at train and test time.\nUsed ReduceLROnPlateau\n\nStill the test loss is not reducing beyond 0.43, and train loss is around 0.39 after 20 epochs leaderboard accuracy with this model is 85.9, \nIf i train it for some more epochs its starting to overfit.\nHow should i further proceed to reduce my test loss and increase accuracy?",
    "1115549": "What is the loss function you are using? `CrossEntropyLoss` ?",
    "1115554": "loss function labelsmoothing loss",
    "1115561": "Did you try different loss functions? With `CrossEntropyLoss`, without TTA and a \"simple\" model like EF-B1, I was able to achieve 0.8720 CV and 0.869 LB\n\n\n> If i train it for some more epochs its starting to overfit.\n\nAre you using Early-Stopping, Regularisations (e.g. weight-decay), smaller model e.g. EF-B0 etc.",
    "1116712": "Simple resnet50 is giving me 0.87. First try to make baseline then improve with different augmentations etc"
  },
  "source": "meta"
}