{
  "id": 217030,
  "title": "Which your best parameters of Bi-Tempered Logistic?",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/217030",
  "author_name": "yahhogogo",
  "post_date": "2021-02-05T01:01:28.918000",
  "votes": 11,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I implemented Bi-Tempered Logistic function with Efficientnetb3.<br>\n(here is notebook : <a href=\"https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning\" target=\"_blank\">https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning</a>)<br>\nThe results i tried with some parameter sets of t1 and t2 are bellow.</p>\n<p><img src=\"https://github.com/haryuu/files_upload/blob/main/2021-02-05095028.png?raw=true\" alt=\"\"><br>\nNote that the tag of label_smoothing you could see blank is meaning to label_smoothing=0.0.</p>",
  "messages": [
    {
      "id": 1186644,
      "postDate": "2021-02-05T01:01:28.920Z",
      "content": "<p>I implemented Bi-Tempered Logistic function with Efficientnetb3.<br>\n(here is notebook : <a href=\"https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning\" target=\"_blank\">https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning</a>)<br>\nThe results i tried with some parameter sets of t1 and t2 are bellow.</p>\n<p><img src=\"https://github.com/haryuu/files_upload/blob/main/2021-02-05095028.png?raw=true\" alt=\"\"><br>\nNote that the tag of label_smoothing you could see blank is meaning to label_smoothing=0.0.</p>",
      "rawMarkdown": "I implemented Bi-Tempered Logistic function with Efficientnetb3.\n(here is notebook : https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning)\nThe results i tried with some parameter sets of t1 and t2 are bellow.\n\n![](https://github.com/haryuu/files_upload/blob/main/2021-02-05095028.png?raw=true)\nNote that the tag of label_smoothing you could see blank is meaning to label_smoothing=0.0.",
      "votes": 11
    },
    {
      "id": 1186865,
      "postDate": "2021-02-05T05:02:37.533Z",
      "content": "<p>What is the feedback of the LB?</p>",
      "rawMarkdown": "What is the feedback of the LB?",
      "replies": [
        {
          "id": 1186902,
          "postDate": "2021-02-05T05:35:13.527Z",
          "content": "<p>Sorry, I ran out of resources and did not verify the LB. I will do it next week.</p>",
          "rawMarkdown": "Sorry, I ran out of resources and did not verify the LB. I will do it next week."
        }
      ]
    },
    {
      "id": 1186765,
      "postDate": "2021-02-05T03:01:12.770Z",
      "content": "<p>Bi-Tempered loss was not good as CategoryLoss, no matter how I try to fine tuning its parameter. In my setting, the best one is t1:0.8, t2:1.2.</p>",
      "rawMarkdown": "Bi-Tempered loss was not good as CategoryLoss, no matter how I try to fine tuning its parameter. In my setting, the best one is t1:0.8, t2:1.2.",
      "replies": [
        {
          "id": 1186899,
          "postDate": "2021-02-05T05:32:46.437Z",
          "content": "<p>In my case, the val_score of \"Efficientnet-b3 + Bi-Tempered Logistic\" was better than that of \"Efficientnet-b3  + cross entropy\".<br>\nHowever, I needed to tune the parameters as well as \"t1=0.4, t2=2.0\" and so on.</p>\n<table>\n<thead>\n<tr>\n<th>loss function</th>\n<th>val_score</th>\n<th>t1</th>\n<th>t2</th>\n<th>label_smooting</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Bi-Tempered Logistic</td>\n<td>0.897</td>\n<td>0.4</td>\n<td>2.0</td>\n<td>0.0</td>\n</tr>\n<tr>\n<td>cross entropy</td>\n<td>0.893</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>",
          "rawMarkdown": "In my case, the val_score of \"Efficientnet-b3 + Bi-Tempered Logistic\" was better than that of \"Efficientnet-b3  + cross entropy\".\nHowever, I needed to tune the parameters as well as \"t1=0.4, t2=2.0\" and so on.\n\n| loss function | val_score | t1| t2 | label_smooting |\n| --- | --- | --- | --- | --- | \n| Bi-Tempered Logistic |  0.897 |0.4|2.0|0.0|\n|cross entropy|0.893| - | - | - |"
        }
      ]
    },
    {
      "id": 1186667,
      "postDate": "2021-02-05T01:21:40.733Z",
      "content": "<p>i landed on 0.6 and 1.8.  moved them around a bit, in increments of 0.2, but never got better than those.<br>\ni probably should do .05 increments to see if it can be tuned a little better, but we'll see.</p>",
      "rawMarkdown": "i landed on 0.6 and 1.8.  moved them around a bit, in increments of 0.2, but never got better than those.\ni probably should do .05 increments to see if it can be tuned a little better, but we'll see.",
      "replies": [
        {
          "id": 1186671,
          "postDate": "2021-02-05T01:26:13.530Z",
          "content": "<p>Thank you. Do you include label_smoothing in that result?</p>",
          "rawMarkdown": "Thank you. Do you include label_smoothing in that result?"
        },
        {
          "id": 1186679,
          "postDate": "2021-02-05T01:42:08.020Z",
          "content": "<p>yes, 0.2.<br>\ni'm not sure i fully understand it enough to say if that is ideal, it's just what was in the notebook i copied some weeks ago.<br>\ni'll do some runs with different values for smoothing and see if it has an impact.<br>\ncurrently using a technique to just use 10% of the dataset to do some sensitivity runs.  the training only takes about 20 minutes.</p>",
          "rawMarkdown": "yes, 0.2.\ni'm not sure i fully understand it enough to say if that is ideal, it's just what was in the notebook i copied some weeks ago.\ni'll do some runs with different values for smoothing and see if it has an impact.\ncurrently using a technique to just use 10% of the dataset to do some sensitivity runs.  the training only takes about 20 minutes.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1186669,
      "postDate": "2021-02-05T01:25:44.757Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1186865,
      "author_name": "LeoF",
      "author_url": "",
      "post_date": "2021-02-05T05:02:37.533000",
      "content": "<p>What is the feedback of the LB?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1186902,
          "author_name": "yahhogogo",
          "author_url": "",
          "post_date": "2021-02-05T05:35:13.527000",
          "content": "<p>Sorry, I ran out of resources and did not verify the LB. I will do it next week.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1186765,
      "author_name": "Matrix",
      "author_url": "",
      "post_date": "2021-02-05T03:01:12.770000",
      "content": "<p>Bi-Tempered loss was not good as CategoryLoss, no matter how I try to fine tuning its parameter. In my setting, the best one is t1:0.8, t2:1.2.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1186899,
          "author_name": "yahhogogo",
          "author_url": "",
          "post_date": "2021-02-05T05:32:46.437000",
          "content": "<p>In my case, the val_score of \"Efficientnet-b3 + Bi-Tempered Logistic\" was better than that of \"Efficientnet-b3  + cross entropy\".<br>\nHowever, I needed to tune the parameters as well as \"t1=0.4, t2=2.0\" and so on.</p>\n<table>\n<thead>\n<tr>\n<th>loss function</th>\n<th>val_score</th>\n<th>t1</th>\n<th>t2</th>\n<th>label_smooting</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Bi-Tempered Logistic</td>\n<td>0.897</td>\n<td>0.4</td>\n<td>2.0</td>\n<td>0.0</td>\n</tr>\n<tr>\n<td>cross entropy</td>\n<td>0.893</td>\n<td>-</td>\n<td>-</td>\n<td>-</td>\n</tr>\n</tbody>\n</table>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1186667,
      "author_name": "DaveCCampbell",
      "author_url": "",
      "post_date": "2021-02-05T01:21:40.733000",
      "content": "<p>i landed on 0.6 and 1.8.  moved them around a bit, in increments of 0.2, but never got better than those.<br>\ni probably should do .05 increments to see if it can be tuned a little better, but we'll see.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1186671,
          "author_name": "yahhogogo",
          "author_url": "",
          "post_date": "2021-02-05T01:26:13.530000",
          "content": "<p>Thank you. Do you include label_smoothing in that result?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1186679,
          "author_name": "DaveCCampbell",
          "author_url": "",
          "post_date": "2021-02-05T01:42:08.020000",
          "content": "<p>yes, 0.2.<br>\ni'm not sure i fully understand it enough to say if that is ideal, it's just what was in the notebook i copied some weeks ago.<br>\ni'll do some runs with different values for smoothing and see if it has an impact.<br>\ncurrently using a technique to just use 10% of the dataset to do some sensitivity runs.  the training only takes about 20 minutes.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1186669,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-02-05T01:25:44.757000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1186644": "I implemented Bi-Tempered Logistic function with Efficientnetb3.\n(here is notebook : https://www.kaggle.com/ryohatano/efficientnet-and-bi-tempered-w-pytorch-lightning)\nThe results i tried with some parameter sets of t1 and t2 are bellow.\n\n![](https://github.com/haryuu/files_upload/blob/main/2021-02-05095028.png?raw=true)\nNote that the tag of label_smoothing you could see blank is meaning to label_smoothing=0.0.",
    "1186865": "What is the feedback of the LB?",
    "1186765": "Bi-Tempered loss was not good as CategoryLoss, no matter how I try to fine tuning its parameter. In my setting, the best one is t1:0.8, t2:1.2.",
    "1186667": "i landed on 0.6 and 1.8.  moved them around a bit, in increments of 0.2, but never got better than those.\ni probably should do .05 increments to see if it can be tuned a little better, but we'll see.",
    "1186669": ""
  }
}