{
  "id": 102348,
  "title": "CallBack Kappa Metric actually useful?",
  "url": "/competitions/aptos2019-blindness-detection/discussion/102348",
  "author_name": "",
  "post_date": "2019-08-01T14:58:57.539380300Z",
  "votes": 6,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hi everyone!</p>\n\n<p>I have seen that a lot of Kernels use a Callback function during training for the best Kappa score. \nI am not sure if this is actually a good idea. Obviously it's useful to prevent over fitting, but couldn't it be, that we predict our test  set then with a lucky punch during training, mostly based on the distribution of the valuation set?</p>\n\n<p>Might be some food for thought, but I'd love to be proven wrong :)</p>",
  "messages": [
    {
      "id": "589920",
      "postDate": "08/01/2019 14:58:57",
      "content": "<p>Hi everyone!</p>\n\n<p>I have seen that a lot of Kernels use a Callback function during training for the best Kappa score. \nI am not sure if this is actually a good idea. Obviously it's useful to prevent over fitting, but couldn't it be, that we predict our test  set then with a lucky punch during training, mostly based on the distribution of the valuation set?</p>\n\n<p>Might be some food for thought, but I'd love to be proven wrong :)</p>",
      "rawMarkdown": "Hi everyone!\n\nI have seen that a lot of Kernels use a Callback function during training for the best Kappa score. \nI am not sure if this is actually a good idea. Obviously it's useful to prevent over fitting, but couldn't it be, that we predict our test  set then with a lucky punch during training, mostly based on the distribution of the valuation set?\n\nMight be some food for thought, but I'd love to be proven wrong :)",
      "votes": null
    },
    {
      "id": "590879",
      "postDate": "08/02/2019 18:39:08",
      "content": "<p>Hi, \nKappa score is not stable metric, I would rather monitor Losss. Here is an excellent discussion and explanation on this topic <a href=\"https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878\">https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878</a> </p>\n\n<p>all the best =) </p>",
      "rawMarkdown": "Hi, \nKappa score is not stable metric, I would rather monitor Losss. Here is an excellent discussion and explanation on this topic https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878 \n\nall the best =)",
      "votes": null
    },
    {
      "id": "590901",
      "postDate": "08/02/2019 19:20:02",
      "content": "<p>Hey <a href=\"/drhabib\">@drhabib</a> thank you very much! that's what I thought</p>",
      "rawMarkdown": "Hey @drhabib thank you very much! that's what I thought",
      "votes": null
    },
    {
      "id": "591991",
      "postDate": "08/04/2019 15:39:03",
      "content": "<p>Just monitor both and see what works out best :)</p>",
      "rawMarkdown": "Just monitor both and see what works out best :)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 590879,
      "author_name": "drhabib",
      "author_url": "",
      "post_date": "08/02/2019 18:39:08",
      "content": "<p>Hi, \nKappa score is not stable metric, I would rather monitor Losss. Here is an excellent discussion and explanation on this topic <a href=\"https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878\">https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878</a> </p>\n\n<p>all the best =) </p>",
      "votes": null,
      "replies": [
        {
          "id": 590901,
          "author_name": "fanconic",
          "author_url": "",
          "post_date": "08/02/2019 19:20:02",
          "content": "<p>Hey <a href=\"/drhabib\">@drhabib</a> thank you very much! that's what I thought</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 591991,
      "author_name": "sidhanthholalkere",
      "author_url": "",
      "post_date": "08/04/2019 15:39:03",
      "content": "<p>Just monitor both and see what works out best :)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "589920": "Hi everyone!\n\nI have seen that a lot of Kernels use a Callback function during training for the best Kappa score. \nI am not sure if this is actually a good idea. Obviously it's useful to prevent over fitting, but couldn't it be, that we predict our test  set then with a lucky punch during training, mostly based on the distribution of the valuation set?\n\nMight be some food for thought, but I'd love to be proven wrong :)",
    "590879": "Hi, \nKappa score is not stable metric, I would rather monitor Losss. Here is an excellent discussion and explanation on this topic https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/97860#573878 \n\nall the best =)",
    "590901": "Hey @drhabib thank you very much! that's what I thought",
    "591991": "Just monitor both and see what works out best :)"
  },
  "source": "meta"
}