{
  "id": 388375,
  "title": "Am I understanding the evaluation correctly ?",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/388375",
  "author_name": "",
  "post_date": "2023-02-17T06:17:13.179707800Z",
  "votes": 4,
  "comment_count": 4,
  "views": 0,
  "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1265131%2Fdbdbcd5311679e0eb86ccd143dc8df33%2Fchrome_lEn30rC9o1.png?generation=1676614497643751&amp;alt=media\" alt=\"image\"></p>\n<p>Sorry, I am just starting the competition, and I cannot understand the evaluation metric correctly.<br>\nIf you see the image, if we just predict one for each session-question pair, we get an F1-score of 0.8, too high compared to what's there on the leaderboard. What am I missing ?</p>",
  "messages": [
    {
      "id": "2148092",
      "postDate": "02/17/2023 06:17:13",
      "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1265131%2Fdbdbcd5311679e0eb86ccd143dc8df33%2Fchrome_lEn30rC9o1.png?generation=1676614497643751&amp;alt=media\" alt=\"image\"></p>\n<p>Sorry, I am just starting the competition, and I cannot understand the evaluation metric correctly.<br>\nIf you see the image, if we just predict one for each session-question pair, we get an F1-score of 0.8, too high compared to what's there on the leaderboard. What am I missing ?</p>",
      "rawMarkdown": "![image](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1265131%2Fdbdbcd5311679e0eb86ccd143dc8df33%2Fchrome_lEn30rC9o1.png?generation=1676614497643751&alt=media)\n\nSorry, I am just starting the competition, and I cannot understand the evaluation metric correctly.\nIf you see the image, if we just predict one for each session-question pair, we get an F1-score of 0.8, too high compared to what's there on the leaderboard. What am I missing ?",
      "votes": null
    },
    {
      "id": "2148105",
      "postDate": "02/17/2023 06:23:27",
      "content": "<p>You should use f1_score(targets['correct'], np.ones(len(targets)), average='macro') for this competition.</p>",
      "rawMarkdown": "You should use f1_score(targets['correct'], np.ones(len(targets)), average='macro') for this competition.",
      "votes": null
    },
    {
      "id": "2148107",
      "postDate": "02/17/2023 06:24:33",
      "content": "<p>oh thanks <a href=\"https://www.kaggle.com/buumoo\" target=\"_blank\">@buumoo</a> , never realized it's macro averaged, my bad</p>",
      "rawMarkdown": "oh thanks @buumoo , never realized it's macro averaged, my bad",
      "votes": null
    },
    {
      "id": "2148110",
      "postDate": "02/17/2023 06:26:36",
      "content": "<p>see <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315\" target=\"_blank\">https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315</a></p>",
      "rawMarkdown": "see https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315",
      "votes": null
    },
    {
      "id": "2148417",
      "postDate": "02/17/2023 12:32:17",
      "content": "<p>You may find this interesting as well:<br>\n<a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677\" target=\"_blank\">https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677</a></p>",
      "rawMarkdown": "You may find this interesting as well:\nhttps://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2148105,
      "author_name": "buumoo",
      "author_url": "",
      "post_date": "02/17/2023 06:23:27",
      "content": "<p>You should use f1_score(targets['correct'], np.ones(len(targets)), average='macro') for this competition.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2148107,
          "author_name": "nikhilmishradev",
          "author_url": "",
          "post_date": "02/17/2023 06:24:33",
          "content": "<p>oh thanks <a href=\"https://www.kaggle.com/buumoo\" target=\"_blank\">@buumoo</a> , never realized it's macro averaged, my bad</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2148110,
      "author_name": "steubk",
      "author_url": "",
      "post_date": "02/17/2023 06:26:36",
      "content": "<p>see <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315\" target=\"_blank\">https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315</a></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2148417,
      "author_name": "mohammad2012191",
      "author_url": "",
      "post_date": "02/17/2023 12:32:17",
      "content": "<p>You may find this interesting as well:<br>\n<a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677\" target=\"_blank\">https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2148092": "![image](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1265131%2Fdbdbcd5311679e0eb86ccd143dc8df33%2Fchrome_lEn30rC9o1.png?generation=1676614497643751&alt=media)\n\nSorry, I am just starting the competition, and I cannot understand the evaluation metric correctly.\nIf you see the image, if we just predict one for each session-question pair, we get an F1-score of 0.8, too high compared to what's there on the leaderboard. What am I missing ?",
    "2148105": "You should use f1_score(targets['correct'], np.ones(len(targets)), average='macro') for this competition.",
    "2148107": "oh thanks @buumoo , never realized it's macro averaged, my bad",
    "2148110": "see https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384315",
    "2148417": "You may find this interesting as well:\nhttps://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/384677"
  },
  "source": "meta"
}