{
  "id": 243090,
  "title": "Verification of methods on leaderboard?",
  "url": "/competitions/semi-inat-2021/discussion/243090",
  "author_name": "",
  "post_date": "2021-06-01T05:55:14.024205600Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>simple_test_test team has set a good example for obeying the rules. So I am curious about whether we have a verification process for methods on the leaderboard. <br>\nFrom the Challenge Guidelines, we are only allowed to use training, validation, and unlabeled images. We are not allowed to use the test set for semi-supervised learning,  pretrained model other than 1k imagenet,  etc.  <br>\nSo I am curious about how do we keep the process fair and do we need to verify each teams' validity?</p>",
  "messages": [
    {
      "id": "1330830",
      "postDate": "06/01/2021 05:55:14",
      "content": "<p>simple_test_test team has set a good example for obeying the rules. So I am curious about whether we have a verification process for methods on the leaderboard. <br>\nFrom the Challenge Guidelines, we are only allowed to use training, validation, and unlabeled images. We are not allowed to use the test set for semi-supervised learning,  pretrained model other than 1k imagenet,  etc.  <br>\nSo I am curious about how do we keep the process fair and do we need to verify each teams' validity?</p>",
      "rawMarkdown": "simple_test_test team has set a good example for obeying the rules. So I am curious about whether we have a verification process for methods on the leaderboard. \nFrom the Challenge Guidelines, we are only allowed to use training, validation, and unlabeled images. We are not allowed to use the test set for semi-supervised learning,  pretrained model other than 1k imagenet,  etc.  \nSo I am curious about how do we keep the process fair and do we need to verify each teams' validity?",
      "votes": null
    },
    {
      "id": "1331780",
      "postDate": "06/01/2021 17:57:58",
      "content": "<p>Thanks for your comment. Since we did not ask competitors to upload their models, we cannot verify if they are following the rules or not. However, we ask every competitor to fill in the Google Form where solution and method should be provided:<br>\n<a href=\"https://forms.gle/RAuVEKjZv4z1WfhD6\" target=\"_blank\">https://forms.gle/RAuVEKjZv4z1WfhD6</a></p>",
      "rawMarkdown": "Thanks for your comment. Since we did not ask competitors to upload their models, we cannot verify if they are following the rules or not. However, we ask every competitor to fill in the Google Form where solution and method should be provided:\nhttps://forms.gle/RAuVEKjZv4z1WfhD6",
      "votes": null
    },
    {
      "id": "1333741",
      "postDate": "06/03/2021 03:02:02",
      "content": "<p>Thanks for your reply. Since this is an integrity game,  I think every team has obeyed the rules. </p>",
      "rawMarkdown": "Thanks for your reply. Since this is an integrity game,  I think every team has obeyed the rules.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1331780,
      "author_name": "jcfredsu",
      "author_url": "",
      "post_date": "06/01/2021 17:57:58",
      "content": "<p>Thanks for your comment. Since we did not ask competitors to upload their models, we cannot verify if they are following the rules or not. However, we ask every competitor to fill in the Google Form where solution and method should be provided:<br>\n<a href=\"https://forms.gle/RAuVEKjZv4z1WfhD6\" target=\"_blank\">https://forms.gle/RAuVEKjZv4z1WfhD6</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1333741,
          "author_name": "kaiwu123",
          "author_url": "",
          "post_date": "06/03/2021 03:02:02",
          "content": "<p>Thanks for your reply. Since this is an integrity game,  I think every team has obeyed the rules. </p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1330830": "simple_test_test team has set a good example for obeying the rules. So I am curious about whether we have a verification process for methods on the leaderboard. \nFrom the Challenge Guidelines, we are only allowed to use training, validation, and unlabeled images. We are not allowed to use the test set for semi-supervised learning,  pretrained model other than 1k imagenet,  etc.  \nSo I am curious about how do we keep the process fair and do we need to verify each teams' validity?",
    "1331780": "Thanks for your comment. Since we did not ask competitors to upload their models, we cannot verify if they are following the rules or not. However, we ask every competitor to fill in the Google Form where solution and method should be provided:\nhttps://forms.gle/RAuVEKjZv4z1WfhD6",
    "1333741": "Thanks for your reply. Since this is an integrity game,  I think every team has obeyed the rules."
  },
  "source": "meta"
}