{
  "id": 172471,
  "title": "Baseline model",
  "url": "/competitions/landmark-retrieval-2020/discussion/172471",
  "author_name": "",
  "post_date": "2020-08-05T07:09:46.313144100Z",
  "votes": 8,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I sincerely hope that the baseline model has random predictions on private LB.  People who try hard to build a model should be rewarded more than those who just submit a csv file they haven't produced.  Even if their model is weaker than baseline.</p>",
  "messages": [
    {
      "id": "958844",
      "postDate": "08/05/2020 07:09:46",
      "content": "<p>I sincerely hope that the baseline model has random predictions on private LB.  People who try hard to build a model should be rewarded more than those who just submit a csv file they haven't produced.  Even if their model is weaker than baseline.</p>",
      "rawMarkdown": "I sincerely hope that the baseline model has random predictions on private LB.  People who try hard to build a model should be rewarded more than those who just submit a csv file they haven't produced.  Even if their model is weaker than baseline.",
      "votes": null
    },
    {
      "id": "959021",
      "postDate": "08/05/2020 09:39:20",
      "content": "<p>&gt; I sincerely hope that the baseline model has random predictions on private LB</p>\n\n<p>I don't think it will happen. Organizers need to weed out the forked baseline submissions from the LB. If I am not wrong it can be done easily with a checksum check on the submission.zip.\n<a href=\"/wcukierski\">@wcukierski</a> <a href=\"/andrefaraujo\">@andrefaraujo</a> </p>",
      "rawMarkdown": "&gt; I sincerely hope that the baseline model has random predictions on private LB\n\nI don't think it will happen. Organizers need to weed out the forked baseline submissions from the LB. If I am not wrong it can be done easily with a checksum check on the submission.zip.\n@wcukierski @andrefaraujo",
      "votes": null
    },
    {
      "id": "959141",
      "postDate": "08/05/2020 11:30:09",
      "content": "<p>Organizers can't do that as nothing in the rules prevent people from submitting the baseline.  Did I miss it?</p>",
      "rawMarkdown": "Organizers can't do that as nothing in the rules prevent people from submitting the baseline.  Did I miss it?",
      "votes": null
    },
    {
      "id": "959220",
      "postDate": "08/05/2020 12:25:23",
      "content": "<p>Baseline model seems legit if you get predictions for the training set. It is difficult to make a model do random predictions on similar dataset. But if it was overfit on the public test set, then you may see something similar to what you wish. (I don't think it was trained on public test set.)</p>",
      "rawMarkdown": "Baseline model seems legit if you get predictions for the training set. It is difficult to make a model do random predictions on similar dataset. But if it was overfit on the public test set, then you may see something similar to what you wish. (I don't think it was trained on public test set.)",
      "votes": null
    },
    {
      "id": "959226",
      "postDate": "08/05/2020 12:33:08",
      "content": "<p>They could have made constant prediction on private, but then people would know the public/private test split.  I know my wish isn't realistic.  I still wish it...</p>",
      "rawMarkdown": "They could have made constant prediction on private, but then people would know the public/private test split.  I know my wish isn't realistic.  I still wish it...",
      "votes": null
    },
    {
      "id": "959229",
      "postDate": "08/05/2020 12:34:28",
      "content": "<p>Someone thinks it is fine to remove submissions that are compliant with competition rules apparently (based on downvote I just got).  This is a dangerous slope.</p>",
      "rawMarkdown": "Someone thinks it is fine to remove submissions that are compliant with competition rules apparently (based on downvote I just got).  This is a dangerous slope.",
      "votes": null
    },
    {
      "id": "959318",
      "postDate": "08/05/2020 13:49:54",
      "content": "<p><a href=\"/cpmpml\">@cpmpml</a> the baseline does not produce predictions or csv file. It produces embeddings used in KNN.\nIt cannot produce constant embeddings for the test set.</p>\n\n<p>P.S: unless they have a long list of hardcoded file names in the image prep function !</p>",
      "rawMarkdown": "cpmpml the baseline does not produce predictions or csv file. It produces embeddings used in KNN.\nIt cannot produce constant embeddings for the test set.\n\nP.S: unless they have a long list of hardcoded file names in the image prep function !",
      "votes": null
    },
    {
      "id": "959777",
      "postDate": "08/05/2020 22:06:20",
      "content": "<p>I believe it is the same model that can be found in the DELG repository, which is mentioned in the paper.. The only difference is that they pruned the local features head</p>",
      "rawMarkdown": "I believe it is the same model that can be found in the DELG repository, which is mentioned in the paper.. The only difference is that they pruned the local features head",
      "votes": null
    },
    {
      "id": "959779",
      "postDate": "08/05/2020 22:07:41",
      "content": "<p>Tx, this is very interesting.</p>",
      "rawMarkdown": "Tx, this is very interesting.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 959021,
      "author_name": "mayukh18",
      "author_url": "",
      "post_date": "08/05/2020 09:39:20",
      "content": "<p>&gt; I sincerely hope that the baseline model has random predictions on private LB</p>\n\n<p>I don't think it will happen. Organizers need to weed out the forked baseline submissions from the LB. If I am not wrong it can be done easily with a checksum check on the submission.zip.\n<a href=\"/wcukierski\">@wcukierski</a> <a href=\"/andrefaraujo\">@andrefaraujo</a> </p>",
      "votes": null,
      "replies": [
        {
          "id": 959141,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/05/2020 11:30:09",
          "content": "<p>Organizers can't do that as nothing in the rules prevent people from submitting the baseline.  Did I miss it?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 959229,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/05/2020 12:34:28",
          "content": "<p>Someone thinks it is fine to remove submissions that are compliant with competition rules apparently (based on downvote I just got).  This is a dangerous slope.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 959220,
      "author_name": "aerdem4",
      "author_url": "",
      "post_date": "08/05/2020 12:25:23",
      "content": "<p>Baseline model seems legit if you get predictions for the training set. It is difficult to make a model do random predictions on similar dataset. But if it was overfit on the public test set, then you may see something similar to what you wish. (I don't think it was trained on public test set.)</p>",
      "votes": null,
      "replies": [
        {
          "id": 959226,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/05/2020 12:33:08",
          "content": "<p>They could have made constant prediction on private, but then people would know the public/private test split.  I know my wish isn't realistic.  I still wish it...</p>",
          "votes": null,
          "replies": [
            {
              "id": 959318,
              "author_name": "ogrellier",
              "author_url": "",
              "post_date": "08/05/2020 13:49:54",
              "content": "<p><a href=\"/cpmpml\">@cpmpml</a> the baseline does not produce predictions or csv file. It produces embeddings used in KNN.\nIt cannot produce constant embeddings for the test set.</p>\n\n<p>P.S: unless they have a long list of hardcoded file names in the image prep function !</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 959777,
          "author_name": "arc144",
          "author_url": "",
          "post_date": "08/05/2020 22:06:20",
          "content": "<p>I believe it is the same model that can be found in the DELG repository, which is mentioned in the paper.. The only difference is that they pruned the local features head</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 959779,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "08/05/2020 22:07:41",
          "content": "<p>Tx, this is very interesting.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "958844": "I sincerely hope that the baseline model has random predictions on private LB.  People who try hard to build a model should be rewarded more than those who just submit a csv file they haven't produced.  Even if their model is weaker than baseline.",
    "959021": "&gt; I sincerely hope that the baseline model has random predictions on private LB\n\nI don't think it will happen. Organizers need to weed out the forked baseline submissions from the LB. If I am not wrong it can be done easily with a checksum check on the submission.zip.\n@wcukierski @andrefaraujo",
    "959141": "Organizers can't do that as nothing in the rules prevent people from submitting the baseline.  Did I miss it?",
    "959220": "Baseline model seems legit if you get predictions for the training set. It is difficult to make a model do random predictions on similar dataset. But if it was overfit on the public test set, then you may see something similar to what you wish. (I don't think it was trained on public test set.)",
    "959226": "They could have made constant prediction on private, but then people would know the public/private test split.  I know my wish isn't realistic.  I still wish it...",
    "959229": "Someone thinks it is fine to remove submissions that are compliant with competition rules apparently (based on downvote I just got).  This is a dangerous slope.",
    "959318": "cpmpml the baseline does not produce predictions or csv file. It produces embeddings used in KNN.\nIt cannot produce constant embeddings for the test set.\n\nP.S: unless they have a long list of hardcoded file names in the image prep function !",
    "959777": "I believe it is the same model that can be found in the DELG repository, which is mentioned in the paper.. The only difference is that they pruned the local features head",
    "959779": "Tx, this is very interesting."
  },
  "source": "meta"
}