{
  "id": 250977,
  "title": "Public/Private data were sepaleted by collections",
  "url": "/competitions/google-smartphone-decimeter-challenge/discussion/250977",
  "author_name": "",
  "post_date": "2021-07-05T09:39:39.870703100Z",
  "votes": 30,
  "comment_count": 9,
  "views": 0,
  "content": "<p>In this competion, the leaderboard is calculated with approximately 53% of the test data and the final results will be based on the other 47%.</p>\n<p>so, I tried to check which ones corresponded to private LB data. <br>\nAs a result, it turns out that all the data in some collections are contained in private LB data.</p>\n<p>(updated 07/06)<br>\n<a href=\"https://www.kaggle.com/minomonter\" target=\"_blank\">@minomonter</a> and <a href=\"https://www.kaggle.com/t88take\" target=\"_blank\">@t88take</a>, Thanks for sharing!</p>\n<p><strong>Public:</strong><br>\n2020-05-28-US-MTV-2<br>\n2020-06-10-US-MTV-1<br>\n2020-06-10-US-MTV-2<br>\n2020-08-13-US-MTV-1<br>\n2021-03-16-US-MTV-2<br>\n2021-04-02-US-SJC-1<br>\n2021-04-21-US-MTV-1<br>\n2021-04-26-US-SVL-2<br>\n2021-04-28-US-MTV-2<br>\n2021-04-29-US-MTV-2<br>\n2021-04-29-US-SJC-3<br>\n48302 points, 52.8 %</p>\n<p><strong>Private:</strong><br>\n2020-05-15-US-MTV-1<br>\n2020-05-28-US-MTV-1<br>\n2020-06-04-US-MTV-2<br>\n2020-08-03-US-MTV-2<br>\n2021-03-16-US-RWC-2<br>\n2021-03-25-US-PAO-1<br>\n2021-04-08-US-MTV-1<br>\n2021-04-22-US-SJC-2<br>\n43184 points, 47.2 %</p>\n<h3>how to make submission to check</h3>\n<pre><code>target_collection = &lt;TARGET_COLLECTION_NAME&gt;\n\nsub_df = pd.read_csv('submission.csv')\nsub_df['collectionName'] = sub_df['phone'].str.split('_', expand=True)[0]\nsub_df.loc[sub_df['collectionName']==target_collection, 'latDeg'] = 0\nsub_df[['phone','millisSinceGpsEpoch','latDeg','lngDeg']].to_csv(f'{target_collection}.csv', index=False)\n</code></pre>\n<p>score is</p>\n<ul>\n<li>change -&gt; all(or some) data is public</li>\n<li>no change -&gt; all data is private</li>\n</ul>\n<p><br>\n-&gt;all the collections are neatly divided into Public and Private</p>\n<p><br>\n-&gt; COMPLETED</p>\n<p>If there are any mistakes, please teach me!</p>",
  "messages": [
    {
      "id": "1376730",
      "postDate": "07/05/2021 09:39:39",
      "content": "<p>In this competion, the leaderboard is calculated with approximately 53% of the test data and the final results will be based on the other 47%.</p>\n<p>so, I tried to check which ones corresponded to private LB data. <br>\nAs a result, it turns out that all the data in some collections are contained in private LB data.</p>\n<p>(updated 07/06)<br>\n<a href=\"https://www.kaggle.com/minomonter\" target=\"_blank\">@minomonter</a> and <a href=\"https://www.kaggle.com/t88take\" target=\"_blank\">@t88take</a>, Thanks for sharing!</p>\n<p><strong>Public:</strong><br>\n2020-05-28-US-MTV-2<br>\n2020-06-10-US-MTV-1<br>\n2020-06-10-US-MTV-2<br>\n2020-08-13-US-MTV-1<br>\n2021-03-16-US-MTV-2<br>\n2021-04-02-US-SJC-1<br>\n2021-04-21-US-MTV-1<br>\n2021-04-26-US-SVL-2<br>\n2021-04-28-US-MTV-2<br>\n2021-04-29-US-MTV-2<br>\n2021-04-29-US-SJC-3<br>\n48302 points, 52.8 %</p>\n<p><strong>Private:</strong><br>\n2020-05-15-US-MTV-1<br>\n2020-05-28-US-MTV-1<br>\n2020-06-04-US-MTV-2<br>\n2020-08-03-US-MTV-2<br>\n2021-03-16-US-RWC-2<br>\n2021-03-25-US-PAO-1<br>\n2021-04-08-US-MTV-1<br>\n2021-04-22-US-SJC-2<br>\n43184 points, 47.2 %</p>\n<h3>how to make submission to check</h3>\n<pre><code>target_collection = &lt;TARGET_COLLECTION_NAME&gt;\n\nsub_df = pd.read_csv('submission.csv')\nsub_df['collectionName'] = sub_df['phone'].str.split('_', expand=True)[0]\nsub_df.loc[sub_df['collectionName']==target_collection, 'latDeg'] = 0\nsub_df[['phone','millisSinceGpsEpoch','latDeg','lngDeg']].to_csv(f'{target_collection}.csv', index=False)\n</code></pre>\n<p>score is</p>\n<ul>\n<li>change -&gt; all(or some) data is public</li>\n<li>no change -&gt; all data is private</li>\n</ul>\n<p><br>\n-&gt;all the collections are neatly divided into Public and Private</p>\n<p><br>\n-&gt; COMPLETED</p>\n<p>If there are any mistakes, please teach me!</p>",
      "rawMarkdown": "In this competion, the leaderboard is calculated with approximately 53% of the test data and the final results will be based on the other 47%.\n\nso, I tried to check which ones corresponded to private LB data. \nAs a result, it turns out that all the data in some collections are contained in private LB data.\n\n(updated 07/06)\n@minomonter and @t88take, Thanks for sharing!\n\n**Public:**\n2020-05-28-US-MTV-2\n2020-06-10-US-MTV-1\n2020-06-10-US-MTV-2\n2020-08-13-US-MTV-1\n2021-03-16-US-MTV-2\n2021-04-02-US-SJC-1\n2021-04-21-US-MTV-1\n2021-04-26-US-SVL-2\n2021-04-28-US-MTV-2\n2021-04-29-US-MTV-2\n2021-04-29-US-SJC-3\n48302 points, 52.8 %\n\n**Private:**\n2020-05-15-US-MTV-1\n2020-05-28-US-MTV-1\n2020-06-04-US-MTV-2\n2020-08-03-US-MTV-2\n2021-03-16-US-RWC-2\n2021-03-25-US-PAO-1\n2021-04-08-US-MTV-1\n2021-04-22-US-SJC-2\n43184 points, 47.2 %\n\n\n### how to make submission to check\n```python\ntarget_collection = <TARGET_COLLECTION_NAME>\n\nsub_df = pd.read_csv('submission.csv')\nsub_df['collectionName'] = sub_df['phone'].str.split('_', expand=True)[0]\nsub_df.loc[sub_df['collectionName']==target_collection, 'latDeg'] = 0\nsub_df[['phone','millisSinceGpsEpoch','latDeg','lngDeg']].to_csv(f'{target_collection}.csv', index=False)\n```\n score is\n- change -> all(or some) data is public\n- no change -> all data is private\n\n~~Of course, I don't know if all the remaining collections are neatly divided into Public and Private, but it's worth checking.~~\n->all the collections are neatly divided into Public and Private\n\n~~As shown above, there are many collections that have not yet been checked.\nSo if anyone has already checked, or is willing to check, I would appreciate it if you could share the information in this discussion. ~~\n-> COMPLETED\n\nIf there are any mistakes, please teach me!",
      "votes": null
    },
    {
      "id": "1376760",
      "postDate": "07/05/2021 10:05:19",
      "content": "<p>Thanks for sharing!  <br>\nIt's surprised that there is a collection where all the data belongs to private.<br>\nI tried to use the method you shared for one collection.  <br>\n2021-04-02-US-SJC-1 : score is changed.</p>",
      "rawMarkdown": "Thanks for sharing!  \nIt's surprised that there is a collection where all the data belongs to private.\nI tried to use the method you shared for one collection.  \n2021-04-02-US-SJC-1 : score is changed.",
      "votes": null
    },
    {
      "id": "1376772",
      "postDate": "07/05/2021 10:17:41",
      "content": "<p>Are you sure? From what I have observed some pathes are divided in parts, that being first half of one of the MTV routes goes to private and another one goes to test.</p>",
      "rawMarkdown": "Are you sure? From what I have observed some pathes are divided in parts, that being first half of one of the MTV routes goes to private and another one goes to test.",
      "votes": null
    },
    {
      "id": "1376793",
      "postDate": "07/05/2021 10:40:30",
      "content": "<p>At least the collection whose score does not change by the above method can be considered private-LB data.</p>\n<p>As you pointed out, there seems to be a collection that is split into both public and private, so I updated some of the above text.<br>\nThanks for your info!</p>",
      "rawMarkdown": "At least the collection whose score does not change by the above method can be considered private-LB data.\n\nAs you pointed out, there seems to be a collection that is split into both public and private, so I updated some of the above text.\nThanks for your info!",
      "votes": null
    },
    {
      "id": "1376802",
      "postDate": "07/05/2021 10:58:27",
      "content": "<p>Thanks for your trying! <br>\nI'll update info later.</p>",
      "rawMarkdown": "Thanks for your trying! \nI'll update info later.",
      "votes": null
    },
    {
      "id": "1377517",
      "postDate": "07/05/2021 23:40:05",
      "content": "<p>We also agree that public/private are separated by collections. Here's what we've got:</p>\n<p>Public:<br>\n2020-05-28-US-MTV-2<br>\n2020-06-10-US-MTV-1<br>\n2020-06-10-US-MTV-2<br>\n2020-08-13-US-MTV-1<br>\n2021-03-16-US-MTV-2<br>\n2021-04-02-US-SJC-1<br>\n2021-04-21-US-MTV-1<br>\n2021-04-26-US-SVL-2<br>\n2021-04-28-US-MTV-2<br>\n2021-04-29-US-MTV-2<br>\n2021-04-29-US-SJC-3<br>\n48302 points, 52.8 %</p>\n<p>Private:<br>\n2020-05-15-US-MTV-1<br>\n2020-05-28-US-MTV-1<br>\n2020-06-04-US-MTV-2<br>\n2020-08-03-US-MTV-2<br>\n2021-03-16-US-RWC-2<br>\n2021-03-25-US-PAO-1<br>\n2021-04-08-US-MTV-1<br>\n2021-04-22-US-SJC-2<br>\n43184 points, 47.2 %</p>",
      "rawMarkdown": "We also agree that public/private are separated by collections. Here's what we've got:\n\nPublic:\n2020-05-28-US-MTV-2\n2020-06-10-US-MTV-1\n2020-06-10-US-MTV-2\n2020-08-13-US-MTV-1\n2021-03-16-US-MTV-2\n2021-04-02-US-SJC-1\n2021-04-21-US-MTV-1\n2021-04-26-US-SVL-2\n2021-04-28-US-MTV-2\n2021-04-29-US-MTV-2\n2021-04-29-US-SJC-3\n48302 points, 52.8 %\n\nPrivate:\n2020-05-15-US-MTV-1\n2020-05-28-US-MTV-1\n2020-06-04-US-MTV-2\n2020-08-03-US-MTV-2\n2021-03-16-US-RWC-2\n2021-03-25-US-PAO-1\n2021-04-08-US-MTV-1\n2021-04-22-US-SJC-2\n43184 points, 47.2 %",
      "votes": null
    },
    {
      "id": "1377519",
      "postDate": "07/05/2021 23:49:30",
      "content": "<p>Wow, thanks for sharing!</p>",
      "rawMarkdown": "Wow, thanks for sharing!",
      "votes": null
    },
    {
      "id": "1377534",
      "postDate": "07/06/2021 00:06:05",
      "content": "<p>I verified your information just to make sure it was correct, and it is correct.</p>\n<p>I'll update information based on your sharing, Thanks!</p>",
      "rawMarkdown": "I verified your information just to make sure it was correct, and it is correct.\n\n I'll update information based on your sharing, Thanks!",
      "votes": null
    },
    {
      "id": "1377663",
      "postDate": "07/06/2021 03:37:12",
      "content": "<p>Thank you for sharing!<br>\nIt means the last boss is sleeping in the private zone.<br>\n2021-04-22-US-SJC-2 😴</p>",
      "rawMarkdown": "Thank you for sharing!\nIt means the last boss is sleeping in the private zone.\n2021-04-22-US-SJC-2 😴",
      "votes": null
    },
    {
      "id": "1377690",
      "postDate": "07/06/2021 03:59:20",
      "content": "<p>Yes, Exactly!</p>",
      "rawMarkdown": "Yes, Exactly!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1376760,
      "author_name": "t88take",
      "author_url": "",
      "post_date": "07/05/2021 10:05:19",
      "content": "<p>Thanks for sharing!  <br>\nIt's surprised that there is a collection where all the data belongs to private.<br>\nI tried to use the method you shared for one collection.  <br>\n2021-04-02-US-SJC-1 : score is changed.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1376802,
          "author_name": "kuto0633",
          "author_url": "",
          "post_date": "07/05/2021 10:58:27",
          "content": "<p>Thanks for your trying! <br>\nI'll update info later.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1376772,
      "author_name": "avtobusbratiev",
      "author_url": "",
      "post_date": "07/05/2021 10:17:41",
      "content": "<p>Are you sure? From what I have observed some pathes are divided in parts, that being first half of one of the MTV routes goes to private and another one goes to test.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1376793,
          "author_name": "kuto0633",
          "author_url": "",
          "post_date": "07/05/2021 10:40:30",
          "content": "<p>At least the collection whose score does not change by the above method can be considered private-LB data.</p>\n<p>As you pointed out, there seems to be a collection that is split into both public and private, so I updated some of the above text.<br>\nThanks for your info!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1377517,
      "author_name": "minomonter",
      "author_url": "",
      "post_date": "07/05/2021 23:40:05",
      "content": "<p>We also agree that public/private are separated by collections. Here's what we've got:</p>\n<p>Public:<br>\n2020-05-28-US-MTV-2<br>\n2020-06-10-US-MTV-1<br>\n2020-06-10-US-MTV-2<br>\n2020-08-13-US-MTV-1<br>\n2021-03-16-US-MTV-2<br>\n2021-04-02-US-SJC-1<br>\n2021-04-21-US-MTV-1<br>\n2021-04-26-US-SVL-2<br>\n2021-04-28-US-MTV-2<br>\n2021-04-29-US-MTV-2<br>\n2021-04-29-US-SJC-3<br>\n48302 points, 52.8 %</p>\n<p>Private:<br>\n2020-05-15-US-MTV-1<br>\n2020-05-28-US-MTV-1<br>\n2020-06-04-US-MTV-2<br>\n2020-08-03-US-MTV-2<br>\n2021-03-16-US-RWC-2<br>\n2021-03-25-US-PAO-1<br>\n2021-04-08-US-MTV-1<br>\n2021-04-22-US-SJC-2<br>\n43184 points, 47.2 %</p>",
      "votes": null,
      "replies": [
        {
          "id": 1377519,
          "author_name": "kuto0633",
          "author_url": "",
          "post_date": "07/05/2021 23:49:30",
          "content": "<p>Wow, thanks for sharing!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1377534,
          "author_name": "kuto0633",
          "author_url": "",
          "post_date": "07/06/2021 00:06:05",
          "content": "<p>I verified your information just to make sure it was correct, and it is correct.</p>\n<p>I'll update information based on your sharing, Thanks!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1377663,
      "author_name": "museas",
      "author_url": "",
      "post_date": "07/06/2021 03:37:12",
      "content": "<p>Thank you for sharing!<br>\nIt means the last boss is sleeping in the private zone.<br>\n2021-04-22-US-SJC-2 😴</p>",
      "votes": null,
      "replies": [
        {
          "id": 1377690,
          "author_name": "kuto0633",
          "author_url": "",
          "post_date": "07/06/2021 03:59:20",
          "content": "<p>Yes, Exactly!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1376730": "In this competion, the leaderboard is calculated with approximately 53% of the test data and the final results will be based on the other 47%.\n\nso, I tried to check which ones corresponded to private LB data. \nAs a result, it turns out that all the data in some collections are contained in private LB data.\n\n(updated 07/06)\n@minomonter and @t88take, Thanks for sharing!\n\n**Public:**\n2020-05-28-US-MTV-2\n2020-06-10-US-MTV-1\n2020-06-10-US-MTV-2\n2020-08-13-US-MTV-1\n2021-03-16-US-MTV-2\n2021-04-02-US-SJC-1\n2021-04-21-US-MTV-1\n2021-04-26-US-SVL-2\n2021-04-28-US-MTV-2\n2021-04-29-US-MTV-2\n2021-04-29-US-SJC-3\n48302 points, 52.8 %\n\n**Private:**\n2020-05-15-US-MTV-1\n2020-05-28-US-MTV-1\n2020-06-04-US-MTV-2\n2020-08-03-US-MTV-2\n2021-03-16-US-RWC-2\n2021-03-25-US-PAO-1\n2021-04-08-US-MTV-1\n2021-04-22-US-SJC-2\n43184 points, 47.2 %\n\n\n### how to make submission to check\n```python\ntarget_collection = <TARGET_COLLECTION_NAME>\n\nsub_df = pd.read_csv('submission.csv')\nsub_df['collectionName'] = sub_df['phone'].str.split('_', expand=True)[0]\nsub_df.loc[sub_df['collectionName']==target_collection, 'latDeg'] = 0\nsub_df[['phone','millisSinceGpsEpoch','latDeg','lngDeg']].to_csv(f'{target_collection}.csv', index=False)\n```\n score is\n- change -> all(or some) data is public\n- no change -> all data is private\n\n~~Of course, I don't know if all the remaining collections are neatly divided into Public and Private, but it's worth checking.~~\n->all the collections are neatly divided into Public and Private\n\n~~As shown above, there are many collections that have not yet been checked.\nSo if anyone has already checked, or is willing to check, I would appreciate it if you could share the information in this discussion. ~~\n-> COMPLETED\n\nIf there are any mistakes, please teach me!",
    "1376760": "Thanks for sharing!  \nIt's surprised that there is a collection where all the data belongs to private.\nI tried to use the method you shared for one collection.  \n2021-04-02-US-SJC-1 : score is changed.",
    "1376772": "Are you sure? From what I have observed some pathes are divided in parts, that being first half of one of the MTV routes goes to private and another one goes to test.",
    "1376793": "At least the collection whose score does not change by the above method can be considered private-LB data.\n\nAs you pointed out, there seems to be a collection that is split into both public and private, so I updated some of the above text.\nThanks for your info!",
    "1376802": "Thanks for your trying! \nI'll update info later.",
    "1377517": "We also agree that public/private are separated by collections. Here's what we've got:\n\nPublic:\n2020-05-28-US-MTV-2\n2020-06-10-US-MTV-1\n2020-06-10-US-MTV-2\n2020-08-13-US-MTV-1\n2021-03-16-US-MTV-2\n2021-04-02-US-SJC-1\n2021-04-21-US-MTV-1\n2021-04-26-US-SVL-2\n2021-04-28-US-MTV-2\n2021-04-29-US-MTV-2\n2021-04-29-US-SJC-3\n48302 points, 52.8 %\n\nPrivate:\n2020-05-15-US-MTV-1\n2020-05-28-US-MTV-1\n2020-06-04-US-MTV-2\n2020-08-03-US-MTV-2\n2021-03-16-US-RWC-2\n2021-03-25-US-PAO-1\n2021-04-08-US-MTV-1\n2021-04-22-US-SJC-2\n43184 points, 47.2 %",
    "1377519": "Wow, thanks for sharing!",
    "1377534": "I verified your information just to make sure it was correct, and it is correct.\n\n I'll update information based on your sharing, Thanks!",
    "1377663": "Thank you for sharing!\nIt means the last boss is sleeping in the private zone.\n2021-04-22-US-SJC-2 😴",
    "1377690": "Yes, Exactly!"
  },
  "source": "meta"
}