{ "id": 235813, "title": "Hand Labeled Waypoints", "url": "/competitions/indoor-location-navigation/discussion/235813", "author_name": "", "post_date": "2021-05-01T11:04:07.504759100Z", "votes": 38, "comment_count": 21, "views": 0, "content": "

Our team is using manually generated waypoints for post-processing and I published the dataset.
\nIndoor Navigation - Hand Labeled Waypoints

", "messages": [ { "id": "1289778", "postDate": "05/01/2021 11:04:07", "content": "

Our team is using manually generated waypoints for post-processing and I published the dataset.
\nIndoor Navigation - Hand Labeled Waypoints

", "rawMarkdown": "Our team is using manually generated waypoints for post-processing and I published the dataset.\n[Indoor Navigation - Hand Labeled Waypoints](https://www.kaggle.com/saitodevel01/indoor-navigation-hand-labeled-waypoints)", "votes": null }, { "id": "1289839", "postDate": "05/01/2021 12:18:01", "content": "

Thanks for publishing!

\n

I was a bit worried that this might be close to \"hand labeling\" test data - can anyone comment on that? If you are just looking at the site plans and manually adding waypoints where they seem to be missing, that seems ok (?) (as long as you aren't manually hand labeling the test points?)

\n

[I'm pretty new to kaggle, so wasn't 100% sure about what would constitute \"hand labeling\" of test data in the rules]. Thanks!

", "rawMarkdown": "Thanks for publishing!\n\nI was a bit worried that this might be close to \"hand labeling\" test data - can anyone comment on that? If you are just looking at the site plans and manually adding waypoints where they seem to be missing, that seems ok (?) (as long as you aren't manually hand labeling the test points?)\n\n[I'm pretty new to kaggle, so wasn't 100% sure about what would constitute \"hand labeling\" of test data in the rules]. Thanks!", "votes": null }, { "id": "1289875", "postDate": "05/01/2021 12:54:17", "content": "

I'm worried about that too. I'm wondering if using the data in post-processing is likely to get essentially equivalent results to hand labeling the test waypoints. If this kind of data is allowed to be used, I think the competition will collapse…

\n

However, I think it will be acceptable, at least if it was generated by some algorithm or machine learning model

", "rawMarkdown": "I'm worried about that too. I'm wondering if using the data in post-processing is likely to get essentially equivalent results to hand labeling the test waypoints. If this kind of data is allowed to be used, I think the competition will collapse...\n\nHowever, I think it will be acceptable, at least if it was generated by some algorithm or machine learning model", "votes": null }, { "id": "1289876", "postDate": "05/01/2021 12:55:01", "content": "

Thanks akio for uploading the dataset.
\nOur team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:
\nSubmissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n

@juliaelliott @addisonhoward @yuanchaoshu
\n1) Is it allowed to use hand labelled possible grid points for maps?
\n2) If people use hand labelled possible grid points, does the dataset have to be uploaded?

\n

For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec

\n

I just want to check my understanding, thanks!

\n

EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).

", "rawMarkdown": "Thanks akio for uploading the dataset. \nOur team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:\n`Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.`\n\n@juliaelliott @addisonhoward @yuanchaoshu\n1) Is it allowed to use hand labelled possible grid points for maps?\n2) If people use hand labelled possible grid points, does the dataset have to be uploaded?\n\nFor example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec\n\nI just want to check my understanding, thanks!\n\nEDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).", "votes": null }, { "id": "1289894", "postDate": "05/01/2021 13:16:46", "content": "

This is the visualization of the same sample in the akio's dataset: https://imgur.com/a/U8ulxNy

", "rawMarkdown": "This is the visualization of the same sample in the akio's dataset: https://imgur.com/a/U8ulxNy", "votes": null }, { "id": "1289939", "postDate": "05/01/2021 13:57:12", "content": "

I'm also afraid that this can be equivalent to the direct test set labelling, which is against the rule. but personally I think the competition won't collapse anyway, because the prediction below 3 looks very good for me and hard to fix by human. I thought the competition could collapse by hand labelling when the score was around 4, though.

", "rawMarkdown": "I'm also afraid that this can be equivalent to the direct test set labelling, which is against the rule. but personally I think the competition won't collapse anyway, because the prediction below 3 looks very good for me and hard to fix by human. I thought the competition could collapse by hand labelling when the score was around 4, though.", "votes": null }, { "id": "1290420", "postDate": "05/02/2021 00:11:39", "content": "

Thank you for sharing @saitodevel01. As I was interested in similar point, it seems very useful.
\nWould you elaborate more about how you created this dataset?

\n

In a generate_grid_points and generate_individual_points functions, there are specific parameters for each site and floor.

\n
    generate_grid_points(\n    site  = '5d2709d403f801723c32bd39',\n    floor = 0,\n    top_left     = np.array((86.4, 135.0)),\n    top_right    = np.array((93.4, 133.7)),\n    bottom_left  = np.array((83.9, 117.3)),\n    bottom_right = np.array((88.7, 116.9)),\n    N_h = 3,\n    N_v = 10,\n    )\n
\n
    generate_individual_points(\n        site  = '5da1382d4db8ce0c98bbe92e',\n        floor = 1,\n        points = [\n            (126, 160),\n            (114.5, 162),\n            (114.5, 156),\n            (119.5, 156),\n            (119.5, 147.5),\n            (107, 157),\n            (107.5, 145.5),\n            (113.5, 145.5),\n        ])\n
\n

Where these values come from?
\nDid you define them based solely on train waypoints and floor maps (and not based on test data, LB score, or other external data)?

", "rawMarkdown": "Thank you for sharing @saitodevel01. As [I was interested in similar point](https://www.kaggle.com/c/indoor-location-navigation/discussion/235733), it seems very useful.\nWould you elaborate more about how you created this dataset?\n\nIn a generate_grid_points and generate_individual_points functions, there are specific parameters for each site and floor.\n```\n generate_grid_points(\n site = '5d2709d403f801723c32bd39',\n floor = 0,\n top_left = np.array((86.4, 135.0)),\n top_right = np.array((93.4, 133.7)),\n bottom_left = np.array((83.9, 117.3)),\n bottom_right = np.array((88.7, 116.9)),\n N_h = 3,\n N_v = 10,\n )\n```\n ```\n generate_individual_points(\n site = '5da1382d4db8ce0c98bbe92e',\n floor = 1,\n points = [\n (126, 160),\n (114.5, 162),\n (114.5, 156),\n (119.5, 156),\n (119.5, 147.5),\n (107, 157),\n (107.5, 145.5),\n (113.5, 145.5),\n ])\n```\n\nWhere these values come from?\nDid you define them based solely on train waypoints and floor maps (and not based on test data, LB score, or other external data)?", "votes": null }, { "id": "1290520", "postDate": "05/02/2021 04:49:47", "content": "

I manually added possible missing grid points around my predictions using floor plan. I'm also afraid that this can be equivalent to the direct test set labelling.

", "rawMarkdown": "I manually added possible missing grid points around my predictions using floor plan. I'm also afraid that this can be equivalent to the direct test set labelling.", "votes": null }, { "id": "1290541", "postDate": "05/02/2021 05:25:09", "content": "

Thank you for clarifying your method.
\nAdding new waypoints based on test predictions seems to be a bit dangerous if it is incorporated with snap-to-grid. It is practically similar to manually editing the test predictions.
\nI also think that adding new waypoints is not hand labeling of train set, as they are not the labels of train data actually. I am not sure it should be regarded as hand labeling of test set, but at least, new waypoints are the candidates of test labels that are not in train ones.

\n

As I am a novice, I am not sure about it. I am happy when anyone correct me if I am wrong.
\nI never meant to blame any teams using hand labeled waypoints.

", "rawMarkdown": "Thank you for clarifying your method.\nAdding new waypoints based on test predictions seems to be a bit dangerous if it is incorporated with [snap-to-grid](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing). It is practically similar to manually editing the test predictions. \nI also think that adding new waypoints is not hand labeling of train set, as they are not the labels of train data actually. I am not sure it should be regarded as hand labeling of test set, but at least, new waypoints are the candidates of test labels that are not in train ones.\n\nAs I am a novice, I am not sure about it. I am happy when anyone correct me if I am wrong. \nI never meant to blame any teams using hand labeled waypoints.", "votes": null }, { "id": "1290882", "postDate": "05/02/2021 14:00:14", "content": "

As @tomooinubushi said, adding waypoints based on test predictions seems to be somewhat gray, as it can be close to the hand labelling of the test set. However, as @chris62 says, I think manually adding missing waypoints by looking at the map is ok, because it doesn't use the information of the test set, so not the hand labelling of the test set. My teammate, Reza, who made extra_waypoints.csv, said he generated missing way_point for the coverage hole. So I'm going to use our extra_waypoints.csv if @juliaelliott @addisonhoward @yuanchaoshu doesn't prohibit the use of possible missing grid points. Maybe we will use the 2 versions (with/without extra_waypoints.csv) for the final submissions for the safety, though.

", "rawMarkdown": "As @tomooinubushi said, adding waypoints based on test predictions seems to be somewhat gray, as it can be close to the hand labelling of the test set. However, as @chris62 says, I think manually adding missing waypoints by looking at the map is ok, because it doesn't use the information of the test set, so not the hand labelling of the test set. My teammate, Reza, who made extra_waypoints.csv, said he generated missing way_point for the coverage hole. So I'm going to use our extra_waypoints.csv if @juliaelliott @addisonhoward @yuanchaoshu doesn't prohibit the use of possible missing grid points. Maybe we will use the 2 versions (with/without extra_waypoints.csv) for the final submissions for the safety, though.", "votes": null }, { "id": "1295892", "postDate": "05/06/2021 19:26:31", "content": "

Does anyone else have more thoughts about this, or has there been any official word from Kaggle about it?

", "rawMarkdown": "Does anyone else have more thoughts about this, or has there been any official word from Kaggle about it?", "votes": null }, { "id": "1296827", "postDate": "05/07/2021 14:18:36", "content": "

It is prohibited by the rules to hand label test data (this includes the public and private test set). However, it is permitted to relabel or handlabel the training set.

", "rawMarkdown": "It is prohibited by the rules to hand label test data (this includes the public and private test set). However, it is permitted to relabel or handlabel the training set.", "votes": null }, { "id": "1296978", "postDate": "05/07/2021 16:06:17", "content": "

@juliaelliott Thanks you for answering. I want to check one. Adding possible missing grid points by hand for post processing is permitted? (like @mamasinkgs show here)

", "rawMarkdown": "juliaelliott Thanks you for answering. I want to check one. Adding possible missing grid points by hand for post processing is permitted? (like @mamasinkgs show [here](https://www.kaggle.com/c/indoor-location-navigation/discussion/235813#1289894))", "votes": null }, { "id": "1296983", "postDate": "05/07/2021 16:14:46", "content": "

I\"m sorry, the link above didn't work.
\nWhat I mentioned is following thread.

\n
\n

Thanks akio for uploading the dataset.
\n Our team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:
\n Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n

@juliaelliott @addisonhoward @yuanchaoshu
\n 1) Is it allowed to use hand labelled possible grid points for maps?
\n 2) If people use hand labelled possible grid points, does the dataset have to be uploaded?

\n

For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec

\n

I just want to check my understanding, thanks!

\n

EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).

\n
", "rawMarkdown": "I\"m sorry, the link above didn't work.\nWhat I mentioned is following thread.\n\n> Thanks akio for uploading the dataset. \n> Our team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:\n> `Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.`\n> \n> @juliaelliott @addisonhoward @yuanchaoshu\n> 1) Is it allowed to use hand labelled possible grid points for maps?\n> 2) If people use hand labelled possible grid points, does the dataset have to be uploaded?\n> \n> For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec\n> \n> I just want to check my understanding, thanks!\n> \n> EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).", "votes": null }, { "id": "1297203", "postDate": "05/07/2021 19:40:29", "content": "

Yes, I'm sure all the teams are looking forward to the kaggle team and host ( @juliaelliott @addisonhoward @yuanchaoshu)'s clarification of the use of hand-labelled possible missing grid points for the postprocessing.

", "rawMarkdown": "Yes, I'm sure all the teams are looking forward to the kaggle team and host ( @juliaelliott @addisonhoward @yuanchaoshu)'s clarification of the use of hand-labelled possible missing grid points for the postprocessing.", "votes": null }, { "id": "1297306", "postDate": "05/07/2021 22:40:12", "content": "

The guiding general principle is this: \"Would this method still be applicable to a completely new, unseen test set?\" If the answer is YES, then it is allowable. If the answer is, NO then it is not allowable.

\n

So, in this case, hand labeling of grid points that would be used for Test traces would not be acceptable. An automated process to fill in grid points that would work for new locations would be acceptable.

", "rawMarkdown": "The guiding general principle is this: \"Would this method still be applicable to a completely new, unseen test set?\" If the answer is YES, then it is allowable. If the answer is, NO then it is not allowable.\n\nSo, in this case, *hand* labeling of grid points that would be used for Test traces would not be acceptable. An automated process to fill in grid points that would work for new locations would be acceptable.", "votes": null }, { "id": "1297320", "postDate": "05/07/2021 23:10:07", "content": "

@inversion Thank you for the clarification! Then, our team won't choose the sub that uses the hand-labelled missing grid points for the final submission.

", "rawMarkdown": "inversion Thank you for the clarification! Then, our team won't choose the sub that uses the hand-labelled missing grid points for the final submission.", "votes": null }, { "id": "1297321", "postDate": "05/07/2021 23:13:12", "content": "

Thank you @inversion for the clarification. We won't use submissions with hand labeled missing grid points either.

", "rawMarkdown": "Thank you @inversion for the clarification. We won't use submissions with hand labeled missing grid points either.", "votes": null }, { "id": "1297325", "postDate": "05/07/2021 23:21:07", "content": "

@inversion Thank you very much. I understand it. Our team will no longer use hand labeling waypoints.

", "rawMarkdown": "inversion Thank you very much. I understand it. Our team will no longer use hand labeling waypoints.", "votes": null }, { "id": "1297392", "postDate": "05/08/2021 01:23:02", "content": "

@inversion
\nThank you.
\nI think that in Saito's dataset, grid points generated by \"generate_grid_points\" is still OK because they are generated automatically, but points generated by \"generate_individual_points\" are NG because they are manually.
\nor am I wrong and both are NG?

", "rawMarkdown": "inversion\nThank you.\nI think that in Saito's dataset, grid points generated by \"generate_grid_points\" is still OK because they are generated automatically, but points generated by \"generate_individual_points\" are NG because they are manually.\nor am I wrong and both are NG?", "votes": null }, { "id": "1297491", "postDate": "05/08/2021 04:48:05", "content": "

Thank you very much @inversion.

", "rawMarkdown": "Thank you very much @inversion.", "votes": null }, { "id": "1297492", "postDate": "05/08/2021 04:48:27", "content": "

Our team has developed an automated process for generating additional grid points and generate_grid_points.py is no longer used.

", "rawMarkdown": "Our team has developed an automated process for generating additional grid points and generate_grid_points.py is no longer used.", "votes": null } ], "comments": [ { "id": 1289839, "author_name": "chris62", "author_url": "", "post_date": "05/01/2021 12:18:01", "content": "

Thanks for publishing!

\n

I was a bit worried that this might be close to \"hand labeling\" test data - can anyone comment on that? If you are just looking at the site plans and manually adding waypoints where they seem to be missing, that seems ok (?) (as long as you aren't manually hand labeling the test points?)

\n

[I'm pretty new to kaggle, so wasn't 100% sure about what would constitute \"hand labeling\" of test data in the rules]. Thanks!

", "votes": null, "replies": [ { "id": 1290882, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/02/2021 14:00:14", "content": "

As @tomooinubushi said, adding waypoints based on test predictions seems to be somewhat gray, as it can be close to the hand labelling of the test set. However, as @chris62 says, I think manually adding missing waypoints by looking at the map is ok, because it doesn't use the information of the test set, so not the hand labelling of the test set. My teammate, Reza, who made extra_waypoints.csv, said he generated missing way_point for the coverage hole. So I'm going to use our extra_waypoints.csv if @juliaelliott @addisonhoward @yuanchaoshu doesn't prohibit the use of possible missing grid points. Maybe we will use the 2 versions (with/without extra_waypoints.csv) for the final submissions for the safety, though.

", "votes": null, "replies": [] } ] }, { "id": 1289875, "author_name": "yamsam", "author_url": "", "post_date": "05/01/2021 12:54:17", "content": "

I'm worried about that too. I'm wondering if using the data in post-processing is likely to get essentially equivalent results to hand labeling the test waypoints. If this kind of data is allowed to be used, I think the competition will collapse…

\n

However, I think it will be acceptable, at least if it was generated by some algorithm or machine learning model

", "votes": null, "replies": [ { "id": 1289939, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/01/2021 13:57:12", "content": "

I'm also afraid that this can be equivalent to the direct test set labelling, which is against the rule. but personally I think the competition won't collapse anyway, because the prediction below 3 looks very good for me and hard to fix by human. I thought the competition could collapse by hand labelling when the score was around 4, though.

", "votes": null, "replies": [] } ] }, { "id": 1289876, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/01/2021 12:55:01", "content": "

Thanks akio for uploading the dataset.
\nOur team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:
\nSubmissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n

@juliaelliott @addisonhoward @yuanchaoshu
\n1) Is it allowed to use hand labelled possible grid points for maps?
\n2) If people use hand labelled possible grid points, does the dataset have to be uploaded?

\n

For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec

\n

I just want to check my understanding, thanks!

\n

EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).

", "votes": null, "replies": [ { "id": 1289894, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/01/2021 13:16:46", "content": "

This is the visualization of the same sample in the akio's dataset: https://imgur.com/a/U8ulxNy

", "votes": null, "replies": [] } ] }, { "id": 1290420, "author_name": "tomooinubushi", "author_url": "", "post_date": "05/02/2021 00:11:39", "content": "

Thank you for sharing @saitodevel01. As I was interested in similar point, it seems very useful.
\nWould you elaborate more about how you created this dataset?

\n

In a generate_grid_points and generate_individual_points functions, there are specific parameters for each site and floor.

\n
    generate_grid_points(\n    site  = '5d2709d403f801723c32bd39',\n    floor = 0,\n    top_left     = np.array((86.4, 135.0)),\n    top_right    = np.array((93.4, 133.7)),\n    bottom_left  = np.array((83.9, 117.3)),\n    bottom_right = np.array((88.7, 116.9)),\n    N_h = 3,\n    N_v = 10,\n    )\n
\n
    generate_individual_points(\n        site  = '5da1382d4db8ce0c98bbe92e',\n        floor = 1,\n        points = [\n            (126, 160),\n            (114.5, 162),\n            (114.5, 156),\n            (119.5, 156),\n            (119.5, 147.5),\n            (107, 157),\n            (107.5, 145.5),\n            (113.5, 145.5),\n        ])\n
\n

Where these values come from?
\nDid you define them based solely on train waypoints and floor maps (and not based on test data, LB score, or other external data)?

", "votes": null, "replies": [] }, { "id": 1290520, "author_name": "saitodevel01", "author_url": "", "post_date": "05/02/2021 04:49:47", "content": "

I manually added possible missing grid points around my predictions using floor plan. I'm also afraid that this can be equivalent to the direct test set labelling.

", "votes": null, "replies": [ { "id": 1290541, "author_name": "tomooinubushi", "author_url": "", "post_date": "05/02/2021 05:25:09", "content": "

Thank you for clarifying your method.
\nAdding new waypoints based on test predictions seems to be a bit dangerous if it is incorporated with snap-to-grid. It is practically similar to manually editing the test predictions.
\nI also think that adding new waypoints is not hand labeling of train set, as they are not the labels of train data actually. I am not sure it should be regarded as hand labeling of test set, but at least, new waypoints are the candidates of test labels that are not in train ones.

\n

As I am a novice, I am not sure about it. I am happy when anyone correct me if I am wrong.
\nI never meant to blame any teams using hand labeled waypoints.

", "votes": null, "replies": [] } ] }, { "id": 1295892, "author_name": "chris62", "author_url": "", "post_date": "05/06/2021 19:26:31", "content": "

Does anyone else have more thoughts about this, or has there been any official word from Kaggle about it?

", "votes": null, "replies": [] }, { "id": 1296827, "author_name": "juliaelliott", "author_url": "", "post_date": "05/07/2021 14:18:36", "content": "

It is prohibited by the rules to hand label test data (this includes the public and private test set). However, it is permitted to relabel or handlabel the training set.

", "votes": null, "replies": [ { "id": 1296978, "author_name": "kuto0633", "author_url": "", "post_date": "05/07/2021 16:06:17", "content": "

@juliaelliott Thanks you for answering. I want to check one. Adding possible missing grid points by hand for post processing is permitted? (like @mamasinkgs show here)

", "votes": null, "replies": [] }, { "id": 1296983, "author_name": "kuto0633", "author_url": "", "post_date": "05/07/2021 16:14:46", "content": "

I\"m sorry, the link above didn't work.
\nWhat I mentioned is following thread.

\n
\n

Thanks akio for uploading the dataset.
\n Our team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:
\n Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.

\n

@juliaelliott @addisonhoward @yuanchaoshu
\n 1) Is it allowed to use hand labelled possible grid points for maps?
\n 2) If people use hand labelled possible grid points, does the dataset have to be uploaded?

\n

For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec

\n

I just want to check my understanding, thanks!

\n

EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).

\n
", "votes": null, "replies": [] }, { "id": 1297203, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/07/2021 19:40:29", "content": "

Yes, I'm sure all the teams are looking forward to the kaggle team and host ( @juliaelliott @addisonhoward @yuanchaoshu)'s clarification of the use of hand-labelled possible missing grid points for the postprocessing.

", "votes": null, "replies": [] }, { "id": 1297306, "author_name": "inversion", "author_url": "", "post_date": "05/07/2021 22:40:12", "content": "

The guiding general principle is this: \"Would this method still be applicable to a completely new, unseen test set?\" If the answer is YES, then it is allowable. If the answer is, NO then it is not allowable.

\n

So, in this case, hand labeling of grid points that would be used for Test traces would not be acceptable. An automated process to fill in grid points that would work for new locations would be acceptable.

", "votes": null, "replies": [] }, { "id": 1297320, "author_name": "mamasinkgs", "author_url": "", "post_date": "05/07/2021 23:10:07", "content": "

@inversion Thank you for the clarification! Then, our team won't choose the sub that uses the hand-labelled missing grid points for the final submission.

", "votes": null, "replies": [] }, { "id": 1297321, "author_name": "higepon", "author_url": "", "post_date": "05/07/2021 23:13:12", "content": "

Thank you @inversion for the clarification. We won't use submissions with hand labeled missing grid points either.

", "votes": null, "replies": [] }, { "id": 1297325, "author_name": "kuto0633", "author_url": "", "post_date": "05/07/2021 23:21:07", "content": "

@inversion Thank you very much. I understand it. Our team will no longer use hand labeling waypoints.

", "votes": null, "replies": [] }, { "id": 1297392, "author_name": "iwatatakuya", "author_url": "", "post_date": "05/08/2021 01:23:02", "content": "

@inversion
\nThank you.
\nI think that in Saito's dataset, grid points generated by \"generate_grid_points\" is still OK because they are generated automatically, but points generated by \"generate_individual_points\" are NG because they are manually.
\nor am I wrong and both are NG?

", "votes": null, "replies": [] }, { "id": 1297491, "author_name": "saitodevel01", "author_url": "", "post_date": "05/08/2021 04:48:05", "content": "

Thank you very much @inversion.

", "votes": null, "replies": [] }, { "id": 1297492, "author_name": "saitodevel01", "author_url": "", "post_date": "05/08/2021 04:48:27", "content": "

Our team has developed an automated process for generating additional grid points and generate_grid_points.py is no longer used.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1289778": "Our team is using manually generated waypoints for post-processing and I published the dataset.\n[Indoor Navigation - Hand Labeled Waypoints](https://www.kaggle.com/saitodevel01/indoor-navigation-hand-labeled-waypoints)", "1289839": "Thanks for publishing!\n\nI was a bit worried that this might be close to \"hand labeling\" test data - can anyone comment on that? If you are just looking at the site plans and manually adding waypoints where they seem to be missing, that seems ok (?) (as long as you aren't manually hand labeling the test points?)\n\n[I'm pretty new to kaggle, so wasn't 100% sure about what would constitute \"hand labeling\" of test data in the rules]. Thanks!", "1289875": "I'm worried about that too. I'm wondering if using the data in post-processing is likely to get essentially equivalent results to hand labeling the test waypoints. If this kind of data is allowed to be used, I think the competition will collapse...\n\nHowever, I think it will be acceptable, at least if it was generated by some algorithm or machine learning model", "1289876": "Thanks akio for uploading the dataset. \nOur team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:\n`Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.`\n\n@juliaelliott @addisonhoward @yuanchaoshu\n1) Is it allowed to use hand labelled possible grid points for maps?\n2) If people use hand labelled possible grid points, does the dataset have to be uploaded?\n\nFor example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec\n\nI just want to check my understanding, thanks!\n\nEDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).", "1289894": "This is the visualization of the same sample in the akio's dataset: https://imgur.com/a/U8ulxNy", "1289939": "I'm also afraid that this can be equivalent to the direct test set labelling, which is against the rule. but personally I think the competition won't collapse anyway, because the prediction below 3 looks very good for me and hard to fix by human. I thought the competition could collapse by hand labelling when the score was around 4, though.", "1290420": "Thank you for sharing @saitodevel01. As [I was interested in similar point](https://www.kaggle.com/c/indoor-location-navigation/discussion/235733), it seems very useful.\nWould you elaborate more about how you created this dataset?\n\nIn a generate_grid_points and generate_individual_points functions, there are specific parameters for each site and floor.\n```\n generate_grid_points(\n site = '5d2709d403f801723c32bd39',\n floor = 0,\n top_left = np.array((86.4, 135.0)),\n top_right = np.array((93.4, 133.7)),\n bottom_left = np.array((83.9, 117.3)),\n bottom_right = np.array((88.7, 116.9)),\n N_h = 3,\n N_v = 10,\n )\n```\n ```\n generate_individual_points(\n site = '5da1382d4db8ce0c98bbe92e',\n floor = 1,\n points = [\n (126, 160),\n (114.5, 162),\n (114.5, 156),\n (119.5, 156),\n (119.5, 147.5),\n (107, 157),\n (107.5, 145.5),\n (113.5, 145.5),\n ])\n```\n\nWhere these values come from?\nDid you define them based solely on train waypoints and floor maps (and not based on test data, LB score, or other external data)?", "1290520": "I manually added possible missing grid points around my predictions using floor plan. I'm also afraid that this can be equivalent to the direct test set labelling.", "1290541": "Thank you for clarifying your method.\nAdding new waypoints based on test predictions seems to be a bit dangerous if it is incorporated with [snap-to-grid](https://www.kaggle.com/robikscube/indoor-navigation-snap-to-grid-post-processing). It is practically similar to manually editing the test predictions. \nI also think that adding new waypoints is not hand labeling of train set, as they are not the labels of train data actually. I am not sure it should be regarded as hand labeling of test set, but at least, new waypoints are the candidates of test labels that are not in train ones.\n\nAs I am a novice, I am not sure about it. I am happy when anyone correct me if I am wrong. \nI never meant to blame any teams using hand labeled waypoints.", "1290882": "As @tomooinubushi said, adding waypoints based on test predictions seems to be somewhat gray, as it can be close to the hand labelling of the test set. However, as @chris62 says, I think manually adding missing waypoints by looking at the map is ok, because it doesn't use the information of the test set, so not the hand labelling of the test set. My teammate, Reza, who made extra_waypoints.csv, said he generated missing way_point for the coverage hole. So I'm going to use our extra_waypoints.csv if @juliaelliott @addisonhoward @yuanchaoshu doesn't prohibit the use of possible missing grid points. Maybe we will use the 2 versions (with/without extra_waypoints.csv) for the final submissions for the safety, though.", "1295892": "Does anyone else have more thoughts about this, or has there been any official word from Kaggle about it?", "1296827": "It is prohibited by the rules to hand label test data (this includes the public and private test set). However, it is permitted to relabel or handlabel the training set.", "1296978": "juliaelliott Thanks you for answering. I want to check one. Adding possible missing grid points by hand for post processing is permitted? (like @mamasinkgs show [here](https://www.kaggle.com/c/indoor-location-navigation/discussion/235813#1289894))", "1296983": "I\"m sorry, the link above didn't work.\nWhat I mentioned is following thread.\n\n> Thanks akio for uploading the dataset. \n> Our team is also using hand labeled possible (i.e. missing) grid points for the maps and it gave 0.04 improvement in postprocessing. In my understanding, hand labelling on the training dataset is allowed (e.g. https://www.kaggle.com/c/rfcx-species-audio-detection/discussion/220522 ). It should be noted that the rainforest competition has the same sentence about the hand-labelling rule:\n> `Submissions may not use or incorporate information from hand labeling or human prediction of the validation dataset or test data records.`\n> \n> @juliaelliott @addisonhoward @yuanchaoshu\n> 1) Is it allowed to use hand labelled possible grid points for maps?\n> 2) If people use hand labelled possible grid points, does the dataset have to be uploaded?\n> \n> For example, the blue points are hand-labelled possible grid points, and the red points are the grid points on the training dataset. https://imgur.com/a/ISU3Dec\n> \n> I just want to check my understanding, thanks!\n> \n> EDIT: The improvement by hand-labelled possible missing grid point is much bigger now (5/7).", "1297203": "Yes, I'm sure all the teams are looking forward to the kaggle team and host ( @juliaelliott @addisonhoward @yuanchaoshu)'s clarification of the use of hand-labelled possible missing grid points for the postprocessing.", "1297306": "The guiding general principle is this: \"Would this method still be applicable to a completely new, unseen test set?\" If the answer is YES, then it is allowable. If the answer is, NO then it is not allowable.\n\nSo, in this case, *hand* labeling of grid points that would be used for Test traces would not be acceptable. An automated process to fill in grid points that would work for new locations would be acceptable.", "1297320": "inversion Thank you for the clarification! Then, our team won't choose the sub that uses the hand-labelled missing grid points for the final submission.", "1297321": "Thank you @inversion for the clarification. We won't use submissions with hand labeled missing grid points either.", "1297325": "inversion Thank you very much. I understand it. Our team will no longer use hand labeling waypoints.", "1297392": "inversion\nThank you.\nI think that in Saito's dataset, grid points generated by \"generate_grid_points\" is still OK because they are generated automatically, but points generated by \"generate_individual_points\" are NG because they are manually.\nor am I wrong and both are NG?", "1297491": "Thank you very much @inversion.", "1297492": "Our team has developed an automated process for generating additional grid points and generate_grid_points.py is no longer used." }, "source": "meta" }