Our team is currently using only post processing to improve the accuracy.
\nWe have found that the order of post processing changes the accuracy significantly, so we share the results.
The post processing we used for validation is as follows.
\nFirst, I did outlier correction, removing device, Kalman filter, phones mean, and position shift in that order.
\nThe result was CV 4.201 and LB 5.807.
Next, I did outlier correction, Kalman filter, phones mean, removing device, and position shift in that order.
\nThe result was CV 3.948 and LB 5.532. This has improved the accuracy of both CV and LB.
In this competition, you can improve the accuracy by combining various post processing.
\nHowever, if the processing is not done in the correct order, the improvement may not be as great as expected.
This may be an obvious result, but I hope it is helpful.
\nThank you for reading.
Reference
\nOur team is currently using only post processing to improve the accuracy.
\nWe have found that the order of post processing changes the accuracy significantly, so we share the results.
The post processing we used for validation is as follows.
\nFirst, I did outlier correction, removing device, Kalman filter, phones mean, and position shift in that order.
\nThe result was CV 4.201 and LB 5.807.
Next, I did outlier correction, Kalman filter, phones mean, removing device, and position shift in that order.
\nThe result was CV 3.948 and LB 5.532. This has improved the accuracy of both CV and LB.
In this competition, you can improve the accuracy by combining various post processing.
\nHowever, if the processing is not done in the correct order, the improvement may not be as great as expected.
This may be an obvious result, but I hope it is helpful.
\nThank you for reading.
Reference
\n@columbia2131
\n\"phones mean\" outputs the same prediction for all phones,
\nso I thought that the subsequent \"removing device\" may not work.
Either way, your advice is very helpful!
\nI'd like to be careful about the order of post-processing.
Good topic! I did my post processing almost same order as you.
", "rawMarkdown": "Good topic! I did my post processing almost same order as you.", "votes": null }, { "id": "1343762", "postDate": "06/10/2021 12:25:45", "content": "Hi @columbia2131
\nI use the same order of post-processing.
\nBut in my case, 'Position shift' and 'Removing device' are not working🤔
Thank you for your comment, @dehokanta !
\nI'm sorry, but I don't know why they didn't work.
\nHere are the CV results for each process in my case.
outlier correction(CV: 5.288)
\n↓
\nKalman filter(CV: 4.583)
\n↓
\nphones mean(CV: 4.142)
\n↓
\nremoving device(CV: 3.992)
\n↓
\nposition shift(CV: 3.948)
The parameter a for position shift is 0.62.
\nI hope this helps you to solve the problem.
", "rawMarkdown": "Thank you for your comment, @dehokanta !\n\nI'm sorry, but I don't know why they didn't work.\nHere are the CV results for each process in my case.\n\noutlier correction(CV: 5.288)\n↓\nKalman filter(CV: 4.583)\n↓\nphones mean(CV: 4.142)\n↓\nremoving device(CV: 3.992)\n↓\nposition shift(CV: 3.948)\n\nThe parameter ***a*** for position shift is 0.62.\n\nI hope this helps you to solve the problem.", "votes": null }, { "id": "1343899", "postDate": "06/10/2021 14:03:36", "content": "Thanks for the comment and the great idea, @t88take !
\nIt seemed worth a try, and I did outlier correction, Kalman filter, removing device, phones mean, and position shift in this order.
\nHowever, the results weren't good(CV 4.210).
I think it makes sense, so maybe this implementation is my mistake.
", "rawMarkdown": "Thanks for the comment and the great idea, @t88take !\n\nIt seemed worth a try, and I did outlier correction, Kalman filter, removing device, phones mean, and position shift in this order.\nHowever, the results weren't good(CV 4.210).\n\nI think it makes sense, so maybe this implementation is my mistake.", "votes": null }, { "id": "1344236", "postDate": "06/10/2021 18:04:55", "content": "Thanks for sharing this information, very helpful!
", "rawMarkdown": "Thanks for sharing this information, very helpful!", "votes": null }, { "id": "1344289", "postDate": "06/10/2021 19:19:25", "content": "Thanks @columbia2131
\nI'm doing a process that I have not published before outlier correction, that may be the cause.
It would definitely be the cause. The main problem of mixing more advanced outlier correction methods is that sometimes they overlap each other, thus rendering some of the used methods impractical.
\nAlso from what I have observed some radical methods that give a big local CV increase arent guaranteed to perform well on the LB, at least in the some selected order.
", "rawMarkdown": "It would definitely be the cause. The main problem of mixing more advanced outlier correction methods is that sometimes they overlap each other, thus rendering some of the used methods impractical. \n\nAlso from what I have observed some radical methods that give a big local CV increase arent guaranteed to perform well on the LB, at least in the some selected order.", "votes": null }, { "id": "1344415", "postDate": "06/10/2021 22:56:05", "content": "I agree with @avtobusbratiev
\nThere are limits to post-processing, so it will be important to create a good baseline with machine learning.
", "rawMarkdown": "I agree with @avtobusbratiev \n\nThere are limits to post-processing, so it will be important to create a good baseline with machine learning.", "votes": null } ], "comments": [ { "id": 1343659, "author_name": "t88take", "author_url": "", "post_date": "06/10/2021 11:00:37", "content": "@columbia2131
\n\"phones mean\" outputs the same prediction for all phones,
\nso I thought that the subsequent \"removing device\" may not work.
Either way, your advice is very helpful!
\nI'd like to be careful about the order of post-processing.
Thanks for the comment and the great idea, @t88take !
\nIt seemed worth a try, and I did outlier correction, Kalman filter, removing device, phones mean, and position shift in this order.
\nHowever, the results weren't good(CV 4.210).
I think it makes sense, so maybe this implementation is my mistake.
", "votes": null, "replies": [] } ] }, { "id": 1343736, "author_name": "kuto0633", "author_url": "", "post_date": "06/10/2021 11:58:03", "content": "Good topic! I did my post processing almost same order as you.
", "votes": null, "replies": [] }, { "id": 1343762, "author_name": "dehokanta", "author_url": "", "post_date": "06/10/2021 12:25:45", "content": "Hi @columbia2131
\nI use the same order of post-processing.
\nBut in my case, 'Position shift' and 'Removing device' are not working🤔
Thank you for your comment, @dehokanta !
\nI'm sorry, but I don't know why they didn't work.
\nHere are the CV results for each process in my case.
outlier correction(CV: 5.288)
\n↓
\nKalman filter(CV: 4.583)
\n↓
\nphones mean(CV: 4.142)
\n↓
\nremoving device(CV: 3.992)
\n↓
\nposition shift(CV: 3.948)
The parameter a for position shift is 0.62.
\nI hope this helps you to solve the problem.
", "votes": null, "replies": [] }, { "id": 1344289, "author_name": "dehokanta", "author_url": "", "post_date": "06/10/2021 19:19:25", "content": "Thanks @columbia2131
\nI'm doing a process that I have not published before outlier correction, that may be the cause.
It would definitely be the cause. The main problem of mixing more advanced outlier correction methods is that sometimes they overlap each other, thus rendering some of the used methods impractical.
\nAlso from what I have observed some radical methods that give a big local CV increase arent guaranteed to perform well on the LB, at least in the some selected order.
", "votes": null, "replies": [] }, { "id": 1344415, "author_name": "dehokanta", "author_url": "", "post_date": "06/10/2021 22:56:05", "content": "I agree with @avtobusbratiev
\nThere are limits to post-processing, so it will be important to create a good baseline with machine learning.
", "votes": null, "replies": [] } ] }, { "id": 1344236, "author_name": "saurabhbagchi", "author_url": "", "post_date": "06/10/2021 18:04:55", "content": "Thanks for sharing this information, very helpful!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1343441": "Our team is currently using only post processing to improve the accuracy.\nWe have found that the order of post processing changes the accuracy significantly, so we share the results.\n\nThe post processing we used for validation is as follows.\n\n* [Demonstration of the Kalman filter](https://www.kaggle.com/emaerthin/demonstration-of-the-kalman-filter)\n* [post-processing by outlier correction](https://www.kaggle.com/dehokanta/baseline-post-processing-by-outlier-correction)\n* [GSDC phones mean prediction](https://www.kaggle.com/t88take/gsdc-phones-mean-prediction)\n* [device EDA & Interpolate by removing device[en,ja]](https://www.kaggle.com/columbia2131/device-eda-interpolate-by-removing-device-en-ja)\n* [GSDC: Position shift](https://www.kaggle.com/wrrosa/gsdc-position-shift)\n\nFirst, I did outlier correction, removing device, Kalman filter, phones mean, and position shift in that order.\nThe result was CV 4.201 and LB 5.807.\n\nNext, I did outlier correction, Kalman filter, phones mean, removing device, and position shift in that order.\nThe result was CV 3.948 and LB 5.532. This has improved the accuracy of both CV and LB.\n\nIn this competition, you can improve the accuracy by combining various post processing.\nHowever, if the processing is not done in the correct order, the improvement may not be as great as expected.\n\nThis may be an obvious result, but I hope it is helpful.\nThank you for reading.\n\n\n**Reference**\n\n* [Demonstration of the Kalman filter](https://www.kaggle.com/emaerthin/demonstration-of-the-kalman-filter) created by @emaerthin \n* [post-processing by outlier correction](https://www.kaggle.com/dehokanta/baseline-post-processing-by-outlier-correction) created by @dehokanta \n* [GSDC phones mean prediction](https://www.kaggle.com/t88take/gsdc-phones-mean-prediction) created by @t88take \n* [device EDA & Interpolate by removing device[en,ja]](https://www.kaggle.com/columbia2131/device-eda-interpolate-by-removing-device-en-ja) created by @columbia2131 \n* [GSDC: Position shift](https://www.kaggle.com/wrrosa/gsdc-position-shift) created by @wrrosa", "1343659": "columbia2131 \n\"phones mean\" outputs the same prediction for all phones, \nso I thought that the subsequent \"removing device\" may not work.\n\nEither way, your advice is very helpful!\nI'd like to be careful about the order of post-processing.", "1343736": "Good topic! I did my post processing almost same order as you.", "1343762": "Hi @columbia2131 \nI use the same order of post-processing.\nBut in my case, 'Position shift' and 'Removing device' are not working🤔", "1343878": "Thank you for your comment, @dehokanta !\n\nI'm sorry, but I don't know why they didn't work.\nHere are the CV results for each process in my case.\n\noutlier correction(CV: 5.288)\n↓\nKalman filter(CV: 4.583)\n↓\nphones mean(CV: 4.142)\n↓\nremoving device(CV: 3.992)\n↓\nposition shift(CV: 3.948)\n\nThe parameter ***a*** for position shift is 0.62.\n\nI hope this helps you to solve the problem.", "1343899": "Thanks for the comment and the great idea, @t88take !\n\nIt seemed worth a try, and I did outlier correction, Kalman filter, removing device, phones mean, and position shift in this order.\nHowever, the results weren't good(CV 4.210).\n\nI think it makes sense, so maybe this implementation is my mistake.", "1344236": "Thanks for sharing this information, very helpful!", "1344289": "Thanks @columbia2131 \nI'm doing a process that I have not published before outlier correction, that may be the cause.", "1344389": "It would definitely be the cause. The main problem of mixing more advanced outlier correction methods is that sometimes they overlap each other, thus rendering some of the used methods impractical. \n\nAlso from what I have observed some radical methods that give a big local CV increase arent guaranteed to perform well on the LB, at least in the some selected order.", "1344415": "I agree with @avtobusbratiev \n\nThere are limits to post-processing, so it will be important to create a good baseline with machine learning." }, "source": "meta" }