{ "id": 381279, "title": "Weighted PCA for trajectory fit", "url": "/competitions/icecube-neutrinos-in-deep-ice/discussion/381279", "author_name": "", "post_date": "2023-01-25T23:58:04.438493900Z", "votes": 7, "comment_count": 3, "views": 0, "content": "

Like some people in the competition, I thought of the PCA for fitting the trajectory based on the active sensors.

\n

The problem with the PCA is that it does not include the intensity of the charge.
\nTo overcome the issue, it is possible to fit a Weighted PCA which will account for the weight of the sample.

\n

I found out this implementation of the WPCA:
\nhttps://github.com/jakevdp/wpca

\n

I tried it on a few samples, it seems to improve sometimes the output of the trajectory fit, see the image below for event_id 67:
\n\"\"

", "messages": [ { "id": "2115748", "postDate": "01/25/2023 23:58:04", "content": "

Like some people in the competition, I thought of the PCA for fitting the trajectory based on the active sensors.

\n

The problem with the PCA is that it does not include the intensity of the charge.
\nTo overcome the issue, it is possible to fit a Weighted PCA which will account for the weight of the sample.

\n

I found out this implementation of the WPCA:
\nhttps://github.com/jakevdp/wpca

\n

I tried it on a few samples, it seems to improve sometimes the output of the trajectory fit, see the image below for event_id 67:
\n\"\"

", "rawMarkdown": "Like some people in the competition, I thought of the PCA for fitting the trajectory based on the active sensors.\n\nThe problem with the PCA is that it does not include the intensity of the charge. \nTo overcome the issue, it is possible to fit a Weighted PCA which will account for the weight of the sample.\n\nI found out this implementation of the WPCA:\nhttps://github.com/jakevdp/wpca\n\nI tried it on a few samples, it seems to improve sometimes the output of the trajectory fit, see the image below for event_id 67:\n![](https://i.imgur.com/nuDcwA2.png)", "votes": null }, { "id": "2116565", "postDate": "01/26/2023 15:06:54", "content": "

PCA is also a method that incorporates eigen values… Without the right hardware, I reserve myself to sharing these agreements with you! What I see is a distinction between the blue and purple trajectories (where they start and end) that could be interpreted more feasibly with an additional column in the dataset showing covariance, or a model that is specific to those numbers.

", "rawMarkdown": "PCA is also a method that incorporates eigen values... Without the right hardware, I reserve myself to sharing these agreements with you! What I see is a distinction between the blue and purple trajectories (where they start and end) that could be interpreted more feasibly with an additional column in the dataset showing covariance, or a model that is specific to those numbers.", "votes": null }, { "id": "2170798", "postDate": "03/06/2023 09:41:00", "content": "

Hello @bowaka, thank you a lot for publishing this idea, but could you please explain how do you calculate the 'weights' parameter that is passed in the Weighted PCA fit method? And should the weights be calculated for each row or each cell of our input pulse matrix?

", "rawMarkdown": "Hello @bowaka, thank you a lot for publishing this idea, but could you please explain how do you calculate the 'weights' parameter that is passed in the Weighted PCA fit method? And should the weights be calculated for each row or each cell of our input pulse matrix?", "votes": null }, { "id": "2402349", "postDate": "08/22/2023 05:35:57", "content": "

Hi @bowaka - just to let you know, used Weighted PCA in the ICR Identifying Age Related Conditions competition and gave a mention to you and link to this post in Sources. Think it is quite a useful implementation and perhaps not so well known. Thanks for creating awareness of it.
\nHere in Icecubes Neutrinos found it did quite well, but did not manage to finish a solution with it in time. So always keep it in mind!

", "rawMarkdown": "Hi @bowaka - just to let you know, used Weighted PCA in the ICR Identifying Age Related Conditions competition and gave a mention to you and link to this post in Sources. Think it is quite a useful implementation and perhaps not so well known. Thanks for creating awareness of it. \nHere in Icecubes Neutrinos found it did quite well, but did not manage to finish a solution with it in time. So always keep it in mind!", "votes": null } ], "comments": [ { "id": 2116565, "author_name": "joedavnport", "author_url": "", "post_date": "01/26/2023 15:06:54", "content": "

PCA is also a method that incorporates eigen values… Without the right hardware, I reserve myself to sharing these agreements with you! What I see is a distinction between the blue and purple trajectories (where they start and end) that could be interpreted more feasibly with an additional column in the dataset showing covariance, or a model that is specific to those numbers.

", "votes": null, "replies": [] }, { "id": 2170798, "author_name": "averkovanika", "author_url": "", "post_date": "03/06/2023 09:41:00", "content": "

Hello @bowaka, thank you a lot for publishing this idea, but could you please explain how do you calculate the 'weights' parameter that is passed in the Weighted PCA fit method? And should the weights be calculated for each row or each cell of our input pulse matrix?

", "votes": null, "replies": [] }, { "id": 2402349, "author_name": "something4kag", "author_url": "", "post_date": "08/22/2023 05:35:57", "content": "

Hi @bowaka - just to let you know, used Weighted PCA in the ICR Identifying Age Related Conditions competition and gave a mention to you and link to this post in Sources. Think it is quite a useful implementation and perhaps not so well known. Thanks for creating awareness of it.
\nHere in Icecubes Neutrinos found it did quite well, but did not manage to finish a solution with it in time. So always keep it in mind!

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2115748": "Like some people in the competition, I thought of the PCA for fitting the trajectory based on the active sensors.\n\nThe problem with the PCA is that it does not include the intensity of the charge. \nTo overcome the issue, it is possible to fit a Weighted PCA which will account for the weight of the sample.\n\nI found out this implementation of the WPCA:\nhttps://github.com/jakevdp/wpca\n\nI tried it on a few samples, it seems to improve sometimes the output of the trajectory fit, see the image below for event_id 67:\n![](https://i.imgur.com/nuDcwA2.png)", "2116565": "PCA is also a method that incorporates eigen values... Without the right hardware, I reserve myself to sharing these agreements with you! What I see is a distinction between the blue and purple trajectories (where they start and end) that could be interpreted more feasibly with an additional column in the dataset showing covariance, or a model that is specific to those numbers.", "2170798": "Hello @bowaka, thank you a lot for publishing this idea, but could you please explain how do you calculate the 'weights' parameter that is passed in the Weighted PCA fit method? And should the weights be calculated for each row or each cell of our input pulse matrix?", "2402349": "Hi @bowaka - just to let you know, used Weighted PCA in the ICR Identifying Age Related Conditions competition and gave a mention to you and link to this post in Sources. Think it is quite a useful implementation and perhaps not so well known. Thanks for creating awareness of it. \nHere in Icecubes Neutrinos found it did quite well, but did not manage to finish a solution with it in time. So always keep it in mind!" }, "source": "meta" }