{ "id": 403112, "title": "possible language-like [structure] in iceCube [Neutrino data]", "url": "/competitions/icecube-neutrinos-in-deep-ice/discussion/403112", "author_name": "", "post_date": "2023-04-21T07:57:02.876107600Z", "votes": 1, "comment_count": 3, "views": 0, "content": "

after multiple notebooks and the latest notebook ( after 77 versions ), found that not just gpt can predict the neutrino particle’s direction, but from the pattern how gpt predict the neutrino particle’s direction, leads to that the dataset from iceCube may contain language-like [structure].

\n

abstract [what]

\n

\"\"

\n

gpt is mainly for language model, to prediction next word in sequence. however, this notebook shows [datasets] from the iceCube Neutrino Observatory may contain language-like [structures], after using gpt to predict neutrino particle’s di![]

\n

detail of the notebook and output log:
\nhttps://www.kaggle.com/tyeestudio/language-from-outer-space-in-icecube-data

", "messages": [ { "id": "2229272", "postDate": "04/21/2023 07:57:02", "content": "

after multiple notebooks and the latest notebook ( after 77 versions ), found that not just gpt can predict the neutrino particle’s direction, but from the pattern how gpt predict the neutrino particle’s direction, leads to that the dataset from iceCube may contain language-like [structure].

\n

abstract [what]

\n

\"\"

\n

gpt is mainly for language model, to prediction next word in sequence. however, this notebook shows [datasets] from the iceCube Neutrino Observatory may contain language-like [structures], after using gpt to predict neutrino particle’s di![]

\n

detail of the notebook and output log:
\nhttps://www.kaggle.com/tyeestudio/language-from-outer-space-in-icecube-data

", "rawMarkdown": "after multiple notebooks and the latest notebook ( after 77 versions ), found that not just gpt can predict the neutrino particle’s direction, but from the pattern how gpt predict the neutrino particle’s direction, leads to that the dataset from iceCube may contain language-like [structure]. \n\n\n# abstract [what]\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1304308%2F191b377b513a20b0c93c4028443233c7%2Fgalaxy.more.avatar.2.png?generation=1682063465253444&alt=media)\n\ngpt is mainly for language model, to prediction next word in sequence. however, this notebook shows [datasets] from the iceCube Neutrino Observatory may contain language-like [structures], after using gpt to predict neutrino particle’s di![]\n\ndetail of the notebook and output log:\nhttps://www.kaggle.com/tyeestudio/language-from-outer-space-in-icecube-data", "votes": null }, { "id": "2229288", "postDate": "04/21/2023 08:12:48", "content": "

highlighted screenshot from part of out log

\n

\"\"

", "rawMarkdown": "highlighted screenshot from part of out log\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1304308%2Ffd43a4f8e4d9b8769185111892800343%2Fnotebook.result.png?generation=1682064750597190&alt=media)", "votes": null }, { "id": "2229773", "postDate": "04/21/2023 17:03:51", "content": "

GPT is also based on transformers (NN layers using self-attention), which is what most of the high scoring notebooks here used.

\n

So I would say it's not that there's language-like structure in Neutrino data
\nbut transformer is a good architecture to find patterns in sequential data.

", "rawMarkdown": "GPT is also based on transformers (NN layers using self-attention), which is what most of the high scoring notebooks here used. \n\nSo I would say it's not that there's language-like structure in Neutrino data\nbut transformer is a good architecture to find patterns in sequential data.", "votes": null }, { "id": "2232399", "postDate": "04/24/2023 10:04:27", "content": "

thanks for the comment.

\n
\n
\n

which is what most of the high scoring notebooks here used.

\n
\n
\n

these result does not indicate that there is not [language-like structure] in Neutrino data], it just mean they got high score, that is not surprise, and they do proved my very first open notebook i published a month ago, which show gpt based ( transformer based ) model can learn, and it can predict the result.

\n

I don't know if you saw the 3 open notebooks i published, the output from these notebook shows gpt ( transformer ) based model can predict the neutrino particle’s direction, specially the 2nd notebook gpt-based-prediction-watch-it-learn, it shows on the screen, how amazingly that gpt based model can learn from very far away from the target (neutrino particle’s direction), to the exactly match the target. then i added angular-dist-score function in my private notebook ( which released, link in this discussion ). the score is 0 after only a few thousand iters, and there is a very good prediction pattern for each training/test dataset.

\n

here are list of open notebooks ( all my open notebooks are only focus on gpt based model on this competition )
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-watch-it-learn
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-no-chatgpt
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-live-loss-plot

", "rawMarkdown": "thanks for the comment.\n\n>>which is what most of the high scoring notebooks here used.\n\nthese result does not indicate that **there is not** [language-like structure] in Neutrino data], it just mean they got high score, that is not surprise, and they **do proved** my very first open notebook i published **a month** ago, which show gpt based ( transformer based ) model can learn, and it can predict the result. \n\nI don't know if you saw the 3 open notebooks i published, the output from these notebook shows gpt ( transformer ) based model can predict the neutrino particle’s direction, specially the 2nd notebook gpt-based-prediction-watch-it-learn, it shows on the screen, how amazingly that gpt based model can learn **from very far away** from the target (neutrino particle’s direction), **to the exactly match** the target. then i added angular-dist-score function in my private notebook ( which released, link in this discussion ). the score is 0 after only a few thousand iters, and there is a very good prediction pattern for each training/test dataset. \n\nhere are list of open notebooks ( all my open notebooks are only focus on gpt based model on this competition )\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-watch-it-learn\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-no-chatgpt\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-live-loss-plot", "votes": null } ], "comments": [ { "id": 2229288, "author_name": "tyeestudio", "author_url": "", "post_date": "04/21/2023 08:12:48", "content": "

highlighted screenshot from part of out log

\n

\"\"

", "votes": null, "replies": [] }, { "id": 2229773, "author_name": "junglebeastds", "author_url": "", "post_date": "04/21/2023 17:03:51", "content": "

GPT is also based on transformers (NN layers using self-attention), which is what most of the high scoring notebooks here used.

\n

So I would say it's not that there's language-like structure in Neutrino data
\nbut transformer is a good architecture to find patterns in sequential data.

", "votes": null, "replies": [ { "id": 2232399, "author_name": "tyeestudio", "author_url": "", "post_date": "04/24/2023 10:04:27", "content": "

thanks for the comment.

\n
\n
\n

which is what most of the high scoring notebooks here used.

\n
\n
\n

these result does not indicate that there is not [language-like structure] in Neutrino data], it just mean they got high score, that is not surprise, and they do proved my very first open notebook i published a month ago, which show gpt based ( transformer based ) model can learn, and it can predict the result.

\n

I don't know if you saw the 3 open notebooks i published, the output from these notebook shows gpt ( transformer ) based model can predict the neutrino particle’s direction, specially the 2nd notebook gpt-based-prediction-watch-it-learn, it shows on the screen, how amazingly that gpt based model can learn from very far away from the target (neutrino particle’s direction), to the exactly match the target. then i added angular-dist-score function in my private notebook ( which released, link in this discussion ). the score is 0 after only a few thousand iters, and there is a very good prediction pattern for each training/test dataset.

\n

here are list of open notebooks ( all my open notebooks are only focus on gpt based model on this competition )
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-watch-it-learn
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-no-chatgpt
\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-live-loss-plot

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2229272": "after multiple notebooks and the latest notebook ( after 77 versions ), found that not just gpt can predict the neutrino particle’s direction, but from the pattern how gpt predict the neutrino particle’s direction, leads to that the dataset from iceCube may contain language-like [structure]. \n\n\n# abstract [what]\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1304308%2F191b377b513a20b0c93c4028443233c7%2Fgalaxy.more.avatar.2.png?generation=1682063465253444&alt=media)\n\ngpt is mainly for language model, to prediction next word in sequence. however, this notebook shows [datasets] from the iceCube Neutrino Observatory may contain language-like [structures], after using gpt to predict neutrino particle’s di![]\n\ndetail of the notebook and output log:\nhttps://www.kaggle.com/tyeestudio/language-from-outer-space-in-icecube-data", "2229288": "highlighted screenshot from part of out log\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1304308%2Ffd43a4f8e4d9b8769185111892800343%2Fnotebook.result.png?generation=1682064750597190&alt=media)", "2229773": "GPT is also based on transformers (NN layers using self-attention), which is what most of the high scoring notebooks here used. \n\nSo I would say it's not that there's language-like structure in Neutrino data\nbut transformer is a good architecture to find patterns in sequential data.", "2232399": "thanks for the comment.\n\n>>which is what most of the high scoring notebooks here used.\n\nthese result does not indicate that **there is not** [language-like structure] in Neutrino data], it just mean they got high score, that is not surprise, and they **do proved** my very first open notebook i published **a month** ago, which show gpt based ( transformer based ) model can learn, and it can predict the result. \n\nI don't know if you saw the 3 open notebooks i published, the output from these notebook shows gpt ( transformer ) based model can predict the neutrino particle’s direction, specially the 2nd notebook gpt-based-prediction-watch-it-learn, it shows on the screen, how amazingly that gpt based model can learn **from very far away** from the target (neutrino particle’s direction), **to the exactly match** the target. then i added angular-dist-score function in my private notebook ( which released, link in this discussion ). the score is 0 after only a few thousand iters, and there is a very good prediction pattern for each training/test dataset. \n\nhere are list of open notebooks ( all my open notebooks are only focus on gpt based model on this competition )\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-watch-it-learn\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-no-chatgpt\nhttps://www.kaggle.com/code/tyeestudio/gpt-based-prediction-live-loss-plot" }, "source": "meta" }