![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15856017%2F0d1936f038226d1d0a8b9cd88f618846%2FScreenshot%202024-12-28%20110051.png?generation=1735363891714947&alt=media\")
In the preprint about this competition and the dataset - https://www.biorxiv.org/content/10.1101/2024.11.04.621686v1
\n
For holdout as well as K fold splits
\nThe Kolmogorov-Smirnov test (KS test) implementation is available for -
\n1-D => https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.ks_2samp.html
\n2-D => https://github.com/o-laurent/multivariate-ks-test
\n3-D => None
\n.
\n.
\n.
\nn-D => None
Did someone find the implementations?
\nI would also request the competition hosts {@kharrington , @rezaparaan } to publish a notebook on the \"how to use KS test to replicate the same split strategy as in preprint\"
", "messages": [ { "id": "3082395", "postDate": "12/28/2024 05:43:58", "content": "In the preprint about this competition and the dataset - https://www.biorxiv.org/content/10.1101/2024.11.04.621686v1
\n
For holdout as well as K fold splits
\nThe Kolmogorov-Smirnov test (KS test) implementation is available for -
\n1-D => https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.ks_2samp.html
\n2-D => https://github.com/o-laurent/multivariate-ks-test
\n3-D => None
\n.
\n.
\n.
\nn-D => None
Did someone find the implementations?
\nI would also request the competition hosts {@kharrington , @rezaparaan } to publish a notebook on the \"how to use KS test to replicate the same split strategy as in preprint\"
", "rawMarkdown": "In the preprint about this competition and the dataset - https://www.biorxiv.org/content/10.1101/2024.11.04.621686v1\n\n\n\n### I want replicate the same split strategy as mentioned in the preprint\nFor holdout as well as K fold splits\n\nThe Kolmogorov-Smirnov test (KS test) implementation is available for -\n1-D => https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.stats.ks_2samp.html\n2-D => https://github.com/o-laurent/multivariate-ks-test\n3-D => None\n.\n.\n.\nn-D => None\n\nDid someone find the implementations?\n\n\n## I don't know why \n## but I feel this is important\n\n\nI would also request the competition hosts {@kharrington , @rezaparaan } to publish a notebook on the ***\"how to use KS test to replicate the same split strategy as in preprint\"***", "votes": null }, { "id": "3082397", "postDate": "12/28/2024 05:49:50", "content": "Another question:
\nWhat does those 16 bins mean?
\nWhere did they come from?
The Kolmogorov-Smirnov Test is a statistical test used to determine whether two distributions were generated from the same or different random processes. I believe all they're saying above is they used that test the ensure the training sets they gave us are a fair representation of both the public and private tests sets our models are scored against. In terms of what they tested against, it's probably just the numbers of each type of particle in each tomogram.
", "rawMarkdown": "The Kolmogorov-Smirnov Test is a statistical test used to determine whether two distributions were generated from the same or different random processes. I believe all they're saying above is they used that test the ensure the training sets they gave us are a fair representation of both the public and private tests sets our models are scored against. In terms of what they tested against, it's probably just the numbers of each type of particle in each tomogram.", "votes": null }, { "id": "3082421", "postDate": "12/28/2024 06:38:42", "content": "I suspect they found 16 different groupings of tomograms with similar KS scores. If true, it's interesting they went from 16 to 7 in coming up with the training set, but I don't think it's anything to be concerned about. We're all in the same boat regardless.
", "rawMarkdown": "I suspect they found 16 different groupings of tomograms with similar KS scores. If true, it's interesting they went from 16 to 7 in coming up with the training set, but I don't think it's anything to be concerned about. We're all in the same boat regardless.", "votes": null }, { "id": "3085598", "postDate": "01/01/2025 11:09:21", "content": " terms what they tested against, it probably just the numbers each particle each tomogram.\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15856017%2F5329f3590f5d39a785319b0b4f92fe09%2FScreenshot%202025-01-01%20163449.png?generation=1735729577800489&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F15856017%2Fe6f565a286ad59c5b9dd2c7118ed0ea7%2FScreenshot%202025-01-01%20163535.png?generation=1735729591090986&alt=media\")
Another question:
\nWhat does those 16 bins mean?
\nWhere did they come from?
I suspect they found 16 different groupings of tomograms with similar KS scores. If true, it's interesting they went from 16 to 7 in coming up with the training set, but I don't think it's anything to be concerned about. We're all in the same boat regardless.
", "votes": null, "replies": [] } ] }, { "id": 3082416, "author_name": "davidlist", "author_url": "", "post_date": "12/28/2024 06:30:32", "content": "The Kolmogorov-Smirnov Test is a statistical test used to determine whether two distributions were generated from the same or different random processes. I believe all they're saying above is they used that test the ensure the training sets they gave us are a fair representation of both the public and private tests sets our models are scored against. In terms of what they tested against, it's probably just the numbers of each type of particle in each tomogram.
", "votes": null, "replies": [ { "id": 3085598, "author_name": "vigneshwar472", "author_url": "", "post_date": "01/01/2025 11:09:21", "content": " terms what they tested against, it probably just the numbers each particle each tomogram.\n