I converted Roland's notebook into separate python scripts.
\nhttps://github.com/anlthms/rsna-miccai-2021
\nThis was originally for use with Luminide. Publishing it here in case it is useful to other folks who are training their models outside of Kaggleโฆ
", "messages": [ { "id": "1452976", "postDate": "08/05/2021 18:21:28", "content": "I converted Roland's notebook into separate python scripts.
\nhttps://github.com/anlthms/rsna-miccai-2021
\nThis was originally for use with Luminide. Publishing it here in case it is useful to other folks who are training their models outside of Kaggleโฆ
", "rawMarkdown": "I converted [Roland's notebook](https://www.kaggle.com/rluethy/efficientnet3d-with-one-mri-type) into separate python scripts. \n\nhttps://github.com/anlthms/rsna-miccai-2021\n\nThis was originally for use with [Luminide](https://luminide.com). Publishing it here in case it is useful to other folks who are training their models outside of Kaggle...", "votes": null }, { "id": "1453708", "postDate": "08/06/2021 01:21:36", "content": "An AUC of 0.684 sounded great until I learned that the test set has 87 samples in total. Here's some fun code:
\nimport numpy as np\nfrom sklearn.metrics import roc_auc_score\n\nnp.random.seed(317)\nN = 87\nlabels = np.random.random(N).round()\npreds = np.random.random(N)\nauc = roc_auc_score(labels, preds)\nprint(f'auc={auc:.3f}')\nThis outputs auc=0.690 ๐
I still believe that the repo linked above is a great starting point. The 0.684 score might have come from a lucky run, though.
", "rawMarkdown": "An AUC of 0.684 sounded great until I learned that the test set has 87 samples in total. Here's some fun code:\n```\nimport numpy as np\nfrom sklearn.metrics import roc_auc_score\n\nnp.random.seed(317)\nN = 87\nlabels = np.random.random(N).round()\npreds = np.random.random(N)\nauc = roc_auc_score(labels, preds)\nprint(f'auc={auc:.3f}')\n```\n\nThis outputs `auc=0.690` ๐\n\nI still believe that the repo linked above is a great starting point. The 0.684 score might have come from a lucky run, though.", "votes": null }, { "id": "1458405", "postDate": "08/07/2021 21:09:44", "content": "Funny how this thread might be a piece of evidence that Leaderboard is pretty much random, but nobody comments here
", "rawMarkdown": "Funny how this thread might be a piece of evidence that Leaderboard is pretty much random, but nobody comments here", "votes": null }, { "id": "1458596", "postDate": "08/08/2021 01:00:22", "content": "I agree that this was a lucky run. I tried different seeds with the same notebook and got much worse LB AUC.
", "rawMarkdown": "I agree that this was a lucky run. I tried different seeds with the same notebook and got much worse LB AUC.", "votes": null }, { "id": "1458621", "postDate": "08/08/2021 01:30:58", "content": "Hi, I thought you are participated in the FGVC8 competition and got 3rd place using Luminide.
\nBecause I am writing my capstone project based on the FGVC8 competition, I am impressed with your solution.
\nI want to try Luminide as well, but I can't find the pricing page. What is the pricing of Luminide? Can Luminide access by the public? is there any tutorial of Luminide?
\nCould you provide some information about Luminide? much thanks.
Hi Shanshan, please add yourself to our queue by using the \"Request early access\" button on the website. We will notify you as soon as there is availability.
", "rawMarkdown": "Hi Shanshan, please add yourself to our queue by using the \"Request early access\" button on the [website](https://luminide.com/). We will notify you as soon as there is availability.", "votes": null }, { "id": "1460920", "postDate": "08/09/2021 06:14:44", "content": "The log loss hovering around -log(0.5)=0.6931 should have clued me in ๐ค
", "rawMarkdown": "The log loss hovering around -log(0.5)=0.6931 should have clued me in ๐ค", "votes": null }, { "id": "1461970", "postDate": "08/09/2021 16:29:55", "content": "Hello, I am new to the deep learning field.
\ncould you please explain what you mean by log-loss hovering , is that regarding parameter tunning in 3D efficient net training?
The output of this notebook shows that the validation log loss is often close to 0.6931. This indicates that the model may not have learned anything useful. For a binary classification problem, 0.6931 is the expected negative log loss if you set all predictions to 0.5.
", "rawMarkdown": "The [output of this notebook](https://www.kaggle.com/rluethy/efficientnet3d-with-one-mri-type?scriptVersionId=70464234) shows that the validation log loss is often close to 0.6931. This indicates that the model may not have learned anything useful. For a binary classification problem, 0.6931 is the expected [negative log loss](https://en.wikipedia.org/wiki/Cross_entropy) if you set all predictions to 0.5.", "votes": null }, { "id": "1470638", "postDate": "08/13/2021 16:25:10", "content": "What could be a reason for such performance?
", "rawMarkdown": "What could be a reason for such performance?", "votes": null }, { "id": "1472206", "postDate": "08/14/2021 17:50:53", "content": "@novice03 small data is the biggest issue in this competition. That's why model is not improving.
", "rawMarkdown": "novice03 small data is the biggest issue in this competition. That's why model is not improving.", "votes": null } ], "comments": [ { "id": 1453708, "author_name": "anlthms", "author_url": "", "post_date": "08/06/2021 01:21:36", "content": "An AUC of 0.684 sounded great until I learned that the test set has 87 samples in total. Here's some fun code:
\nimport numpy as np\nfrom sklearn.metrics import roc_auc_score\n\nnp.random.seed(317)\nN = 87\nlabels = np.random.random(N).round()\npreds = np.random.random(N)\nauc = roc_auc_score(labels, preds)\nprint(f'auc={auc:.3f}')\nThis outputs auc=0.690 ๐
I still believe that the repo linked above is a great starting point. The 0.684 score might have come from a lucky run, though.
", "votes": null, "replies": [ { "id": 1458596, "author_name": "rluethy", "author_url": "", "post_date": "08/08/2021 01:00:22", "content": "I agree that this was a lucky run. I tried different seeds with the same notebook and got much worse LB AUC.
", "votes": null, "replies": [] }, { "id": 1460920, "author_name": "anlthms", "author_url": "", "post_date": "08/09/2021 06:14:44", "content": "The log loss hovering around -log(0.5)=0.6931 should have clued me in ๐ค
", "votes": null, "replies": [] }, { "id": 1461970, "author_name": "wfarzana", "author_url": "", "post_date": "08/09/2021 16:29:55", "content": "Hello, I am new to the deep learning field.
\ncould you please explain what you mean by log-loss hovering , is that regarding parameter tunning in 3D efficient net training?
The output of this notebook shows that the validation log loss is often close to 0.6931. This indicates that the model may not have learned anything useful. For a binary classification problem, 0.6931 is the expected negative log loss if you set all predictions to 0.5.
", "votes": null, "replies": [] }, { "id": 1470638, "author_name": "novice03", "author_url": "", "post_date": "08/13/2021 16:25:10", "content": "What could be a reason for such performance?
", "votes": null, "replies": [] }, { "id": 1472206, "author_name": "kfk42kfk", "author_url": "", "post_date": "08/14/2021 17:50:53", "content": "@novice03 small data is the biggest issue in this competition. That's why model is not improving.
", "votes": null, "replies": [] } ] }, { "id": 1458405, "author_name": "ivanpan", "author_url": "", "post_date": "08/07/2021 21:09:44", "content": "Funny how this thread might be a piece of evidence that Leaderboard is pretty much random, but nobody comments here
", "votes": null, "replies": [] }, { "id": 1458621, "author_name": "bcghost", "author_url": "", "post_date": "08/08/2021 01:30:58", "content": "Hi, I thought you are participated in the FGVC8 competition and got 3rd place using Luminide.
\nBecause I am writing my capstone project based on the FGVC8 competition, I am impressed with your solution.
\nI want to try Luminide as well, but I can't find the pricing page. What is the pricing of Luminide? Can Luminide access by the public? is there any tutorial of Luminide?
\nCould you provide some information about Luminide? much thanks.
Hi Shanshan, please add yourself to our queue by using the \"Request early access\" button on the website. We will notify you as soon as there is availability.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1452976": "I converted [Roland's notebook](https://www.kaggle.com/rluethy/efficientnet3d-with-one-mri-type) into separate python scripts. \n\nhttps://github.com/anlthms/rsna-miccai-2021\n\nThis was originally for use with [Luminide](https://luminide.com). Publishing it here in case it is useful to other folks who are training their models outside of Kaggle...", "1453708": "An AUC of 0.684 sounded great until I learned that the test set has 87 samples in total. Here's some fun code:\n```\nimport numpy as np\nfrom sklearn.metrics import roc_auc_score\n\nnp.random.seed(317)\nN = 87\nlabels = np.random.random(N).round()\npreds = np.random.random(N)\nauc = roc_auc_score(labels, preds)\nprint(f'auc={auc:.3f}')\n```\n\nThis outputs `auc=0.690` ๐\n\nI still believe that the repo linked above is a great starting point. The 0.684 score might have come from a lucky run, though.", "1458405": "Funny how this thread might be a piece of evidence that Leaderboard is pretty much random, but nobody comments here", "1458596": "I agree that this was a lucky run. I tried different seeds with the same notebook and got much worse LB AUC.", "1458621": "Hi, I thought you are participated in the FGVC8 competition and got 3rd place using Luminide.\nBecause I am writing my capstone project based on the FGVC8 competition, I am impressed with your solution.\nI want to try Luminide as well, but I can't find the pricing page. What is the pricing of Luminide? Can Luminide access by the public? is there any tutorial of Luminide?\nCould you provide some information about Luminide? much thanks.", "1460916": "Hi Shanshan, please add yourself to our queue by using the \"Request early access\" button on the [website](https://luminide.com/). We will notify you as soon as there is availability.", "1460920": "The log loss hovering around -log(0.5)=0.6931 should have clued me in ๐ค", "1461970": "Hello, I am new to the deep learning field.\ncould you please explain what you mean by log-loss hovering , is that regarding parameter tunning in 3D efficient net training?", "1462223": "The [output of this notebook](https://www.kaggle.com/rluethy/efficientnet3d-with-one-mri-type?scriptVersionId=70464234) shows that the validation log loss is often close to 0.6931. This indicates that the model may not have learned anything useful. For a binary classification problem, 0.6931 is the expected [negative log loss](https://en.wikipedia.org/wiki/Cross_entropy) if you set all predictions to 0.5.", "1470638": "What could be a reason for such performance?", "1472206": "novice03 small data is the biggest issue in this competition. That's why model is not improving." }, "source": "meta" }