{
  "id": 164916,
  "title": "Colab KaggleDatasets() Issue",
  "url": "/competitions/siim-isic-melanoma-classification/discussion/164916",
  "author_name": "",
  "post_date": "2020-07-07T20:21:06.018786600Z",
  "votes": 7,
  "comment_count": 12,
  "views": 0,
  "content": "<p>Hello Everyone,</p>\n\n<p>I just started using colab (with TPU) for Kaggle competition.\nIt looks like i cannot do\n<code>from kaggle_datasets import KaggleDatasets</code> \nin colab and connect the GCS path with KaggleDatasets().get_gcs_path.</p>\n\n<p>Does anyone know how to solve the issue? Searched a lot but did not find a proper solution.\nCould not find a place to install kaggle_dataset package....</p>\n\n<p>Thank you for your time. :)</p>",
  "messages": [
    {
      "id": "919380",
      "postDate": "07/07/2020 20:21:06",
      "content": "<p>Hello Everyone,</p>\n\n<p>I just started using colab (with TPU) for Kaggle competition.\nIt looks like i cannot do\n<code>from kaggle_datasets import KaggleDatasets</code> \nin colab and connect the GCS path with KaggleDatasets().get_gcs_path.</p>\n\n<p>Does anyone know how to solve the issue? Searched a lot but did not find a proper solution.\nCould not find a place to install kaggle_dataset package....</p>\n\n<p>Thank you for your time. :)</p>",
      "rawMarkdown": "Hello Everyone,\n\nI just started using colab (with TPU) for Kaggle competition.\nIt looks like i cannot do\n`from kaggle_datasets import KaggleDatasets` \nin colab and connect the GCS path with KaggleDatasets().get_gcs_path.\n\nDoes anyone know how to solve the issue? Searched a lot but did not find a proper solution.\nCould not find a place to install kaggle_dataset package....\n\nThank you for your time. :)",
      "votes": null
    },
    {
      "id": "919389",
      "postDate": "07/07/2020 20:27:59",
      "content": "<p><code>KaggleDatasets</code> does not work on Colab. If you want to run your notebook on Colab, you will need to enter the GSC bucket addresses for the datasets manually. These addresses change once in every 5-7 days, so you will also need to update them periodically.  </p>\n\n<p>Here is my public notebook which should also work on Colab after some minor adjustments (like setting <code>colab=1</code>, entering your local paths, and updating GSC bucket addresses):</p>\n\n<p><a href=\"https://www.kaggle.com/graf10a/efficientnet-bn-tabular-features-tf-cv5-512x512\">EfficientNet BN+Tabular Features TF CV5 512x512</a></p>\n\n<p>You can use it as a starting point.</p>",
      "rawMarkdown": "`KaggleDatasets` does not work on Colab. If you want to run your notebook on Colab, you will need to enter the GSC bucket addresses for the datasets manually. These addresses change once in every 5-7 days, so you will also need to update them periodically.  \n\nHere is my public notebook which should also work on Colab after some minor adjustments (like setting `colab=1`, entering your local paths, and updating GSC bucket addresses):\n\n[EfficientNet BN+Tabular Features TF CV5 512x512](https://www.kaggle.com/graf10a/efficientnet-bn-tabular-features-tf-cv5-512x512)\n\n You can use it as a starting point.",
      "votes": null
    },
    {
      "id": "919401",
      "postDate": "07/07/2020 20:37:35",
      "content": "<p><a href=\"/graf10a\">@graf10a</a> \nThank you so much Alex for your help and quick reply :) 👍 </p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2313184%2F95a3ace1dfbe6e95a5bae5ab42a573a9%2FScreen%20Shot%202020-07-07%20at%204.34.31%20PM.png?generation=1594154110883532&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "graf10a \nThank you so much Alex for your help and quick reply :) 👍 \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2313184%2F95a3ace1dfbe6e95a5bae5ab42a573a9%2FScreen%20Shot%202020-07-07%20at%204.34.31%20PM.png?generation=1594154110883532&amp;alt=media)",
      "votes": null
    },
    {
      "id": "919437",
      "postDate": "07/07/2020 21:18:49",
      "content": "<p><a href=\"/graf10a\">@graf10a</a> Another question, I can see Colab is using 3.6, is there any way to update to 3.7 without influencing its other environment? I tried to upgrade it to 3.7 and looks like the environment is broken :(</p>\n\n<p>I can see there are some warnings if I use 3.6. (Or I can just ignore it?)</p>\n\n<p>Thank you for your time.</p>",
      "rawMarkdown": "graf10a Another question, I can see Colab is using 3.6, is there any way to update to 3.7 without influencing its other environment? I tried to upgrade it to 3.7 and looks like the environment is broken :(\n\nI can see there are some warnings if I use 3.6. (Or I can just ignore it?)\n\nThank you for your time.",
      "votes": null
    },
    {
      "id": "919487",
      "postDate": "07/07/2020 22:55:57",
      "content": "<p><a href=\"/reputation18\">@reputation18</a> Sorry, I have never tried updating Python on Colab, so I don't know much about it. </p>",
      "rawMarkdown": "reputation18 Sorry, I have never tried updating Python on Colab, so I don't know much about it.",
      "votes": null
    },
    {
      "id": "919490",
      "postDate": "07/07/2020 23:00:33",
      "content": "<p>No problem. Thank you. Looks like the model is running fine, with some warnings.</p>",
      "rawMarkdown": "No problem. Thank you. Looks like the model is running fine, with some warnings.",
      "votes": null
    },
    {
      "id": "919588",
      "postDate": "07/08/2020 01:57:10",
      "content": "<p><a href=\"/reputation18\">@reputation18</a> <a href=\"/graf10a\">@graf10a</a> Any idea how to remove the 'WARNING' and 'INFO' while running in colab. It makes the notebook look kind of messy. I have tried <strong>warnings.filterwarnings('ignore')</strong> but seems like it is not working.</p>",
      "rawMarkdown": "reputation18 @graf10a Any idea how to remove the 'WARNING' and 'INFO' while running in colab. It makes the notebook look kind of messy. I have tried **warnings.filterwarnings('ignore')** but seems like it is not working.",
      "votes": null
    },
    {
      "id": "919592",
      "postDate": "07/08/2020 02:00:05",
      "content": "<p>I ignore them =.= \nIt only shown up once actually during modeling...</p>",
      "rawMarkdown": "I ignore them =.= \nIt only shown up once actually during modeling...",
      "votes": null
    },
    {
      "id": "919601",
      "postDate": "07/08/2020 02:19:58",
      "content": "<p><a href=\"/josealways123\">@josealways123</a>  Are you talking about the messages generated when you are initializing your TPU? If so, I do not know.  I tried to Google it but was not able to find how to suppress this output. So, I made my peace with it and moved on.☹️ </p>",
      "rawMarkdown": "josealways123  Are you talking about the messages generated when you are initializing your TPU? If so, I do not know.  I tried to Google it but was not able to find how to suppress this output. So, I made my peace with it and moved on.☹️",
      "votes": null
    },
    {
      "id": "919872",
      "postDate": "07/08/2020 06:43:54",
      "content": "<p><a href=\"/graf10a\">@graf10a</a>  will a simple filterwarnings statement not work here? Something like\nImport warnings\nwarnings.filterwarnings(ignore)</p>",
      "rawMarkdown": "graf10a  will a simple filterwarnings statement not work here? Something like\nImport warnings\nwarnings.filterwarnings(ignore)",
      "votes": null
    },
    {
      "id": "919879",
      "postDate": "07/08/2020 06:50:07",
      "content": "<p>I think I tried it with no success. Maybe I messed up somewhere. Anyhow, it does not bother me much.</p>",
      "rawMarkdown": "I think I tried it with no success. Maybe I messed up somewhere. Anyhow, it does not bother me much.",
      "votes": null
    },
    {
      "id": "920023",
      "postDate": "07/08/2020 08:50:39",
      "content": "<p><a href=\"/aryanpandey1109\">@aryanpandey1109</a> Yeah tried that but not working. Not much of an issue though. Thanks for the answers :)</p>",
      "rawMarkdown": "aryanpandey1109 Yeah tried that but not working. Not much of an issue though. Thanks for the answers :)",
      "votes": null
    },
    {
      "id": "920118",
      "postDate": "07/08/2020 10:56:35",
      "content": "<p>I use the method that I found <a href=\"https://medium.com/analytics-vidhya/how-to-fetch-kaggle-datasets-into-google-colab-ea682569851a\">here.</a> That works for me at least.</p>",
      "rawMarkdown": "I use the method that I found [here.](https://medium.com/analytics-vidhya/how-to-fetch-kaggle-datasets-into-google-colab-ea682569851a) That works for me at least.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 919389,
      "author_name": "graf10a",
      "author_url": "",
      "post_date": "07/07/2020 20:27:59",
      "content": "<p><code>KaggleDatasets</code> does not work on Colab. If you want to run your notebook on Colab, you will need to enter the GSC bucket addresses for the datasets manually. These addresses change once in every 5-7 days, so you will also need to update them periodically.  </p>\n\n<p>Here is my public notebook which should also work on Colab after some minor adjustments (like setting <code>colab=1</code>, entering your local paths, and updating GSC bucket addresses):</p>\n\n<p><a href=\"https://www.kaggle.com/graf10a/efficientnet-bn-tabular-features-tf-cv5-512x512\">EfficientNet BN+Tabular Features TF CV5 512x512</a></p>\n\n<p>You can use it as a starting point.</p>",
      "votes": null,
      "replies": [
        {
          "id": 919401,
          "author_name": "reputation18",
          "author_url": "",
          "post_date": "07/07/2020 20:37:35",
          "content": "<p><a href=\"/graf10a\">@graf10a</a> \nThank you so much Alex for your help and quick reply :) 👍 </p>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2313184%2F95a3ace1dfbe6e95a5bae5ab42a573a9%2FScreen%20Shot%202020-07-07%20at%204.34.31%20PM.png?generation=1594154110883532&amp;alt=media\" alt=\"\"></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919437,
          "author_name": "reputation18",
          "author_url": "",
          "post_date": "07/07/2020 21:18:49",
          "content": "<p><a href=\"/graf10a\">@graf10a</a> Another question, I can see Colab is using 3.6, is there any way to update to 3.7 without influencing its other environment? I tried to upgrade it to 3.7 and looks like the environment is broken :(</p>\n\n<p>I can see there are some warnings if I use 3.6. (Or I can just ignore it?)</p>\n\n<p>Thank you for your time.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919487,
          "author_name": "graf10a",
          "author_url": "",
          "post_date": "07/07/2020 22:55:57",
          "content": "<p><a href=\"/reputation18\">@reputation18</a> Sorry, I have never tried updating Python on Colab, so I don't know much about it. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919490,
          "author_name": "reputation18",
          "author_url": "",
          "post_date": "07/07/2020 23:00:33",
          "content": "<p>No problem. Thank you. Looks like the model is running fine, with some warnings.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919588,
          "author_name": "josealways123",
          "author_url": "",
          "post_date": "07/08/2020 01:57:10",
          "content": "<p><a href=\"/reputation18\">@reputation18</a> <a href=\"/graf10a\">@graf10a</a> Any idea how to remove the 'WARNING' and 'INFO' while running in colab. It makes the notebook look kind of messy. I have tried <strong>warnings.filterwarnings('ignore')</strong> but seems like it is not working.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919592,
          "author_name": "reputation18",
          "author_url": "",
          "post_date": "07/08/2020 02:00:05",
          "content": "<p>I ignore them =.= \nIt only shown up once actually during modeling...</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919601,
          "author_name": "graf10a",
          "author_url": "",
          "post_date": "07/08/2020 02:19:58",
          "content": "<p><a href=\"/josealways123\">@josealways123</a>  Are you talking about the messages generated when you are initializing your TPU? If so, I do not know.  I tried to Google it but was not able to find how to suppress this output. So, I made my peace with it and moved on.☹️ </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919872,
          "author_name": "aryanpandey1109",
          "author_url": "",
          "post_date": "07/08/2020 06:43:54",
          "content": "<p><a href=\"/graf10a\">@graf10a</a>  will a simple filterwarnings statement not work here? Something like\nImport warnings\nwarnings.filterwarnings(ignore)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 919879,
          "author_name": "graf10a",
          "author_url": "",
          "post_date": "07/08/2020 06:50:07",
          "content": "<p>I think I tried it with no success. Maybe I messed up somewhere. Anyhow, it does not bother me much.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 920023,
          "author_name": "josealways123",
          "author_url": "",
          "post_date": "07/08/2020 08:50:39",
          "content": "<p><a href=\"/aryanpandey1109\">@aryanpandey1109</a> Yeah tried that but not working. Not much of an issue though. Thanks for the answers :)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 920118,
      "author_name": "jaredsavage",
      "author_url": "",
      "post_date": "07/08/2020 10:56:35",
      "content": "<p>I use the method that I found <a href=\"https://medium.com/analytics-vidhya/how-to-fetch-kaggle-datasets-into-google-colab-ea682569851a\">here.</a> That works for me at least.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "919380": "Hello Everyone,\n\nI just started using colab (with TPU) for Kaggle competition.\nIt looks like i cannot do\n`from kaggle_datasets import KaggleDatasets` \nin colab and connect the GCS path with KaggleDatasets().get_gcs_path.\n\nDoes anyone know how to solve the issue? Searched a lot but did not find a proper solution.\nCould not find a place to install kaggle_dataset package....\n\nThank you for your time. :)",
    "919389": "`KaggleDatasets` does not work on Colab. If you want to run your notebook on Colab, you will need to enter the GSC bucket addresses for the datasets manually. These addresses change once in every 5-7 days, so you will also need to update them periodically.  \n\nHere is my public notebook which should also work on Colab after some minor adjustments (like setting `colab=1`, entering your local paths, and updating GSC bucket addresses):\n\n[EfficientNet BN+Tabular Features TF CV5 512x512](https://www.kaggle.com/graf10a/efficientnet-bn-tabular-features-tf-cv5-512x512)\n\n You can use it as a starting point.",
    "919401": "graf10a \nThank you so much Alex for your help and quick reply :) 👍 \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F2313184%2F95a3ace1dfbe6e95a5bae5ab42a573a9%2FScreen%20Shot%202020-07-07%20at%204.34.31%20PM.png?generation=1594154110883532&amp;alt=media)",
    "919437": "graf10a Another question, I can see Colab is using 3.6, is there any way to update to 3.7 without influencing its other environment? I tried to upgrade it to 3.7 and looks like the environment is broken :(\n\nI can see there are some warnings if I use 3.6. (Or I can just ignore it?)\n\nThank you for your time.",
    "919487": "reputation18 Sorry, I have never tried updating Python on Colab, so I don't know much about it.",
    "919490": "No problem. Thank you. Looks like the model is running fine, with some warnings.",
    "919588": "reputation18 @graf10a Any idea how to remove the 'WARNING' and 'INFO' while running in colab. It makes the notebook look kind of messy. I have tried **warnings.filterwarnings('ignore')** but seems like it is not working.",
    "919592": "I ignore them =.= \nIt only shown up once actually during modeling...",
    "919601": "josealways123  Are you talking about the messages generated when you are initializing your TPU? If so, I do not know.  I tried to Google it but was not able to find how to suppress this output. So, I made my peace with it and moved on.☹️",
    "919872": "graf10a  will a simple filterwarnings statement not work here? Something like\nImport warnings\nwarnings.filterwarnings(ignore)",
    "919879": "I think I tried it with no success. Maybe I messed up somewhere. Anyhow, it does not bother me much.",
    "920023": "aryanpandey1109 Yeah tried that but not working. Not much of an issue though. Thanks for the answers :)",
    "920118": "I use the method that I found [here.](https://medium.com/analytics-vidhya/how-to-fetch-kaggle-datasets-into-google-colab-ea682569851a) That works for me at least."
  },
  "source": "meta"
}