{"cells":[{"metadata":{},"cell_type":"markdown","source":"## The purpose of this notebook\n\nThe discussion topic: \n\n[An easy and convenient way to look up GCS bucket addresses!](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/165739)\n\nFor those of us who are using tfrecords and doing most of their training on Colab, it is very important to have an easy way to look up the Google Cloud Storage (GCS) bucket addresses of various public data sets. It is not enough to do it just once -- you have to update the addresses every few days because they change. This notebook provides a simple and convenient tool for identifying the most current GCS bucket addresses -- the addresses are printed to the screen every time you run the notebook.\n\nThe notebook includes the competition data, the tfrecords datasets made by me, the tfrecords datasets made by [Chris Deotte](https://www.kaggle.com/cdeotte), and the dataset storing [advanced hair augmentation]() images made by [Roman]().","execution_count":null},{"metadata":{"id":"wkB3rarVuglH","outputId":"903039f5-bb33-4c1e-ee38-7755fa3fd8c2","trusted":true},"cell_type":"code","source":"from kaggle_datasets import KaggleDatasets\n\nn=65 # used just for decoration","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Competition data","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(KaggleDatasets().get_gcs_path('siim-isic-melanoma-classification'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Alexey Pronin's work\n\nFor more information see the following discussion topics: \n\n[TF/ TPU: from .tfrec to cross-validation. Step-by-step](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/158395)\n\n[Implementation of Stratified Group K-Folds](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/156002)\n\n\n[Shades of Gray prepossessed data (both JPEG's and tfrecords)](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161719)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"1024x1024, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-1024-train-part-1'))\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-1024-train-part-2'))\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-1024-part-3'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n1024x1024, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-1024-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"768x768, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-768-train-part-1'))\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-768-train-part-2'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n768x768, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-768-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"768x768, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-768-train-part-1'))\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-768-train-part-2'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n768x768, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-768-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n768x768, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-768-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"640x640, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-640-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n640x640, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-640-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n640x640, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-640-test'))","execution_count":null,"outputs":[]},{"metadata":{"id":"Q-C8b_Tnuglo","outputId":"3879f586-7939-435c-ea11-3c5d2ed0dee9","trusted":true},"cell_type":"code","source":"print(\"512x512, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-512x512-tfrec-q95'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n512x512, External data (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-external-data-isic-2019-tfrec-512'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n512x512, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-512x512-tfrec-q95-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"512x512, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-512-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n512x512, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-512-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n512x512, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-512-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"384x384, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-384-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n384x384, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-384-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"384x384, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-384-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n384x384, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-384-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n384x384, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-384-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"256x256, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-256-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n256x256, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-256-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"256x256, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-256-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n256x256, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-256-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n256x256, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-256-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"224x224, train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-224-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n224x224, test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-224-test'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"128x128, Color constant data train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-128-train'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n128x128, Color constant data external (ISIC 2019), only train:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-128-external'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\n128x128, Color constant data test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('siim-tfrec-cc-128-test'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Chris Deotte's work\n\nFor more information see the following discussion topics: \n\n[Triple Stratified Leak-Free KFold CV](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/165526)\n\n[How To Use Last Years 2019 Comp Data (and 2018, 2017)](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/164910)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 1024x1024, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-1024x1024'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 768x768, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-768x768'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 768x768, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-768x768'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 512x512, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-512x512'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 512x512, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-512x512'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 384x384, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-384x384'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 384x384, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-384x384'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 256x256, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-256x256'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 256x256, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-256x256'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 192x192, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-192x192'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 192x192, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-192x192'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Chris, 128x128, train and test:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-128x128'))\n\nprint(\"\")\nprint(\"=\"*n)\n\nprint(\"\\nChris, 128x128, external data:\\n\")\n\nprint(KaggleDatasets().get_gcs_path('isic2019-128x128'))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Roman's work\n\nFor more information see the following discussion topics: \n\n[Advanced hair augmentation](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/159176)\n\n[Advanced hair augmentation in TensorFlow](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/163909)\n\nand the following public notebook:\n\n[SIIM: Advanced Hair Augmentation in TensorFlow](https://www.kaggle.com/graf10a/siim-advanced-hair-augmentation-in-tensorflow)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"Roman, hairs files (originally designed for 256x256 image size):\\n\")\n\nprint(KaggleDatasets().get_gcs_path('melanoma-hairs'))","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}