New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with!
\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!
\nNew to Kaggle? Take a look at a few videos to learn a bit more about site etiquette, Kaggle lingo, and how to enter a competition using Kaggle Notebooks.
\nReady to dive into this competition? Review the Overview Description and start to work with the Data!
\n\n", "messages": [ { "id": 1084243, "postDate": "2020-11-19T21:20:37.733Z", "content": "Remember: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our Kaggle community guidelines.
\n
New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with!
\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!
\nNew to Kaggle? Take a look at a few videos to learn a bit more about site etiquette, Kaggle lingo, and how to enter a competition using Kaggle Notebooks.
\nReady to dive into this competition? Review the Overview Description and start to work with the Data!
\n\n", "rawMarkdown": "New to machine learning and data science? No question is too basic or too simple. Feel free to start your own thread, or use this thread as a place to post any first-timer clarifying questions for the Kaggle community to help you with! \n\nIf you would consider yourself a beginner but don't know where to get started, let other Kagglers help you take your first steps here!\n\nNew to Kaggle? Take a look at a few videos to learn a bit more about [site etiquette](https://www.youtube.com/watch?v=aIus8si_Et0), [Kaggle lingo](https://www.youtube.com/watch?v=sEJHyuWKd-s), and [how to enter a competition using Kaggle Notebooks](https://www.youtube.com/watch?&v=GJBOMWpLpTQ).\n\nReady to dive into this competition? Review the [Overview Description](https://www.kaggle.com/c/cassava-leaf-disease-classification/overview) and start to work with the [Data](https://www.kaggle.com/c/cassava-leaf-disease-classification/data)!\n\n> **Remember**: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our [Kaggle community guidelines](https://www.kaggle.com/community-guidelines).", "votes": 75 }, { "id": 1085046, "postDate": "2020-11-20T16:21:30.780Z", "content": "Remember: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our Kaggle community guidelines.
\n
How do majority of people start? Do they download the entire data set on their local machine? Or do they use the Kaggle notebook to work with the data set? Cause 6GB is huge..
", "rawMarkdown": "How do majority of people start? Do they download the entire data set on their local machine? Or do they use the Kaggle notebook to work with the data set? Cause 6GB is huge.. ", "votes": 4, "replies": [ { "id": 1086728, "postDate": "2020-11-22T02:05:03.953Z", "content": "What is need of downloading dataset when all you need is on kaggle. You leverage GPU and TPU also, without any impact on your hardware. Plus you show your work to others and contribute to ML AND DL society.
", "rawMarkdown": "What is need of downloading dataset when all you need is on kaggle. You leverage GPU and TPU also, without any impact on your hardware. Plus you show your work to others and contribute to ML AND DL society.", "votes": 8 }, { "id": 1090622, "postDate": "2020-11-25T13:39:33.993Z", "content": "The majority of people start with the kaggle courses to learn data science and machine learning.
\nWe use the Kaggle notebook to work with the data set on the website, and we usually don't download the entire dat set to work on our local machines because there is no need for that when you can just work on the cloud and run it on their servers that are more powerful in terms on GPU and TPU.
", "rawMarkdown": "The majority of people start with the kaggle courses to learn data science and machine learning.\n\nWe use the Kaggle notebook to work with the data set on the website, and we usually don't download the entire dat set to work on our local machines because there is no need for that when you can just work on the cloud and run it on their servers that are more powerful in terms on GPU and TPU.", "votes": 3 }, { "id": 1093098, "postDate": "2020-11-27T12:49:17.660Z", "content": "no we import data sets using kaggle API .once check it out
", "rawMarkdown": "no we import data sets using kaggle API .once check it out ", "votes": 2 }, { "id": 1093146, "postDate": "2020-11-27T13:39:54.073Z", "content": "You can also use Kaggle notebook or you can use the dataset in environments such as google colab using the kaggle api.
\nYou can find below resources related this topic.
https://medium.com/@galhever/how-to-import-data-from-kaggle-to-google-colab-8160caa11e2
\nhttps://colab.research.google.com/github/corrieann/kaggle/blob/master/kaggle_api_in_colab.ipynb
I'm having some issues with submission and I could use some help.
\nHowever when I try to submit I get this error…
\n\n\nYour Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version
\n
looking through the codebook, this is from this line no longer works
\n\n\n1 GCS_DS_PATH = KaggleDatasets().get_gcs_path('cassava-leaf-disease-classification') # you can list the bucket with \"!gsutil ls $GCS_DS_PATH\"
\nConnectionError: Connection error trying to communicate with service.
\n
So how do I read the test filenames if I have to switch off Internet Access?
", "rawMarkdown": "I'm having some issues with submission and I could use some help.\n\n- Trained a model with TPU - done\n- Saved model.h5 as dataset and imported into another GPU notebook - done\n- run new notebook and get a prediction - done\n\nHowever when I try to submit I get this error...\n\n> Your Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version\n\nlooking through the codebook, this is from this line no longer works\n> 1 GCS_DS_PATH = KaggleDatasets().get_gcs_path('cassava-leaf-disease-classification') # you can list the bucket with \"!gsutil ls $GCS_DS_PATH\"\n\n> ConnectionError: Connection error trying to communicate with service.\n\n\nSo how do I read the test filenames if I have to switch off Internet Access?", "votes": 1, "replies": [ { "id": 1193927, "postDate": "2021-02-10T01:27:41.197Z", "content": "Am in the same boat, did you figure out how to do this w/o an internet connection?
", "rawMarkdown": "Am in the same boat, did you figure out how to do this w/o an internet connection?" }, { "id": 1196621, "postDate": "2021-02-11T14:10:31.393Z", "content": "Yes I figured it out. I changed the file location from
\nGCS_DS_PATH = KaggleDatasets().get_gcs_path
to
\n'/kaggle/input/'
", "rawMarkdown": "Yes I figured it out. I changed the file location from \nGCS_DS_PATH = KaggleDatasets().get_gcs_path\n\nto \n\n'/kaggle/input/'" } ] }, { "id": 1122004, "postDate": "2020-12-22T06:01:46.023Z", "content": "Can the public leaderboard please show up to 4-5 digits of the scores?
", "rawMarkdown": "Can the public leaderboard please show up to 4-5 digits of the scores?", "votes": 1 }, { "id": 1121918, "postDate": "2020-12-22T02:49:28.470Z", "content": "are there examples of doing cross validation with pytorch?
", "rawMarkdown": "are there examples of doing cross validation with pytorch? " }, { "id": 3013668, "postDate": "2024-10-10T11:56:48.293Z", "content": "I am a product manager and started a master's in data science, I don't have a technical background and my program is demanding.
\nShould I quit the job and focus on my master's to be able to make tangible progress? any experience or advice is highly appreciated
How is it finds well for a data scientist?
", "rawMarkdown": "How is it finds well for a data scientist?" }, { "id": 1672275, "postDate": "2022-02-02T01:48:47.247Z", "content": "good work. cool ……………..
", "rawMarkdown": "good work. cool ................." }, { "id": 1277618, "postDate": "2021-04-19T02:09:29.623Z", "content": "cool………..
", "rawMarkdown": "cool..........." }, { "id": 1211078, "postDate": "2021-02-19T23:48:01.920Z", "content": "Why doesn't my score show up in the leaderboard? I submitted 173!
", "rawMarkdown": "Why doesn't my score show up in the leaderboard? I submitted 173!" }, { "id": 1206339, "postDate": "2021-02-17T08:31:59.457Z", "content": "I've just done my first simple CNN and tried to submit it to the competition. Could someone check my notebook and let me know what is wrong. Huge Thanks
\nhttps://www.kaggle.com/godzill22/casava-leaves-comp-first-cnn
I manage to do it
", "rawMarkdown": "I manage to do it", "votes": 1 } ] }, { "id": 1206220, "postDate": "2021-02-17T07:54:35.993Z", "content": "I want to do ensemble learning. Must I open the submissions to use? Or I can do it in private.
", "rawMarkdown": "I want to do ensemble learning. Must I open the submissions to use? Or I can do it in private." }, { "id": 1194203, "postDate": "2021-02-10T05:02:29.560Z", "content": "Hello, hope y'all doin good. I have a doubt here. how do I move the images from 'train_images\" to different subfolders based on the image_id given in the csv file? like I want all the images with id '0' to be placed in one folder. can anyone help me with this?
", "rawMarkdown": "Hello, hope y'all doin good. I have a doubt here. how do I move the images from 'train_images\" to different subfolders based on the image_id given in the csv file? like I want all the images with id '0' to be placed in one folder. can anyone help me with this? ", "replies": [ { "id": 1197897, "postDate": "2021-02-12T14:18:40.243Z", "content": "You will have to write a utility function to do that. You can try by reading the csv file; using the image_id to create the source path and creating the destination path by using the label value. Further you can use shutil to copy images from source path to destination path.
", "rawMarkdown": "You will have to write a utility function to do that. You can try by reading the csv file; using the image_id to create the source path and creating the destination path by using the label value. Further you can use shutil to copy images from source path to destination path.", "votes": 1 } ] }, { "id": 1191934, "postDate": "2021-02-08T19:30:34.170Z", "content": "Is there a reason why Effnet and Resnet are so popular in this competition, and densenet doesn't show up much ?
", "rawMarkdown": "Is there a reason why Effnet and Resnet are so popular in this competition, and densenet doesn't show up much ?" }, { "id": 1184898, "postDate": "2021-02-03T20:53:57.837Z", "content": "Hi Kagglers!
\nCould someone help me on how to submit my submission file properly. I keep getting the error submission score error . This is my first ever work on kaggle and I finding it quite difficult to traverse around.
\nI have attached a screenshot of the error and code used to create the submission.csv file
@atiaisaac click on the I buttom for more information about the error, your CSV file is in the format expected by competition
", "rawMarkdown": "@atiaisaac click on the I buttom for more information about the error, your CSV file is in the format expected by competition" } ] }, { "id": 1168290, "postDate": "2021-01-24T20:43:09.907Z", "content": "Could you please help me in
\n1- reading the following data-set https://www.kaggle.com/c/cassava-leaf-disease-classification and shape it as:
\n (X_train, y_train), (X_test, y_test)
\n2- applying nr.1 in two cases after balancing the data with augmentation. And without balancing the data
I have ran out of space to upload more models, \"allocated memory ran out\". Is there a way to relocate this to somewhere else? Where are the files saved if I am working on kaggle kernels? Quite new to using kaggle's kernels and how they operate. Thanks!
", "rawMarkdown": "I have ran out of space to upload more models, \"allocated memory ran out\". Is there a way to relocate this to somewhere else? Where are the files saved if I am working on kaggle kernels? Quite new to using kaggle's kernels and how they operate. Thanks!" }, { "id": 1155091, "postDate": "2021-01-16T07:45:53.353Z", "content": "i am using pretrained weights from keras pretrained model dataset for ResNet50. i have saved the weights to directory \"~/.keras/models\" . but when i am using that in my pretrained model. it is redownloading weight and following error is given.
\nError :
\n\"A local file was found, but it seems to be incomplete or outdated because the auto file hash does not match the original value of 4d473c1dd8becc155b73f8504c6f6626 so we will re-download the data.\"
\nPlease suggest what i have done wrong.
Can you share code
", "rawMarkdown": "Can you share code", "votes": 1 }, { "id": 1155112, "postDate": "2021-01-16T08:06:09.280Z", "content": "from tensorflow.keras.applications import ResNet50
\nfrom tensorflow.keras.applications.resnet50 import preprocess_input
\nfrom os import makedirs
\nfrom os.path import join, exists, expanduser
cache_dir = expanduser(join('~', '.keras'))
\nif not exists(cache_dir):
\n makedirs(cache_dir)
\nmodels_dir = join(cache_dir, 'models')
\nif not exists(models_dir):
\n makedirs(models_dir)
!cp ../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5 ~/.keras/models/
\n!cp ../input/resnet50/imagenet_class_index.json ~/.keras/models/
\n!ls ~/.keras/models
base_model = ResNet50(weights=\"imagenet\", include_top=False)
\nbase_model.trainable = False
It is downloading weights because you have put weights=\"imagenet\", if you have added weights then put the path instead
i tried that but it is showing another error.
\nError:
\nshape [1,1,128,512] and [512,256,1,1] is incompatible
Thanks @anantgupt for help. i found the problem. i was actually using an older version of keras pretrained weight dataset.
", "rawMarkdown": "Thanks @anantgupt for help. i found the problem. i was actually using an older version of keras pretrained weight dataset." }, { "id": 1155510, "postDate": "2021-01-16T12:40:27.527Z", "content": "Glad I can be helpful 😃
", "rawMarkdown": "Glad I can be helpful 😃", "votes": 2 } ] }, { "id": 1154113, "postDate": "2021-01-15T12:12:46.533Z", "content": "As internet access is not allowed then how will I use pretrained model?
", "rawMarkdown": "As internet access is not allowed then how will I use pretrained model?", "replies": [ { "id": 1154135, "postDate": "2021-01-15T12:33:24.613Z", "content": "There a number of dataset containing model weight. You can add those datasets otherwise you can create 2 seperate notebook and upload your model in dataset and then use it.
", "rawMarkdown": "There a number of dataset containing model weight. You can add those datasets otherwise you can create 2 seperate notebook and upload your model in dataset and then use it.", "votes": 2 }, { "id": 1154140, "postDate": "2021-01-15T12:37:27.730Z", "content": "Thanks for help.
", "rawMarkdown": "Thanks for help.", "votes": 1 } ] }, { "id": 1152899, "postDate": "2021-01-14T14:53:28.523Z", "content": "Helpful 🙏
", "rawMarkdown": "Helpful 🙏" }, { "id": 1152224, "postDate": "2021-01-14T00:18:05.487Z", "content": "How do I submit my attempt?
", "rawMarkdown": "How do I submit my attempt?" }, { "id": 1145899, "postDate": "2021-01-09T12:18:19.707Z", "content": "Hi everyone
\nGot error on my submissions \"Submission Scoring Error\", worked fine on my local PC …
\nHow can I review the issues and fix? is there model runtime limitation? mine took ~4.5H on my local PC
Thanks
\nYuval.
Hi everyone
\nAny one use tensorboard to review the runs?
\ntry few steps but seems that not working, wandering if this enabled in kaggle environment or/and my wrong usage
model2.compile(
\n optimizer='adam',
\n loss='categorical_crossentropy',
\n metrics=['accuracy'])
RUN_NAME = 'run 1 with 25 nodes'
\nlogger = keras.callbacks.TensorBoard(
\n log_dir='logs/{}'.format(RUN_NAME),
\n histogram_freq=5,
\n write_graph=True
\n)
history = model2.fit_generator(train_datagen_flow,
\n validation_data=valid_datagen_flow,
\n epochs=1,
\n callbacks=[logger]
\n )
tensoboard --logdir=logs
\nthanks
\nYuval
Yes, Kaggle is a good place for beginners like as me. it was better if it had a road map and better courses. For become a data scientist Kaggle is necessity but not enough. I wish they make courses better and more useful.
", "rawMarkdown": "Yes, Kaggle is a good place for beginners like as me. it was better if it had a road map and better courses. For become a data scientist Kaggle is necessity but not enough. I wish they make courses better and more useful." }, { "id": 1140660, "postDate": "2021-01-06T07:04:26.423Z", "content": "Hi,
\nthis is my first competition so have basic question :-)
\nOnce I'm submitting (save & Commit) is it take time to find mine in the leaderboard?
thanks
\nYuval.
By save and commit you are not submitting your work save and saves the notebook for that purpose it runs completely again if your notebook took 1 hr while training then again it will run for 1 hr (when you save and commit it). It will changes in your notebook publically. For submitting to competition you have submit predictions go the my submission button
", "rawMarkdown": "By save and commit you are not submitting your work save and saves the notebook for that purpose it runs completely again if your notebook took 1 hr while training then again it will run for 1 hr (when you save and commit it). It will changes in your notebook publically. For submitting to competition you have submit predictions go the **my submission** button", "votes": 3 }, { "id": 1140975, "postDate": "2021-01-06T12:12:42.943Z", "content": "Thank you Anant
", "rawMarkdown": "Thank you Anant", "votes": 1 } ] }, { "id": 1132687, "postDate": "2020-12-30T16:01:02.340Z", "content": "Is this competition a good start point after finishing the Deep Learning and Computer Vision courses and the previous ones? Is there any other Kaggle course specially recommended for this competition (a part from the specific TPU tutorial)? Thank you!
", "rawMarkdown": "Is this competition a good start point after finishing the Deep Learning and Computer Vision courses and the previous ones? Is there any other Kaggle course specially recommended for this competition (a part from the specific TPU tutorial)? Thank you!" }, { "id": 1130667, "postDate": "2020-12-29T08:03:42.243Z", "content": "Thanks, your suggestions are very helpful to novices
", "rawMarkdown": "Thanks, your suggestions are very helpful to novices" }, { "id": 1121915, "postDate": "2020-12-22T02:46:35.157Z", "content": "I see many examples of using keras on tpu. Are there examples of using pytorch on tpu? Can one create a model on tpu and then load it on gpu or vice-versa?
", "rawMarkdown": "I see many examples of using keras on tpu. Are there examples of using pytorch on tpu? Can one create a model on tpu and then load it on gpu or vice-versa? " }, { "id": 1118287, "postDate": "2020-12-18T22:59:15.367Z", "content": "Hi, everyone! I'm new to Kaggle and this is my first post and my first competition. I am stuck on making a submission and having it scored. When I submit, all of the metrics say that everything ran properly, but the status continually says running until it times out at 9 hours. Should I be importing the trained model and then making the submission with the test file path in a different notebook?
", "rawMarkdown": "Hi, everyone! I'm new to Kaggle and this is my first post and my first competition. I am stuck on making a submission and having it scored. When I submit, all of the metrics say that everything ran properly, but the status continually says running until it times out at 9 hours. Should I be importing the trained model and then making the submission with the test file path in a different notebook?", "replies": [ { "id": 1121924, "postDate": "2020-12-22T02:56:26.910Z", "content": "does your notebook run more than 9 hours? this may happen if you are running your notebook on your own GPU (say aws rented GPU). Kaggle GPU only allows your notebook to run 9 hours and after that it kills it. You may want to create a test only notebook and then upload your created model as a new dataset. Then you submit the test only notebook to the competition. The results may take as much as 10 or 15 minutes to be available.
", "rawMarkdown": "does your notebook run more than 9 hours? this may happen if you are running your notebook on your own GPU (say aws rented GPU). Kaggle GPU only allows your notebook to run 9 hours and after that it kills it. You may want to create a test only notebook and then upload your created model as a new dataset. Then you submit the test only notebook to the competition. The results may take as much as 10 or 15 minutes to be available. ", "votes": 1 } ] }, { "id": 1117400, "postDate": "2020-12-18T02:36:01.923Z", "content": "How can I use keras.applications.EfficientNet in TPU mode ?, I found that it's working in GPU but can not import in GPU
", "rawMarkdown": "How can I use keras.applications.EfficientNet in TPU mode ?, I found that it's working in GPU but can not import in GPU", "replies": [ { "id": 1121146, "postDate": "2020-12-21T12:10:05.590Z", "content": "In TPU mode the TensorFlow version changes to 2.2 which does not include keras.applications.EfficientNet
\nYou can do something like this:
\n!pip install -U efficientnet\n\nimport efficientnet.keras as efn\n\nbase_model = efn.EfficientNetB6(include_top=False, weights=\"imagenet\")\nWhat to do with 5-folds of dataset? Should I train a model on all the 5-folds individually and then take the best one? Or use the metric information from 5-folds to train a model on the complete dataset?
", "rawMarkdown": "What to do with 5-folds of dataset? Should I train a model on all the 5-folds individually and then take the best one? Or use the metric information from 5-folds to train a model on the complete dataset?", "replies": [ { "id": 1116681, "postDate": "2020-12-17T11:45:46.897Z", "content": "5 Folds help you to measure the results robustly. Also you can use trained model from each fold and predict on test set and take \"mode\" to aggregate results.
", "rawMarkdown": "5 Folds help you to measure the results robustly. Also you can use trained model from each fold and predict on test set and take \"mode\" to aggregate results.", "votes": 1 }, { "id": 1117167, "postDate": "2020-12-17T19:21:10.817Z", "content": "Thanks for the information.
", "rawMarkdown": "Thanks for the information." } ] }, { "id": 1110852, "postDate": "2020-12-13T06:22:03.253Z", "content": "not a question, i just wanted to thank you for being so welcoming to us newbies
", "rawMarkdown": "not a question, i just wanted to thank you for being so welcoming to us newbies" }, { "id": 1109223, "postDate": "2020-12-11T12:51:12.130Z", "content": "I don't know if there is a difference if I use my own PC, actually I've heard that many competitions need better GPU or others if you want to get good ranks.
", "rawMarkdown": "I don't know if there is a difference if I use my own PC, actually I've heard that many competitions need better GPU or others if you want to get good ranks.", "replies": [ { "id": 1109309, "postDate": "2020-12-11T14:21:33.573Z", "content": "Your rank does not depend on the GPU you use, the GPU provided by Kaggle is sufficient enough. You rank only depends on the model you train and accuracy and score you achieve. GPU are just there for faster training.
", "rawMarkdown": "Your rank does not depend on the GPU you use, the GPU provided by Kaggle is sufficient enough. You rank only depends on the model you train and accuracy and score you achieve. GPU are just there for faster training.", "votes": 2 } ] }, { "id": 1108858, "postDate": "2020-12-11T04:50:21.677Z", "content": "is it different when I submit a GPU and none-GPU notebook ? My submission with none-GPU is pending in 6 hours, will it faster if I submit with GPU notebook ? thank you
", "rawMarkdown": "is it different when I submit a GPU and none-GPU notebook ? My submission with none-GPU is pending in 6 hours, will it faster if I submit with GPU notebook ? thank you", "replies": [ { "id": 1108868, "postDate": "2020-12-11T05:08:21.273Z", "content": "yes, with GPU, it will be lot faster
", "rawMarkdown": "yes, with GPU, it will be lot faster ", "votes": 2 } ] }, { "id": 1107548, "postDate": "2020-12-09T19:07:40.553Z", "content": "I am new to Kaggle and using my own Computer.
\nCan I upload a .pickle file as dataset on Kaggle in Competition as dataset reading data cause 2min+ delay
\nor i have to execute the pickle writing file code on Kaggle
Safest method is to create the model on Kaggle. I have had difficulty using a model created on one GPU to be loaded onto another GPU. But I am curious to know if my issue is an isolated one and others have experienced the same or there is a solution for this cross-GPU compatibility.
", "rawMarkdown": "Safest method is to create the model on Kaggle. I have had difficulty using a model created on one GPU to be loaded onto another GPU. But I am curious to know if my issue is an isolated one and others have experienced the same or there is a solution for this cross-GPU compatibility.", "votes": 1 } ] }, { "id": 1106472, "postDate": "2020-12-08T21:31:54.900Z", "content": "@juliaelliott Can we get an extra digit on the LB by any chance?
", "rawMarkdown": "@juliaelliott Can we get an extra digit on the LB by any chance?", "replies": [ { "id": 1106596, "postDate": "2020-12-09T00:10:23.743Z", "content": "@lewington We are unlikely to be adding another digit as this becomes vulnerable to unwanted leaderboard probing, but we do keep an eye on the competition as it progresses to see if it becomes necessary to do so.
", "rawMarkdown": "@lewington We are unlikely to be adding another digit as this becomes vulnerable to unwanted leaderboard probing, but we do keep an eye on the competition as it progresses to see if it becomes necessary to do so.", "votes": 1 } ] }, { "id": 1101585, "postDate": "2020-12-04T03:38:41.117Z", "content": "Is there any past project (videos/code) with enough examples to learn all the basics of computer vision? Program written on pytorch would be most welcome.
", "rawMarkdown": "Is there any past project (videos/code) with enough examples to learn all the basics of computer vision? Program written on pytorch would be most welcome. ", "replies": [ { "id": 1107086, "postDate": "2020-12-09T11:29:40.037Z", "content": "https://www.kaggle.com/c/plant-pathology-2020-fgvc7/discussion/154183
\nfound this competition. very similer to this.
Thank you for the resources!
", "rawMarkdown": "Thank you for the resources!" }, { "id": 1136137, "postDate": "2021-01-02T18:56:42.637Z", "content": "great thanks!
", "rawMarkdown": "great thanks!" }, { "id": 1135987, "postDate": "2021-01-02T16:50:43.120Z", "content": "Thanks for suggestions.
", "rawMarkdown": "Thanks for suggestions." } ], "comments": [ { "id": 1085046, "author_name": "Rajat Chaudhari", "author_url": "", "post_date": "2020-11-20T16:21:30.780000", "content": "How do majority of people start? Do they download the entire data set on their local machine? Or do they use the Kaggle notebook to work with the data set? Cause 6GB is huge..
", "votes": 4, "replies": [ { "id": 1086728, "author_name": "Anant Gupta", "author_url": "", "post_date": "2020-11-22T02:05:03.953000", "content": "What is need of downloading dataset when all you need is on kaggle. You leverage GPU and TPU also, without any impact on your hardware. Plus you show your work to others and contribute to ML AND DL society.
", "votes": 8, "replies": [] }, { "id": 1090622, "author_name": "Mau Rua", "author_url": "", "post_date": "2020-11-25T13:39:33.993000", "content": "The majority of people start with the kaggle courses to learn data science and machine learning.
\nWe use the Kaggle notebook to work with the data set on the website, and we usually don't download the entire dat set to work on our local machines because there is no need for that when you can just work on the cloud and run it on their servers that are more powerful in terms on GPU and TPU.
", "votes": 3, "replies": [] }, { "id": 1093098, "author_name": "vijay kumar", "author_url": "", "post_date": "2020-11-27T12:49:17.660000", "content": "no we import data sets using kaggle API .once check it out
", "votes": 2, "replies": [] }, { "id": 1093146, "author_name": "Emretheus", "author_url": "", "post_date": "2020-11-27T13:39:54.073000", "content": "You can also use Kaggle notebook or you can use the dataset in environments such as google colab using the kaggle api.
\nYou can find below resources related this topic.
https://medium.com/@galhever/how-to-import-data-from-kaggle-to-google-colab-8160caa11e2
\nhttps://colab.research.google.com/github/corrieann/kaggle/blob/master/kaggle_api_in_colab.ipynb
I'm having some issues with submission and I could use some help.
\nHowever when I try to submit I get this error…
\n\n\nYour Notebook cannot use internet access in this competition. Please disable internet in the Notebook editor and save a new version
\n
looking through the codebook, this is from this line no longer works
\n\n\n1 GCS_DS_PATH = KaggleDatasets().get_gcs_path('cassava-leaf-disease-classification') # you can list the bucket with \"!gsutil ls $GCS_DS_PATH\"
\nConnectionError: Connection error trying to communicate with service.
\n
So how do I read the test filenames if I have to switch off Internet Access?
", "votes": 1, "replies": [ { "id": 1193927, "author_name": "Claton Hendricks", "author_url": "", "post_date": "2021-02-10T01:27:41.197000", "content": "Am in the same boat, did you figure out how to do this w/o an internet connection?
", "votes": 0, "replies": [] }, { "id": 1196621, "author_name": "Kwez Baba", "author_url": "", "post_date": "2021-02-11T14:10:31.393000", "content": "Yes I figured it out. I changed the file location from
\nGCS_DS_PATH = KaggleDatasets().get_gcs_path
to
\n'/kaggle/input/'
", "votes": 0, "replies": [] } ] }, { "id": 1122004, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2020-12-22T06:01:46.023000", "content": "Can the public leaderboard please show up to 4-5 digits of the scores?
", "votes": 1, "replies": [] }, { "id": 1121918, "author_name": "kmostafavi3", "author_url": "", "post_date": "2020-12-22T02:49:28.470000", "content": "are there examples of doing cross validation with pytorch?
", "votes": 0, "replies": [] }, { "id": 3013668, "author_name": "ibrahim hamouda", "author_url": "", "post_date": "2024-10-10T11:56:48.293000", "content": "I am a product manager and started a master's in data science, I don't have a technical background and my program is demanding.
\nShould I quit the job and focus on my master's to be able to make tangible progress? any experience or advice is highly appreciated
How is it finds well for a data scientist?
", "votes": 0, "replies": [] }, { "id": 1672275, "author_name": "WooyoungLi", "author_url": "", "post_date": "2022-02-02T01:48:47.247000", "content": "good work. cool ……………..
", "votes": 0, "replies": [] }, { "id": 1277618, "author_name": "IlikeCat", "author_url": "", "post_date": "2021-04-19T02:09:29.623000", "content": "cool………..
", "votes": 0, "replies": [] }, { "id": 1211078, "author_name": "zhangeng", "author_url": "", "post_date": "2021-02-19T23:48:01.920000", "content": "Why doesn't my score show up in the leaderboard? I submitted 173!
", "votes": 0, "replies": [] }, { "id": 1206339, "author_name": "Mariusz Bronowicki", "author_url": "", "post_date": "2021-02-17T08:31:59.457000", "content": "I've just done my first simple CNN and tried to submit it to the competition. Could someone check my notebook and let me know what is wrong. Huge Thanks
\nhttps://www.kaggle.com/godzill22/casava-leaves-comp-first-cnn
I manage to do it
", "votes": 1, "replies": [] } ] }, { "id": 1206220, "author_name": "akaneko922", "author_url": "", "post_date": "2021-02-17T07:54:35.993000", "content": "I want to do ensemble learning. Must I open the submissions to use? Or I can do it in private.
", "votes": 0, "replies": [] }, { "id": 1194203, "author_name": "Feroz Khan", "author_url": "", "post_date": "2021-02-10T05:02:29.560000", "content": "Hello, hope y'all doin good. I have a doubt here. how do I move the images from 'train_images\" to different subfolders based on the image_id given in the csv file? like I want all the images with id '0' to be placed in one folder. can anyone help me with this?
", "votes": 0, "replies": [ { "id": 1197897, "author_name": "Vaibhav Rathi", "author_url": "", "post_date": "2021-02-12T14:18:40.243000", "content": "You will have to write a utility function to do that. You can try by reading the csv file; using the image_id to create the source path and creating the destination path by using the label value. Further you can use shutil to copy images from source path to destination path.
", "votes": 1, "replies": [] } ] }, { "id": 1191934, "author_name": "EmericDavid", "author_url": "", "post_date": "2021-02-08T19:30:34.170000", "content": "Is there a reason why Effnet and Resnet are so popular in this competition, and densenet doesn't show up much ?
", "votes": 0, "replies": [] }, { "id": 1184898, "author_name": "atia", "author_url": "", "post_date": "2021-02-03T20:53:57.837000", "content": "Hi Kagglers!
\nCould someone help me on how to submit my submission file properly. I keep getting the error submission score error . This is my first ever work on kaggle and I finding it quite difficult to traverse around.
\nI have attached a screenshot of the error and code used to create the submission.csv file
@atiaisaac click on the I buttom for more information about the error, your CSV file is in the format expected by competition
", "votes": 0, "replies": [] } ] }, { "id": 1168290, "author_name": "Iko_here", "author_url": "", "post_date": "2021-01-24T20:43:09.907000", "content": "Could you please help me in
\n1- reading the following data-set https://www.kaggle.com/c/cassava-leaf-disease-classification and shape it as:
\n (X_train, y_train), (X_test, y_test)
\n2- applying nr.1 in two cases after balancing the data with augmentation. And without balancing the data
I have ran out of space to upload more models, \"allocated memory ran out\". Is there a way to relocate this to somewhere else? Where are the files saved if I am working on kaggle kernels? Quite new to using kaggle's kernels and how they operate. Thanks!
", "votes": 0, "replies": [] }, { "id": 1155091, "author_name": "C.p. Fulwani", "author_url": "", "post_date": "2021-01-16T07:45:53.353000", "content": "i am using pretrained weights from keras pretrained model dataset for ResNet50. i have saved the weights to directory \"~/.keras/models\" . but when i am using that in my pretrained model. it is redownloading weight and following error is given.
\nError :
\n\"A local file was found, but it seems to be incomplete or outdated because the auto file hash does not match the original value of 4d473c1dd8becc155b73f8504c6f6626 so we will re-download the data.\"
\nPlease suggest what i have done wrong.
Can you share code
", "votes": 1, "replies": [] }, { "id": 1155112, "author_name": "C.p. Fulwani", "author_url": "", "post_date": "2021-01-16T08:06:09.280000", "content": "from tensorflow.keras.applications import ResNet50
\nfrom tensorflow.keras.applications.resnet50 import preprocess_input
\nfrom os import makedirs
\nfrom os.path import join, exists, expanduser
cache_dir = expanduser(join('~', '.keras'))
\nif not exists(cache_dir):
\n makedirs(cache_dir)
\nmodels_dir = join(cache_dir, 'models')
\nif not exists(models_dir):
\n makedirs(models_dir)
!cp ../input/resnet50/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5 ~/.keras/models/
\n!cp ../input/resnet50/imagenet_class_index.json ~/.keras/models/
\n!ls ~/.keras/models
base_model = ResNet50(weights=\"imagenet\", include_top=False)
\nbase_model.trainable = False
It is downloading weights because you have put weights=\"imagenet\", if you have added weights then put the path instead
i tried that but it is showing another error.
\nError:
\nshape [1,1,128,512] and [512,256,1,1] is incompatible
Thanks @anantgupt for help. i found the problem. i was actually using an older version of keras pretrained weight dataset.
", "votes": 0, "replies": [] }, { "id": 1155510, "author_name": "Anant Gupta", "author_url": "", "post_date": "2021-01-16T12:40:27.527000", "content": "Glad I can be helpful 😃
", "votes": 2, "replies": [] } ] }, { "id": 1154113, "author_name": "C.p. Fulwani", "author_url": "", "post_date": "2021-01-15T12:12:46.533000", "content": "As internet access is not allowed then how will I use pretrained model?
", "votes": 0, "replies": [ { "id": 1154135, "author_name": "Anant Gupta", "author_url": "", "post_date": "2021-01-15T12:33:24.613000", "content": "There a number of dataset containing model weight. You can add those datasets otherwise you can create 2 seperate notebook and upload your model in dataset and then use it.
", "votes": 2, "replies": [] }, { "id": 1154140, "author_name": "C.p. Fulwani", "author_url": "", "post_date": "2021-01-15T12:37:27.730000", "content": "Thanks for help.
", "votes": 1, "replies": [] } ] }, { "id": 1152899, "author_name": "Pouya Mofidi", "author_url": "", "post_date": "2021-01-14T14:53:28.523000", "content": "Helpful 🙏
", "votes": 0, "replies": [] }, { "id": 1152224, "author_name": "SamuelOrtiz9", "author_url": "", "post_date": "2021-01-14T00:18:05.487000", "content": "How do I submit my attempt?
", "votes": 0, "replies": [] }, { "id": 1145899, "author_name": "Yuval Navot", "author_url": "", "post_date": "2021-01-09T12:18:19.707000", "content": "Hi everyone
\nGot error on my submissions \"Submission Scoring Error\", worked fine on my local PC …
\nHow can I review the issues and fix? is there model runtime limitation? mine took ~4.5H on my local PC
Thanks
\nYuval.
Hi everyone
\nAny one use tensorboard to review the runs?
\ntry few steps but seems that not working, wandering if this enabled in kaggle environment or/and my wrong usage
model2.compile(
\n optimizer='adam',
\n loss='categorical_crossentropy',
\n metrics=['accuracy'])
RUN_NAME = 'run 1 with 25 nodes'
\nlogger = keras.callbacks.TensorBoard(
\n log_dir='logs/{}'.format(RUN_NAME),
\n histogram_freq=5,
\n write_graph=True
\n)
history = model2.fit_generator(train_datagen_flow,
\n validation_data=valid_datagen_flow,
\n epochs=1,
\n callbacks=[logger]
\n )
tensoboard --logdir=logs
\nthanks
\nYuval
Yes, Kaggle is a good place for beginners like as me. it was better if it had a road map and better courses. For become a data scientist Kaggle is necessity but not enough. I wish they make courses better and more useful.
", "votes": 0, "replies": [] }, { "id": 1140660, "author_name": "Yuval Navot", "author_url": "", "post_date": "2021-01-06T07:04:26.423000", "content": "Hi,
\nthis is my first competition so have basic question :-)
\nOnce I'm submitting (save & Commit) is it take time to find mine in the leaderboard?
thanks
\nYuval.
By save and commit you are not submitting your work save and saves the notebook for that purpose it runs completely again if your notebook took 1 hr while training then again it will run for 1 hr (when you save and commit it). It will changes in your notebook publically. For submitting to competition you have submit predictions go the my submission button
", "votes": 3, "replies": [] }, { "id": 1140975, "author_name": "Yuval Navot", "author_url": "", "post_date": "2021-01-06T12:12:42.943000", "content": "Thank you Anant
", "votes": 1, "replies": [] } ] }, { "id": 1132687, "author_name": "Enric Domingo", "author_url": "", "post_date": "2020-12-30T16:01:02.340000", "content": "Is this competition a good start point after finishing the Deep Learning and Computer Vision courses and the previous ones? Is there any other Kaggle course specially recommended for this competition (a part from the specific TPU tutorial)? Thank you!
", "votes": 0, "replies": [] }, { "id": 1130667, "author_name": "邹邹大魔王", "author_url": "", "post_date": "2020-12-29T08:03:42.243000", "content": "Thanks, your suggestions are very helpful to novices
", "votes": 0, "replies": [] }, { "id": 1121915, "author_name": "kmostafavi3", "author_url": "", "post_date": "2020-12-22T02:46:35.157000", "content": "I see many examples of using keras on tpu. Are there examples of using pytorch on tpu? Can one create a model on tpu and then load it on gpu or vice-versa?
", "votes": 0, "replies": [] }, { "id": 1118287, "author_name": "Erik Larsen", "author_url": "", "post_date": "2020-12-18T22:59:15.367000", "content": "Hi, everyone! I'm new to Kaggle and this is my first post and my first competition. I am stuck on making a submission and having it scored. When I submit, all of the metrics say that everything ran properly, but the status continually says running until it times out at 9 hours. Should I be importing the trained model and then making the submission with the test file path in a different notebook?
", "votes": 0, "replies": [ { "id": 1121924, "author_name": "kmostafavi3", "author_url": "", "post_date": "2020-12-22T02:56:26.910000", "content": "does your notebook run more than 9 hours? this may happen if you are running your notebook on your own GPU (say aws rented GPU). Kaggle GPU only allows your notebook to run 9 hours and after that it kills it. You may want to create a test only notebook and then upload your created model as a new dataset. Then you submit the test only notebook to the competition. The results may take as much as 10 or 15 minutes to be available.
", "votes": 1, "replies": [] } ] }, { "id": 1117400, "author_name": "Nguyen Thanh Nghi", "author_url": "", "post_date": "2020-12-18T02:36:01.923000", "content": "How can I use keras.applications.EfficientNet in TPU mode ?, I found that it's working in GPU but can not import in GPU
", "votes": 0, "replies": [ { "id": 1121146, "author_name": "José Moyá", "author_url": "", "post_date": "2020-12-21T12:10:05.590000", "content": "In TPU mode the TensorFlow version changes to 2.2 which does not include keras.applications.EfficientNet
\nYou can do something like this:
\n!pip install -U efficientnet\n\nimport efficientnet.keras as efn\n\nbase_model = efn.EfficientNetB6(include_top=False, weights=\"imagenet\")\n