Hello everyone!
\nI apologize for publishing my best result, but I doubt that given the recent events I will have the time and desire to continue working. But I do not want my work to be wasted. I added to this dataset https://www.kaggle.com/aikhmelnytskyy/eff7-new-768 my colab notebook in which I trained models. Good luck to you all!
\nMy training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093
\nP.S.
\nI believe we will win!
\nСлава Україні!
Hello everyone!
\nI apologize for publishing my best result, but I doubt that given the recent events I will have the time and desire to continue working. But I do not want my work to be wasted. I added to this dataset https://www.kaggle.com/aikhmelnytskyy/eff7-new-768 my colab notebook in which I trained models. Good luck to you all!
\nMy training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093
\nP.S.
\nI believe we will win!
\nСлава Україні!
Thanks !!
\n… and I wish you all the best for all that happens outside of Kaggle :(
", "rawMarkdown": "Thanks !! \n\n... and I wish you all the best for all that happens outside of Kaggle :(\n\n", "votes": 6 }, { "id": 1708233, "postDate": "2022-03-01T09:00:40.637Z", "content": "is anyone able to reproduce this result in pytorch?
", "rawMarkdown": "is anyone able to reproduce this result in pytorch?", "votes": 4, "replies": [ { "id": 1708870, "postDate": "2022-03-01T18:53:42.143Z", "content": "That would be really helpful
", "rawMarkdown": "That would be really helpful", "votes": 1 }, { "id": 1712399, "postDate": "2022-03-04T22:50:25.370Z", "content": "Will try to work on it. 👌
", "rawMarkdown": "Will try to work on it. 👌", "votes": 1 }, { "id": 1717639, "postDate": "2022-03-10T05:11:47.007Z", "content": "yes. Pytorch works, I also used fastai.
", "rawMarkdown": "yes. Pytorch works, I also used fastai.", "votes": 1 } ] }, { "id": 1733098, "postDate": "2022-03-24T02:21:43.327Z", "content": "I have tested your model on fold4 and I find that if the model load your weights at epoch 9,the score seems always higher than the model trained from the beginning.(loading just noicy-student weights) Which is 71.4 to 69.7. I wonder why the previous score is always higher than the latter, how do you get your weights at epoch 9? Thank you!
", "rawMarkdown": "I have tested your model on fold4 and I find that if the model load your weights at epoch 9,the score seems always higher than the model trained from the beginning.(loading just noicy-student weights) Which is 71.4 to 69.7. I wonder why the previous score is always higher than the latter, how do you get your weights at epoch 9? Thank you!\n", "votes": 1, "replies": [ { "id": 1733623, "postDate": "2022-03-24T13:00:26.547Z", "content": "Hi there! Look at the code, the Snapshot class saves the model for the 5th and 8th epochs. If you download and train from the 8th epoch, then in general you train more epochs, so the accuracy may increase as in your case.
", "rawMarkdown": "Hi there! Look at the code, the Snapshot class saves the model for the 5th and 8th epochs. If you download and train from the 8th epoch, then in general you train more epochs, so the accuracy may increase as in your case." } ] }, { "id": 1721253, "postDate": "2022-03-13T14:38:39.413Z", "content": "By the way, you and I have the same city :)
", "rawMarkdown": "By the way, you and I have the same city :)", "votes": 1 }, { "id": 1711321, "postDate": "2022-03-03T20:44:16.687Z", "content": "Congratulations @aikhmelnytskyy, and I hope you all the best.
", "rawMarkdown": "Congratulations @aikhmelnytskyy, and I hope you all the best.", "votes": 1 }, { "id": 1710228, "postDate": "2022-03-02T20:28:14.513Z", "content": "very good congrats
", "rawMarkdown": "very good congrats ", "votes": 1 }, { "id": 1708825, "postDate": "2022-03-01T18:21:04.317Z", "content": "Congratulations
", "rawMarkdown": "Congratulations", "votes": 1 }, { "id": 1708563, "postDate": "2022-03-01T14:36:09.933Z", "content": "Congratulations, I respect your excellent work!
", "rawMarkdown": "Congratulations, I respect your excellent work!", "votes": 1 }, { "id": 1708413, "postDate": "2022-03-01T12:45:06.097Z", "content": "Congratulations @aikhmelnytskyy
", "rawMarkdown": "Congratulations @aikhmelnytskyy ", "votes": 1 }, { "id": 1708246, "postDate": "2022-03-01T09:25:21.747Z", "content": "Congrats!!
", "rawMarkdown": "Congrats!!", "votes": 1 }, { "id": 1708211, "postDate": "2022-03-01T08:17:49.597Z", "content": "Congratulations
", "rawMarkdown": "Congratulations", "votes": 1 }, { "id": 1708194, "postDate": "2022-03-01T07:38:46.053Z", "content": "execllent，good score
", "rawMarkdown": "execllent，good score", "votes": 1 }, { "id": 1708154, "postDate": "2022-03-01T06:49:02.147Z", "content": "Congrats amazing work
", "rawMarkdown": "Congrats amazing work\n", "votes": 1 }, { "id": 1708127, "postDate": "2022-03-01T06:12:20.407Z", "content": "Congrats @aikhmelnytskyy 💛💙
", "rawMarkdown": "Congrats @aikhmelnytskyy 💛💙", "votes": 1 }, { "id": 1708038, "postDate": "2022-03-01T04:20:33.203Z", "content": "Congradullation, good！
", "rawMarkdown": "Congradullation, good！", "votes": 1 }, { "id": 1707626, "postDate": "2022-02-28T16:17:22.497Z", "content": "Congrats! :D
", "rawMarkdown": "Congrats! :D", "votes": 1 }, { "id": 1707522, "postDate": "2022-02-28T14:41:48.370Z", "content": "Congradullation, good score
", "rawMarkdown": "Congradullation, good score", "votes": 1 }, { "id": 1706935, "postDate": "2022-02-28T01:56:56.217Z", "content": "Thanks and Wish you all the best !
", "rawMarkdown": "Thanks and Wish you all the best !", "votes": 1, "replies": [ { "id": 1731272, "postDate": "2022-03-22T07:29:58.440Z", "content": "你需要队友吗，我们可以一起组队吗👀
", "rawMarkdown": "你需要队友吗，我们可以一起组队吗👀" } ] }, { "id": 1706724, "postDate": "2022-02-27T18:18:27.097Z", "content": "Interesting, how did you blend your models by folds? Concated all embeddings by fold or used mean/median?
", "rawMarkdown": "Interesting, how did you blend your models by folds? Concated all embeddings by fold or used mean/median? ", "votes": 1, "replies": [ { "id": 1706801, "postDate": "2022-02-27T20:19:37.303Z", "content": "I used mean. Please check this func get_embeddings_np
", "rawMarkdown": "I used mean. Please check this func get_embeddings_np", "votes": 1 }, { "id": 1706924, "postDate": "2022-02-28T01:32:41.057Z", "content": "Oh. I got it. Thank you. I saw model embed size is 512, but numpy dump embed size is 2048. It confused me.
", "rawMarkdown": "Oh. I got it. Thank you. I saw model embed size is 512, but numpy dump embed size is 2048. It confused me.\n", "votes": 1 }, { "id": 1707030, "postDate": "2022-02-28T04:49:37.153Z", "content": "So what is the img size while inference?
", "rawMarkdown": "So what is the img size while inference?", "votes": -1 }, { "id": 1707041, "postDate": "2022-02-28T05:03:45.823Z", "content": "maybe changed the network :
\n embed = tf.keras.layers.Dropout(0.2)(embed)\n embed = tf.keras.layers.Dense(512)(embed)\n x = margin([embed, label])\n512->2048, that is the point.
\nAll my guess, if not, please let me know, thanks!
Update: the guess is wrong, when I tried using b7 model to load weights in effb7-new-768 kaggle dataset, mismatch error occurs:
\nValueError: You are trying to load a weight file containing 438 layers into a model with 3 layers.\nguys i see i confused you a bit. to see my model go to the dataset I specified and open the colab notebook file (upload to colab or kaggle). The published notebook does not use the model, I just download the results of model predictions in npy format. Train img size - 600
", "rawMarkdown": "guys i see i confused you a bit. to see my model go to the dataset I specified and open the colab notebook file (upload to colab or kaggle). The published notebook does not use the model, I just download the results of model predictions in npy format. Train img size - 600", "votes": 2 }, { "id": 1707102, "postDate": "2022-02-28T06:11:25.780Z", "content": "my training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093
", "rawMarkdown": "my training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093", "votes": 1 }, { "id": 1707118, "postDate": "2022-02-28T06:30:52.450Z", "content": "Thanks for your sharing,👍 now I understood.
", "rawMarkdown": "Thanks for your sharing,👍 now I understood.", "votes": 1 } ] }, { "id": 1706698, "postDate": "2022-02-27T17:51:08.380Z", "content": "thx for sharing. how long did you train use Colab for single fold? Colab pro or Colab pro+?
\nhow It takes for inference?
", "rawMarkdown": "thx for sharing. how long did you train use Colab for single fold? Colab pro or Colab pro+?\n\nhow It takes for inference?", "votes": 1, "replies": [ { "id": 1706713, "postDate": "2022-02-27T18:04:18.960Z", "content": "For image size 600 - 1122 sec / epoch on Colab
", "rawMarkdown": "For image size 600 - 1122 sec / epoch on Colab", "votes": 2 }, { "id": 1707087, "postDate": "2022-02-28T05:56:35.930Z", "content": "colab pro, pro + and normal all get the same TPUS :/ GPU is vastly better in pro+ though
", "rawMarkdown": "colab pro, pro + and normal all get the same TPUS :/ GPU is vastly better in pro+ though\n", "votes": 1 } ] }, { "id": 1726032, "postDate": "2022-03-17T15:39:01.013Z", "content": "I edited some of this code, and tried with a efficient_v2 implementation using https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/efficientnet and got a 0.699 with a V2M (pretrained on imagenet) and currently trying a V2XL (pretrained on imagenet21k-ft1k). Will let you know how it goes. These notebooks still confuse me on what is happening where, but if I can change the model maybe I have a chance.
", "rawMarkdown": "I edited some of this code, and tried with a efficient_v2 implementation using https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/efficientnet and got a 0.699 with a V2M (pretrained on imagenet) and currently trying a V2XL (pretrained on imagenet21k-ft1k). Will let you know how it goes. These notebooks still confuse me on what is happening where, but if I can change the model maybe I have a chance.", "votes": 2, "replies": [ { "id": 1726044, "postDate": "2022-03-17T15:49:22.933Z", "content": "Hi there! My training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-training. In the latest version, I downloaded weights from the previous version, because at the time I was training there was a bug on the kaggle. Check the notebook again, it should be: RESUME = False
\n RESUME_EPOCH = None, otherwise we resume learning from the previous checkpoint instead of starting from the beginning. If the scales are loaded, there is a problem with this code:
\n if config.model_type == 'effnetv1':
\n inp = EFNS [config.EFF_NET] (weights = 'noisy-student', include_top = False, input_shape = [config.IMAGE_SIZE, config.IMAGE_SIZE, 3])
\n inp.layers [0] ._ name = 'inp1'
\n # inp.summary ()
\n x1 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-1] .output)
x2 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-5] .output)\n x3 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-7] .output)\n x4 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-13] .output)\n embed = tf.concat ([x1, x2, x3, x4], axis = 1)\n #embed = tf.keras.layers.GlobalAveragePooling2D () (x)\n elif config.model_type == 'effnetv2':\n FEATURE_VECTOR = f '{EFFNETV2_ROOT} / tfhub_models / efficientnetv2- {config.EFF_NETV2} / feature_vector'\nOh my goodness. I didn't even notice that had been happening. I don't think the 'scales are loaded', but I could be wrong. How would I tell the difference?
\nEither way, thank you for getting back to me about the code. I'll update RESUME_EPOCH and RESUME to the correct values.
", "rawMarkdown": "Oh my goodness. I didn't even notice that had been happening. I don't think the 'scales are loaded', but I could be wrong. How would I tell the difference?\n\nEither way, thank you for getting back to me about the code. I'll update RESUME_EPOCH and RESUME to the correct values." }, { "id": 1726192, "postDate": "2022-03-17T18:16:36.913Z", "content": "Update: V2XL got 0.729, so that's good. I'm going to try to ensemble and see how that goes, then start working on doing some data labeling
", "rawMarkdown": "Update: V2XL got 0.729, so that's good. I'm going to try to ensemble and see how that goes, then start working on doing some data labeling", "votes": 2 }, { "id": 1726199, "postDate": "2022-03-17T18:34:27.533Z", "content": "That's impressive for a single model. May I ask what image size are you using? And your result is single fold or all 5 folds?
", "rawMarkdown": "That's impressive for a single model. May I ask what image size are you using? And your result is single fold or all 5 folds?", "votes": 1 }, { "id": 1726206, "postDate": "2022-03-17T18:45:28.290Z", "content": "So I basically took this notebook, then rewrote the model so it was using V2XL (the largest efficientNet possible), then submitted. I think this notebook was using 5 folds, so that's what I did as well.
\nWhat does folding do? I hear a lot of people talk about it, but not sure what they mean or how it impacts results
", "rawMarkdown": "So I basically took this notebook, then rewrote the model so it was using V2XL (the largest efficientNet possible), then submitted. I think this notebook was using 5 folds, so that's what I did as well.\n\nWhat does folding do? I hear a lot of people talk about it, but not sure what they mean or how it impacts results", "votes": 2 }, { "id": 1726213, "postDate": "2022-03-17T19:02:16.377Z", "content": "Search the internet for folding there are many articles, posts and more even on kaggle. I think you will find a perfect explanation, even better than I can explain)
", "rawMarkdown": "Search the internet for folding there are many articles, posts and more even on kaggle. I think you will find a perfect explanation, even better than I can explain)", "votes": 1 }, { "id": 1727559, "postDate": "2022-03-18T04:02:57.797Z", "content": "When you split the dataset into 5 folds, training on (for example) fold 0 means that you take fold 0 as validation set and use other folds (fold 1, 2, 3, 4) as training set. So a model trained on a single fold would only make use of 4/5 of the whole dataset as training set. This means if you train 5 models on 5 folds and ensemble them together, you would have a slightly better result than each single model.
", "rawMarkdown": "When you split the dataset into 5 folds, training on (for example) fold 0 means that you take fold 0 as validation set and use other folds (fold 1, 2, 3, 4) as training set. So a model trained on a single fold would only make use of 4/5 of the whole dataset as training set. This means if you train 5 models on 5 folds and ensemble them together, you would have a slightly better result than each single model. ", "votes": 1 } ] }, { "id": 1721246, "postDate": "2022-03-13T14:36:44.177Z", "content": "Congratilation! Glory to heroes(Героям Слава!)
", "rawMarkdown": "Congratilation! Glory to heroes(Героям Слава!)" }, { "id": 1707994, "postDate": "2022-03-01T02:32:59.497Z", "content": "ValueError Traceback (most recent call last)
\n/tmp/ipykernel_43/3601991566.py in
\n 9 test_data_list.append([filename,embeddings])
\n 10 ids = get_ids(filename)
\n---> 11 distances,idxs = neigh.kneighbors(embeddings, config.KNN, return_distance=True)
\n 12 test_ids.append(ids)
\n 13 test_nn_idxs.append(idxs)
/opt/conda/lib/python3.7/site-packages/sklearn/neighbors/_base.py in kneighbors(self, X, n_neighbors, return_distance)
\n 604 X = _check_precomputed(X)
\n 605 else:
\n--> 606 X = check_array(X, accept_sparse='csr')
\n 607 else:
\n 608 query_is_train = True
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in inner_f(*args, kwargs)\n 70 FutureWarning)\n 71 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})\n---> 72 return f(kwargs)
\n 73 return inner_f
\n 74
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
\n 643 if force_all_finite:
\n 644 _assert_all_finite(array,
\n--> 645 allow_nan=force_all_finite == 'allow-nan')
\n 646
\n 647 if ensure_min_samples > 0:
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in _assert_all_finite(X, allow_nan, msg_dtype)
\n 97 msg_err.format
\n 98 (type_err,
\n---> 99 msg_dtype if msg_dtype is not None else X.dtype)
\n 100 )
\n 101 # for object dtype data, we only check for NaNs (GH-13254)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').
", "rawMarkdown": "---------------------------------------------------------------------------\nValueError Traceback (most recent call last)\n/tmp/ipykernel_43/3601991566.py inThank you fr sharing @aikhmelnytskyy. Good score!
\nСлава Україні!
@aikhmelnytskyy whats the final validation loss and CV for this result?
", "rawMarkdown": "@aikhmelnytskyy whats the final validation loss and CV for this result?", "votes": -1 }, { "id": 1707998, "postDate": "2022-03-01T02:36:15.890Z", "content": "You can't reason with the weights you provide！Am I using it incorrectly?
\nmodel.load_weights('../input/happywhale-effnet-b7-fork-with-detic-crop-3b3093/effnetv1_b7_loss_4.h5')
", "rawMarkdown": "You can't reason with the weights you provide！Am I using it incorrectly?\n\nmodel.load_weights('../input/happywhale-effnet-b7-fork-with-detic-crop-3b3093/effnetv1_b7_loss_4.h5')", "votes": -1 }, { "id": 1707993, "postDate": "2022-03-01T02:32:22.533Z", "content": "Using your change network training is fine, but does reasoning get an error?
", "rawMarkdown": "Using your change network training is fine, but does reasoning get an error?", "votes": -1 }, { "id": 1707992, "postDate": "2022-03-01T02:31:54.140Z", "content": "ValueError Traceback (most recent call last)
\n/tmp/ipykernel_43/3601991566.py in
\n 9 test_data_list.append([filename,embeddings])
\n 10 ids = get_ids(filename)
\n---> 11 distances,idxs = neigh.kneighbors(embeddings, config.KNN, return_distance=True)
\n 12 test_ids.append(ids)
\n 13 test_nn_idxs.append(idxs)
Not torch, sad
", "rawMarkdown": "Not torch, sad", "votes": -5 }, { "id": 2109657, "postDate": "2023-01-21T15:06:17.537Z", "content": "Congratulations, I respect your excellent work!I hope you will have better grades in the future！
", "rawMarkdown": "Congratulations, I respect your excellent work!I hope you will have better grades in the future！" }, { "id": 1743806, "postDate": "2022-04-03T10:17:34.020Z", "content": "Lightning+timm
\nconvnext_small 384
\nAug.
\nSingle fold:0.711 (PB)
Thanks for sharing and congratulations
", "rawMarkdown": "Thanks for sharing and congratulations", "votes": 1 }, { "id": 1711296, "postDate": "2022-03-03T19:52:55.467Z", "content": "Thank you for sharing!
", "rawMarkdown": "Thank you for sharing!", "votes": 1 } ], "comments": [ { "id": 1706813, "author_name": "FabienDaniel", "author_url": "", "post_date": "2022-02-27T20:48:41.840000", "content": "Thanks !!
\n… and I wish you all the best for all that happens outside of Kaggle :(
", "votes": 6, "replies": [] }, { "id": 1708233, "author_name": "DeepUnderstanding", "author_url": "", "post_date": "2022-03-01T09:00:40.637000", "content": "is anyone able to reproduce this result in pytorch?
", "votes": 4, "replies": [ { "id": 1708870, "author_name": "Chet", "author_url": "", "post_date": "2022-03-01T18:53:42.143000", "content": "That would be really helpful
", "votes": 1, "replies": [] }, { "id": 1712399, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2022-03-04T22:50:25.370000", "content": "Will try to work on it. 👌
", "votes": 1, "replies": [] }, { "id": 1717639, "author_name": "Hao He", "author_url": "", "post_date": "2022-03-10T05:11:47.007000", "content": "yes. Pytorch works, I also used fastai.
", "votes": 1, "replies": [] } ] }, { "id": 1733098, "author_name": "", "author_url": "", "post_date": "2022-03-24T02:21:43.327000", "content": "I have tested your model on fold4 and I find that if the model load your weights at epoch 9,the score seems always higher than the model trained from the beginning.(loading just noicy-student weights) Which is 71.4 to 69.7. I wonder why the previous score is always higher than the latter, how do you get your weights at epoch 9? Thank you!
", "votes": 1, "replies": [ { "id": 1733623, "author_name": "Andrij", "author_url": "", "post_date": "2022-03-24T13:00:26.547000", "content": "Hi there! Look at the code, the Snapshot class saves the model for the 5th and 8th epochs. If you download and train from the 8th epoch, then in general you train more epochs, so the accuracy may increase as in your case.
", "votes": 0, "replies": [] } ] }, { "id": 1721253, "author_name": "", "author_url": "", "post_date": "2022-03-13T14:38:39.413000", "content": "By the way, you and I have the same city :)
", "votes": 1, "replies": [] }, { "id": 1711321, "author_name": "Marcelo Baltar", "author_url": "", "post_date": "2022-03-03T20:44:16.687000", "content": "Congratulations @aikhmelnytskyy, and I hope you all the best.
", "votes": 1, "replies": [] }, { "id": 1710228, "author_name": "tom", "author_url": "", "post_date": "2022-03-02T20:28:14.513000", "content": "very good congrats
", "votes": 1, "replies": [] }, { "id": 1708825, "author_name": "Ishan Mehta115", "author_url": "", "post_date": "2022-03-01T18:21:04.317000", "content": "Congratulations
", "votes": 1, "replies": [] }, { "id": 1708563, "author_name": "tock", "author_url": "", "post_date": "2022-03-01T14:36:09.933000", "content": "Congratulations, I respect your excellent work!
", "votes": 1, "replies": [] }, { "id": 1708413, "author_name": "cyberpiyu", "author_url": "", "post_date": "2022-03-01T12:45:06.097000", "content": "Congratulations @aikhmelnytskyy
", "votes": 1, "replies": [] }, { "id": 1708246, "author_name": "Shivam Taneja", "author_url": "", "post_date": "2022-03-01T09:25:21.747000", "content": "Congrats!!
", "votes": 1, "replies": [] }, { "id": 1708211, "author_name": "P SURESH", "author_url": "", "post_date": "2022-03-01T08:17:49.597000", "content": "Congratulations
", "votes": 1, "replies": [] }, { "id": 1708194, "author_name": "Yip", "author_url": "", "post_date": "2022-03-01T07:38:46.053000", "content": "execllent，good score
", "votes": 1, "replies": [] }, { "id": 1708154, "author_name": "marioeid", "author_url": "", "post_date": "2022-03-01T06:49:02.147000", "content": "Congrats amazing work
", "votes": 1, "replies": [] }, { "id": 1708127, "author_name": "Niek van der Zwaag", "author_url": "", "post_date": "2022-03-01T06:12:20.407000", "content": "Congrats @aikhmelnytskyy 💛💙
", "votes": 1, "replies": [] }, { "id": 1708038, "author_name": "uuuuu9", "author_url": "", "post_date": "2022-03-01T04:20:33.203000", "content": "Congradullation, good！
", "votes": 1, "replies": [] }, { "id": 1707626, "author_name": "Gerardo Huerta Robles", "author_url": "", "post_date": "2022-02-28T16:17:22.497000", "content": "Congrats! :D
", "votes": 1, "replies": [] }, { "id": 1707522, "author_name": "Artem Burenok", "author_url": "", "post_date": "2022-02-28T14:41:48.370000", "content": "Congradullation, good score
", "votes": 1, "replies": [] }, { "id": 1706935, "author_name": "liuzhangzhen", "author_url": "", "post_date": "2022-02-28T01:56:56.217000", "content": "Thanks and Wish you all the best !
", "votes": 1, "replies": [ { "id": 1731272, "author_name": "huyahuya", "author_url": "", "post_date": "2022-03-22T07:29:58.440000", "content": "你需要队友吗，我们可以一起组队吗👀
", "votes": 0, "replies": [] } ] }, { "id": 1706724, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2022-02-27T18:18:27.097000", "content": "Interesting, how did you blend your models by folds? Concated all embeddings by fold or used mean/median?
", "votes": 1, "replies": [ { "id": 1706801, "author_name": "Andrij", "author_url": "", "post_date": "2022-02-27T20:19:37.303000", "content": "I used mean. Please check this func get_embeddings_np
", "votes": 1, "replies": [] }, { "id": 1706924, "author_name": "Sergey Bryansky", "author_url": "", "post_date": "2022-02-28T01:32:41.057000", "content": "Oh. I got it. Thank you. I saw model embed size is 512, but numpy dump embed size is 2048. It confused me.
", "votes": 1, "replies": [] }, { "id": 1707030, "author_name": "Time Master", "author_url": "", "post_date": "2022-02-28T04:49:37.153000", "content": "So what is the img size while inference?
", "votes": -1, "replies": [] }, { "id": 1707041, "author_name": "Time Master", "author_url": "", "post_date": "2022-02-28T05:03:45.823000", "content": "maybe changed the network :
\n embed = tf.keras.layers.Dropout(0.2)(embed)\n embed = tf.keras.layers.Dense(512)(embed)\n x = margin([embed, label])\n512->2048, that is the point.
\nAll my guess, if not, please let me know, thanks!
Update: the guess is wrong, when I tried using b7 model to load weights in effb7-new-768 kaggle dataset, mismatch error occurs:
\nValueError: You are trying to load a weight file containing 438 layers into a model with 3 layers.\nguys i see i confused you a bit. to see my model go to the dataset I specified and open the colab notebook file (upload to colab or kaggle). The published notebook does not use the model, I just download the results of model predictions in npy format. Train img size - 600
", "votes": 2, "replies": [] }, { "id": 1707102, "author_name": "Andrij", "author_url": "", "post_date": "2022-02-28T06:11:25.780000", "content": "my training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093
", "votes": 1, "replies": [] }, { "id": 1707118, "author_name": "Time Master", "author_url": "", "post_date": "2022-02-28T06:30:52.450000", "content": "Thanks for your sharing,👍 now I understood.
", "votes": 1, "replies": [] } ] }, { "id": 1706698, "author_name": "dragon zhang", "author_url": "", "post_date": "2022-02-27T17:51:08.380000", "content": "thx for sharing. how long did you train use Colab for single fold? Colab pro or Colab pro+?
\nhow It takes for inference?
", "votes": 1, "replies": [ { "id": 1706713, "author_name": "Andrij", "author_url": "", "post_date": "2022-02-27T18:04:18.960000", "content": "For image size 600 - 1122 sec / epoch on Colab
", "votes": 2, "replies": [] }, { "id": 1707087, "author_name": "Hannah B", "author_url": "", "post_date": "2022-02-28T05:56:35.930000", "content": "colab pro, pro + and normal all get the same TPUS :/ GPU is vastly better in pro+ though
", "votes": 1, "replies": [] } ] }, { "id": 1726032, "author_name": "Paul", "author_url": "", "post_date": "2022-03-17T15:39:01.013000", "content": "I edited some of this code, and tried with a efficient_v2 implementation using https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/efficientnet and got a 0.699 with a V2M (pretrained on imagenet) and currently trying a V2XL (pretrained on imagenet21k-ft1k). Will let you know how it goes. These notebooks still confuse me on what is happening where, but if I can change the model maybe I have a chance.
", "votes": 2, "replies": [ { "id": 1726044, "author_name": "Andrij", "author_url": "", "post_date": "2022-03-17T15:49:22.933000", "content": "Hi there! My training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-training. In the latest version, I downloaded weights from the previous version, because at the time I was training there was a bug on the kaggle. Check the notebook again, it should be: RESUME = False
\n RESUME_EPOCH = None, otherwise we resume learning from the previous checkpoint instead of starting from the beginning. If the scales are loaded, there is a problem with this code:
\n if config.model_type == 'effnetv1':
\n inp = EFNS [config.EFF_NET] (weights = 'noisy-student', include_top = False, input_shape = [config.IMAGE_SIZE, config.IMAGE_SIZE, 3])
\n inp.layers [0] ._ name = 'inp1'
\n # inp.summary ()
\n x1 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-1] .output)
x2 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-5] .output)\n x3 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-7] .output)\n x4 = tf.keras.layers.GlobalAveragePooling2D () (inp.layers [-13] .output)\n embed = tf.concat ([x1, x2, x3, x4], axis = 1)\n #embed = tf.keras.layers.GlobalAveragePooling2D () (x)\n elif config.model_type == 'effnetv2':\n FEATURE_VECTOR = f '{EFFNETV2_ROOT} / tfhub_models / efficientnetv2- {config.EFF_NETV2} / feature_vector'\nOh my goodness. I didn't even notice that had been happening. I don't think the 'scales are loaded', but I could be wrong. How would I tell the difference?
\nEither way, thank you for getting back to me about the code. I'll update RESUME_EPOCH and RESUME to the correct values.
", "votes": 0, "replies": [] }, { "id": 1726192, "author_name": "Paul", "author_url": "", "post_date": "2022-03-17T18:16:36.913000", "content": "Update: V2XL got 0.729, so that's good. I'm going to try to ensemble and see how that goes, then start working on doing some data labeling
", "votes": 2, "replies": [] }, { "id": 1726199, "author_name": "NaN", "author_url": "", "post_date": "2022-03-17T18:34:27.533000", "content": "That's impressive for a single model. May I ask what image size are you using? And your result is single fold or all 5 folds?
", "votes": 1, "replies": [] }, { "id": 1726206, "author_name": "Paul", "author_url": "", "post_date": "2022-03-17T18:45:28.290000", "content": "So I basically took this notebook, then rewrote the model so it was using V2XL (the largest efficientNet possible), then submitted. I think this notebook was using 5 folds, so that's what I did as well.
\nWhat does folding do? I hear a lot of people talk about it, but not sure what they mean or how it impacts results
", "votes": 2, "replies": [] }, { "id": 1726213, "author_name": "Andrij", "author_url": "", "post_date": "2022-03-17T19:02:16.377000", "content": "Search the internet for folding there are many articles, posts and more even on kaggle. I think you will find a perfect explanation, even better than I can explain)
", "votes": 1, "replies": [] }, { "id": 1727559, "author_name": "NaN", "author_url": "", "post_date": "2022-03-18T04:02:57.797000", "content": "When you split the dataset into 5 folds, training on (for example) fold 0 means that you take fold 0 as validation set and use other folds (fold 1, 2, 3, 4) as training set. So a model trained on a single fold would only make use of 4/5 of the whole dataset as training set. This means if you train 5 models on 5 folds and ensemble them together, you would have a slightly better result than each single model.
", "votes": 1, "replies": [] } ] }, { "id": 1721246, "author_name": "", "author_url": "", "post_date": "2022-03-13T14:36:44.177000", "content": "Congratilation! Glory to heroes(Героям Слава!)
", "votes": 0, "replies": [] }, { "id": 1707994, "author_name": "huyahuya", "author_url": "", "post_date": "2022-03-01T02:32:59.497000", "content": "ValueError Traceback (most recent call last)
\n/tmp/ipykernel_43/3601991566.py in
\n 9 test_data_list.append([filename,embeddings])
\n 10 ids = get_ids(filename)
\n---> 11 distances,idxs = neigh.kneighbors(embeddings, config.KNN, return_distance=True)
\n 12 test_ids.append(ids)
\n 13 test_nn_idxs.append(idxs)
/opt/conda/lib/python3.7/site-packages/sklearn/neighbors/_base.py in kneighbors(self, X, n_neighbors, return_distance)
\n 604 X = _check_precomputed(X)
\n 605 else:
\n--> 606 X = check_array(X, accept_sparse='csr')
\n 607 else:
\n 608 query_is_train = True
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in inner_f(*args, kwargs)\n 70 FutureWarning)\n 71 kwargs.update({k: arg for k, arg in zip(sig.parameters, args)})\n---> 72 return f(kwargs)
\n 73 return inner_f
\n 74
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, estimator)
\n 643 if force_all_finite:
\n 644 _assert_all_finite(array,
\n--> 645 allow_nan=force_all_finite == 'allow-nan')
\n 646
\n 647 if ensure_min_samples > 0:
/opt/conda/lib/python3.7/site-packages/sklearn/utils/validation.py in _assert_all_finite(X, allow_nan, msg_dtype)
\n 97 msg_err.format
\n 98 (type_err,
\n---> 99 msg_dtype if msg_dtype is not None else X.dtype)
\n 100 )
\n 101 # for object dtype data, we only check for NaNs (GH-13254)
ValueError: Input contains NaN, infinity or a value too large for dtype('float32').
", "votes": 0, "replies": [] }, { "id": 1706741, "author_name": "Remek Kinas", "author_url": "", "post_date": "2022-02-27T18:34:55.720000", "content": "Thank you fr sharing @aikhmelnytskyy. Good score!
\nСлава Україні!
@aikhmelnytskyy whats the final validation loss and CV for this result?
", "votes": -1, "replies": [] }, { "id": 1707998, "author_name": "huyahuya", "author_url": "", "post_date": "2022-03-01T02:36:15.890000", "content": "You can't reason with the weights you provide！Am I using it incorrectly?
\nmodel.load_weights('../input/happywhale-effnet-b7-fork-with-detic-crop-3b3093/effnetv1_b7_loss_4.h5')
", "votes": -1, "replies": [] }, { "id": 1707993, "author_name": "huyahuya", "author_url": "", "post_date": "2022-03-01T02:32:22.533000", "content": "Using your change network training is fine, but does reasoning get an error?
", "votes": -1, "replies": [] }, { "id": 1707992, "author_name": "huyahuya", "author_url": "", "post_date": "2022-03-01T02:31:54.140000", "content": "ValueError Traceback (most recent call last)
\n/tmp/ipykernel_43/3601991566.py in
\n 9 test_data_list.append([filename,embeddings])
\n 10 ids = get_ids(filename)
\n---> 11 distances,idxs = neigh.kneighbors(embeddings, config.KNN, return_distance=True)
\n 12 test_ids.append(ids)
\n 13 test_nn_idxs.append(idxs)
Not torch, sad
", "votes": -5, "replies": [] }, { "id": 2109657, "author_name": "", "author_url": "", "post_date": "2023-01-21T15:06:17.537000", "content": "", "votes": 0, "replies": [] }, { "id": 1743806, "author_name": "", "author_url": "", "post_date": "2022-04-03T10:17:34.020000", "content": "", "votes": 0, "replies": [] }, { "id": 1709688, "author_name": "", "author_url": "", "post_date": "2022-03-02T12:25:25.413000", "content": "", "votes": 0, "replies": [] }, { "id": 1718792, "author_name": "", "author_url": "", "post_date": "2022-03-11T06:53:30.647000", "content": "", "votes": 1, "replies": [] }, { "id": 1711296, "author_name": "", "author_url": "", "post_date": "2022-03-03T19:52:55.467000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "1706663": "Hello everyone!\nI apologize for publishing my best result, but I doubt that given the recent events I will have the time and desire to continue working. But I do not want my work to be wasted. I added to this dataset https://www.kaggle.com/aikhmelnytskyy/eff7-new-768 my colab notebook in which I trained models. Good luck to you all!\nMy training notebook https://www.kaggle.com/aikhmelnytskyy/happywhale-effnet-b7-fork-with-detic-crop-3b3093\nP.S.\nI believe we will win!\nСлава Україні!", "1706813": "Thanks !! \n\n... and I wish you all the best for all that happens outside of Kaggle :(\n\n", "1708233": "is anyone able to reproduce this result in pytorch?", "1733098": "I have tested your model on fold4 and I find that if the model load your weights at epoch 9,the score seems always higher than the model trained from the beginning.(loading just noicy-student weights) Which is 71.4 to 69.7. I wonder why the previous score is always higher than the latter, how do you get your weights at epoch 9? Thank you!\n", "1721253": "By the way, you and I have the same city :)", "1711321": "Congratulations @aikhmelnytskyy, and I hope you all the best.", "1710228": "very good congrats ", "1708825": "Congratulations", "1708563": "Congratulations, I respect your excellent work!", "1708413": "Congratulations @aikhmelnytskyy ", "1708246": "Congrats!!", "1708211": "Congratulations", "1708194": "execllent，good score", "1708154": "Congrats amazing work\n", "1708127": "Congrats @aikhmelnytskyy 💛💙", "1708038": "Congradullation, good！", "1707626": "Congrats! :D", "1707522": "Congradullation, good score", "1706935": "Thanks and Wish you all the best !", "1706724": "Interesting, how did you blend your models by folds? Concated all embeddings by fold or used mean/median? ", "1706698": "thx for sharing. how long did you train use Colab for single fold? Colab pro or Colab pro+?\n\nhow It takes for inference?", "1726032": "I edited some of this code, and tried with a efficient_v2 implementation using https://github.com/leondgarse/keras_cv_attention_models/tree/main/keras_cv_attention_models/efficientnet and got a 0.699 with a V2M (pretrained on imagenet) and currently trying a V2XL (pretrained on imagenet21k-ft1k). Will let you know how it goes. These notebooks still confuse me on what is happening where, but if I can change the model maybe I have a chance.", "1721246": "Congratilation! Glory to heroes(Героям Слава!)", "1707994": "---------------------------------------------------------------------------\nValueError Traceback (most recent call last)\n/tmp/ipykernel_43/3601991566.py in