model: efficientnetv2_b1(5fold)
\nimage_size=256 cv=0.8639 lb=0.867
\nimage_size=384 cv=0.86566 lb=0.870
\nimage_size=512 cv=0.8681 lb=0.871
\nimage_size=640 cv=0.86808 lb=0.872
\nNothing else has been adjusted, only image_Size and batch_ size
\nDue to the limited computing power of GPU of personal computer, image_ Size > = 768, there is no test for the data of
update:
\nbatch_size=64 seed=1086 Same computer
\nimage_size=128 cv=0.86448 lb=0.867
\nimage_size=256 cv=0.86712 lb=0.870
\nimage_size=384 cv=0.86812 lb=0.872
\nimage_size=512 cv=0.86884 lb=0.872
\nIt can be seen that batchsize has some influence on CV and LB, but it mainly comes from ImageSize.
\nNow I'll show you the forecast score record.
image=128
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:33<00:00, 4.70it/s]
\n100%|██████████| 883/883 [02:40<00:00, 10.39it/s]
\nval score: 0.8645
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:42<00:00, 4.29it/s]
\n100%|██████████| 883/883 [01:37<00:00, 17.70it/s]
\nval score: 0.8649
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.41it/s]
\n100%|██████████| 883/883 [01:39<00:00, 17.67it/s]
\nval score: 0.8640
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.42it/s]
\n100%|██████████| 883/883 [01:39<00:00, 17.58it/s]
\nval score: 0.8647
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.40it/s]
\n100%|██████████| 883/883 [01:38<00:00, 17.73it/s]
\nval score: 0.8643
image=256
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:08<00:00, 6.57it/s]
\n100%|██████████| 1766/1766 [04:05<00:00, 7.19it/s]
\nval score: 0.8672
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:04<00:00, 6.53it/s]
\n100%|██████████| 1766/1766 [03:59<00:00, 7.37it/s]
\nval score: 0.8671
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:09<00:00, 6.49it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.33it/s]
\nval score: 0.8662
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:11<00:00, 6.52it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.32it/s]
\nval score: 0.8680
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:11<00:00, 6.58it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.31it/s]
\nval score: 0.8671
image=384
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [05:59<00:00, 2.89it/s]
\n100%|██████████| 1766/1766 [11:20<00:00, 3.22it/s]
\nval score: 0.8677
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:03<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:10<00:00, 3.18it/s]
\nval score: 0.8684
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:01<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:11<00:00, 3.22it/s]
\nval score: 0.8673
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:02<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:26<00:00, 3.19it/s]
\nval score: 0.8688
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:02<00:00, 2.95it/s]
\n100%|██████████| 1766/1766 [11:08<00:00, 3.21it/s]
\nval score: 0.8684
image=512
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:44<00:00, 2.30it/s]
\n100%|██████████| 1766/1766 [13:19<00:00, 2.68it/s]
\nval score: 0.8682
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:50<00:00, 2.32it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.66it/s]
\nval score: 0.8691
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:49<00:00, 2.36it/s]
\n100%|██████████| 1766/1766 [12:56<00:00, 2.68it/s]
\nval score: 0.8681
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:49<00:00, 2.38it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.61it/s]
\nval score: 0.8700
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:51<00:00, 2.36it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.68it/s]
\nval score: 0.8688
model: efficientnetv2_b1(5fold)
\nimage_size=256 cv=0.8639 lb=0.867
\nimage_size=384 cv=0.86566 lb=0.870
\nimage_size=512 cv=0.8681 lb=0.871
\nimage_size=640 cv=0.86808 lb=0.872
\nNothing else has been adjusted, only image_Size and batch_ size
\nDue to the limited computing power of GPU of personal computer, image_ Size > = 768, there is no test for the data of
update:
\nbatch_size=64 seed=1086 Same computer
\nimage_size=128 cv=0.86448 lb=0.867
\nimage_size=256 cv=0.86712 lb=0.870
\nimage_size=384 cv=0.86812 lb=0.872
\nimage_size=512 cv=0.86884 lb=0.872
\nIt can be seen that batchsize has some influence on CV and LB, but it mainly comes from ImageSize.
\nNow I'll show you the forecast score record.
image=128
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:33<00:00, 4.70it/s]
\n100%|██████████| 883/883 [02:40<00:00, 10.39it/s]
\nval score: 0.8645
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:42<00:00, 4.29it/s]
\n100%|██████████| 883/883 [01:37<00:00, 17.70it/s]
\nval score: 0.8649
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.41it/s]
\n100%|██████████| 883/883 [01:39<00:00, 17.67it/s]
\nval score: 0.8640
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.42it/s]
\n100%|██████████| 883/883 [01:39<00:00, 17.58it/s]
\nval score: 0.8647
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 438/438 [01:39<00:00, 4.40it/s]
\n100%|██████████| 883/883 [01:38<00:00, 17.73it/s]
\nval score: 0.8643
image=256
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:08<00:00, 6.57it/s]
\n100%|██████████| 1766/1766 [04:05<00:00, 7.19it/s]
\nval score: 0.8672
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:04<00:00, 6.53it/s]
\n100%|██████████| 1766/1766 [03:59<00:00, 7.37it/s]
\nval score: 0.8671
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:09<00:00, 6.49it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.33it/s]
\nval score: 0.8662
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:11<00:00, 6.52it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.32it/s]
\nval score: 0.8680
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [03:11<00:00, 6.58it/s]
\n100%|██████████| 1766/1766 [04:01<00:00, 7.31it/s]
\nval score: 0.8671
image=384
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [05:59<00:00, 2.89it/s]
\n100%|██████████| 1766/1766 [11:20<00:00, 3.22it/s]
\nval score: 0.8677
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:03<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:10<00:00, 3.18it/s]
\nval score: 0.8684
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:01<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:11<00:00, 3.22it/s]
\nval score: 0.8673
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:02<00:00, 2.87it/s]
\n100%|██████████| 1766/1766 [11:26<00:00, 3.19it/s]
\nval score: 0.8688
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:02<00:00, 2.95it/s]
\n100%|██████████| 1766/1766 [11:08<00:00, 3.21it/s]
\nval score: 0.8684
image=512
\n[fold 0]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:44<00:00, 2.30it/s]
\n100%|██████████| 1766/1766 [13:19<00:00, 2.68it/s]
\nval score: 0.8682
\n[fold 1]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:50<00:00, 2.32it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.66it/s]
\nval score: 0.8691
\n[fold 2]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:49<00:00, 2.36it/s]
\n100%|██████████| 1766/1766 [12:56<00:00, 2.68it/s]
\nval score: 0.8681
\n[fold 3]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:49<00:00, 2.38it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.61it/s]
\nval score: 0.8700
\n[fold 4]
\nload imagenet pretrained: True
\ntf_efficientnetv2_b1: 1280
\n100%|██████████| 875/875 [06:51<00:00, 2.36it/s]
\n100%|██████████| 1766/1766 [12:57<00:00, 2.68it/s]
\nval score: 0.8688
thanks for sharing, usually i'd think the higher the more likely overfitting is, since the dimension of data has increased and thus there are more ways for the model to cut decision boundaries, but given both your cv and lb increased so it looks like it should be okay.
\nthe simple reason i can think of is probably the signal of gw requires higher resolution?
\ni have a question though, what did you do to increase the image size? are you using nnAudio to compute const-q transform or PyCBC? i have been using nnAudio to do the q-transform since its faster, so i would use smaller hop length to get higher image size, but overall the transformed image i got from nnAudio and PyCBC are not exactly same to each other
", "rawMarkdown": "thanks for sharing, usually i'd think the higher the more likely overfitting is, since the dimension of data has increased and thus there are more ways for the model to cut decision boundaries, but given both your cv and lb increased so it looks like it should be okay.\n\nthe simple reason i can think of is probably the signal of gw requires higher resolution?\n\ni have a question though, what did you do to increase the image size? are you using nnAudio to compute const-q transform or PyCBC? i have been using nnAudio to do the q-transform since its faster, so i would use smaller hop length to get higher image size, but overall the transformed image i got from nnAudio and PyCBC are not exactly same to each other", "votes": null }, { "id": "1384467", "postDate": "07/11/2021 22:23:05", "content": "I use nnaudio just like you do. Pycbc didn't try…
", "rawMarkdown": "I use nnaudio just like you do. Pycbc didn't try...", "votes": null }, { "id": "1385206", "postDate": "07/12/2021 14:15:59", "content": "There are mutliple reason why this could happen:
\n1.Basically bigger images contain more information which can be used to train a NN. the EfficientNet paper concludes and investigates this:
\n\n\nIt is also well-recognized that bigger input image size will help accuracy with the overhead of more FLOPS
\n
See: https://arxiv.org/pdf/1905.11946.pdf
\n2.Do you use augmentations to train your model? How much did you change your batch_size? Batchsize increase can also lead to an increase in accuracy due to better calculated batchnormalisation values. A batchsize decrease lets to your NN weights being updated more often and potentially lead into an increase in accuracy. You also would have to adapt your learning rate etc. when adjusting the batchsize.
", "rawMarkdown": "There are mutliple reason why this could happen: \n\n1.Basically bigger images contain more information which can be used to train a NN. the EfficientNet paper concludes and investigates this: \n> It is also well-recognized that bigger input image size will help accuracy with the overhead of more FLOPS\n\nSee: https://arxiv.org/pdf/1905.11946.pdf\n\n2.Do you use augmentations to train your model? How much did you change your batch_size? Batchsize increase can also lead to an increase in accuracy due to better calculated batchnormalisation values. A batchsize decrease lets to your NN weights being updated more often and potentially lead into an increase in accuracy. You also would have to adapt your learning rate etc. when adjusting the batchsize.", "votes": null }, { "id": "1385625", "postDate": "07/12/2021 21:05:33", "content": "You also change batch size. Are you sure image size is the reason for better performance?
", "rawMarkdown": "You also change batch size. Are you sure image size is the reason for better performance?", "votes": null }, { "id": "1385727", "postDate": "07/13/2021 01:39:23", "content": "Sorry, this need to be confirmed by experiments.Due to CUDA memory limitation, the larger the image, the smaller the batch_ Size, from batch_Size = 256 - > 128 - > 64 - > 32, batch may also affect the recognition rate, so next, I'm going to test the impact on recognition rate under the same batch size
", "rawMarkdown": "Sorry, this need to be confirmed by experiments.Due to CUDA memory limitation, the larger the image, the smaller the batch_ Size, from batch_Size = 256 - > 128 - > 64 - > 32, batch may also affect the recognition rate, so next, I'm going to test the impact on recognition rate under the same batch size", "votes": null }, { "id": "1385733", "postDate": "07/13/2021 01:47:20", "content": "Thanks! Use only cutmix batch_Size = 256 - > 128 - > 64 - > 32 LR unchanged
", "rawMarkdown": "Thanks! Use only cutmix batch_Size = 256 - > 128 - > 64 - > 32 LR unchanged", "votes": null }, { "id": "1386231", "postDate": "07/13/2021 10:50:46", "content": "Can you pls tell what hop_length you used for different sized images?
hop_length=64 unchange
\nResize in data enhancement Library
Thanks. It may seem a dumb question, but I am finding it really hard to digest this. Let's say you have the wave shape as 12288 after stacking all the three signals horizontally. After applying Q-transform with hop_length=32 you get the shape as (69, 385) which again has 12288 data points. But when I resize it to 224 * 224 (total data_points = 50176), it perfectly resizes. So, how does a 12288 data_point wave gets resized to any shape we want (for example 224 * 224, 320 * 320, etc). It would be really helpful if someone answers this. I have also attached a reference screenshot for the same.
I can only say that I am a 3-channel image, but the specific meaning of 3-channel is? Think about it for yourself. If you think about it, it's very simple
", "rawMarkdown": "I can only say that I am a **3-channel** image, but the specific meaning of **3-channel** is? Think about it for yourself. If you think about it, it's very simple", "votes": null }, { "id": "1387446", "postDate": "07/14/2021 07:33:19", "content": "Hi @zhangeng ,
\nBigger efficientnet backbone is also improving the results, have discussed the same in my discussion thread here.
\nhttps://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/252754
\nThanks and Regards,
\nOld Monk
He does not change the used model during his test.
", "rawMarkdown": "He does not change the used model during his test.", "votes": null }, { "id": "1387458", "postDate": "07/14/2021 07:47:29", "content": "Yes @aliabdin1, my thoughts are we can create a table of different efficient net models and image sizes (coupled with batch sizes perhaps), that might help us to identify where the optimum lies.
", "rawMarkdown": "Yes @aliabdin1, my thoughts are we can create a table of different efficient net models and image sizes (coupled with batch sizes perhaps), that might help us to identify where the optimum lies.", "votes": null }, { "id": "1387460", "postDate": "07/14/2021 07:49:28", "content": "@saurabhbagchi Hello,Old Monk!
\nThanks, I will look at it now, but in my experiments, the model has not been changed. It may be that more complex and deeper models perform better. This is almost certain.
For the same computer, the same batch size, the same seed, and different image_size, the data has been published
", "rawMarkdown": "For the same computer, the same batch size, the same seed, and different image_size, the data has been published", "votes": null }, { "id": "1390701", "postDate": "07/17/2021 02:10:19", "content": "thanks for sharing them, just wondering seems like your one epoch for example size 128 takes around 3mins, is that right? did you perform some optimization so that each epoch takes such short time? My experiment with 128 size took me around 50mins per epoch (or even more with kaggles kernel without caching q-transformed image files), and would hardware also be a factor? (if you dont mind, would also appreciate sharing hardware info)
", "rawMarkdown": "thanks for sharing them, just wondering seems like your one epoch for example size 128 takes around 3mins, is that right? did you perform some optimization so that each epoch takes such short time? My experiment with 128 size took me around 50mins per epoch (or even more with kaggles kernel without caching q-transformed image files), and would hardware also be a factor? (if you dont mind, would also appreciate sharing hardware info)", "votes": null }, { "id": "1390714", "postDate": "07/17/2021 03:19:04", "content": "@samshipengs verification takes 3 minutes and training takes 10 minutes. During data preprocessing, it is saved to the local computer, which is rtx6000.
\ntrain:
\nepoch iteration lr train/loss val/loss val/metric elapsed_time
\n1 7000 0.000279904 0.521752 0.444988 0.857384 595.367
\n2 14000 0.000176047 0.476475 0.433594 0.860317 1223.32
\n3 21000 5.35821e-05 0.464922 0.428673 0.864507 1843.21
\n4 28000 3e-10 0.454592 0.426412 0.864342 2468.73
@zhangeng Thank you for sharing your results. I confirmed the same trend - bigger image / better result. Btw, your training is quite fast. Judge from the iteration count and batch size (875 * 64 * 8 = 448000), are you running it on 8 x RTX6000 with DDP or something?
", "rawMarkdown": "zhangeng Thank you for sharing your results. I confirmed the same trend - bigger image / better result. Btw, your training is quite fast. Judge from the iteration count and batch size (875 * 64 * 8 = 448000), are you running it on 8 x RTX6000 with DDP or something?", "votes": null }, { "id": "1391815", "postDate": "07/18/2021 05:37:04", "content": "@analokamus 1x RTX6000 num_workers=8 or16?I can't remember clearly😅. In the last month, I mainly completed another project (alien), and pin_memory=True
", "rawMarkdown": "analokamus 1x RTX6000 num_workers=8 or16?I can't remember clearly😅. In the last month, I mainly completed another project (alien), and pin_memory=True", "votes": null }, { "id": "1391861", "postDate": "07/18/2021 06:59:00", "content": "With your updated data, 256 seems as good as 512.
", "rawMarkdown": "With your updated data, 256 seems as good as 512.", "votes": null }, { "id": "1391878", "postDate": "07/18/2021 07:21:41", "content": "A little mistake
\nimage_size=256 cv=0.86712 lb=0.870
\nimage_size=384 cv=0.86812 lb=0.872
\nupdate:
\nWith your updated data, 384 seems as good as 512
Hi @zhangeng , thanks for sharing this, can you share the source from where you implemented tf_efficientnetv2_b1?
", "rawMarkdown": "Hi @zhangeng , thanks for sharing this, can you share the source from where you implemented tf_efficientnetv2_b1?", "votes": null }, { "id": "1458500", "postDate": "08/07/2021 22:54:25", "content": "What is controlling your image size? Are you enlarging your spectrograms using bilinear interpolation from a smaller size?
", "rawMarkdown": "What is controlling your image size? Are you enlarging your spectrograms using bilinear interpolation from a smaller size?", "votes": null }, { "id": "1465342", "postDate": "08/11/2021 02:43:34", "content": "!pip install timm
", "rawMarkdown": "!pip install timm", "votes": null }, { "id": "1479409", "postDate": "08/18/2021 13:04:27", "content": "I have the same question. If so (if interpolation is used), what makes you want to try this?
\nSince interpolation doesn't add new information.
The point is that image size itself also affects model performance. For example, EfficientNet was able to achieve SOTA by scaling the input image size, depth, and width.
\nThus it is plausible that small image will result in worse performance.
@analokamus
\nI think what you said is right. In addition, I use the default interpolation algorithm of OpenCV without any change, but you also remind me whether this interpolation algorithm needs to be improved when the image does not lose any information?
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris
thanks for sharing, usually i'd think the higher the more likely overfitting is, since the dimension of data has increased and thus there are more ways for the model to cut decision boundaries, but given both your cv and lb increased so it looks like it should be okay.
\nthe simple reason i can think of is probably the signal of gw requires higher resolution?
\ni have a question though, what did you do to increase the image size? are you using nnAudio to compute const-q transform or PyCBC? i have been using nnAudio to do the q-transform since its faster, so i would use smaller hop length to get higher image size, but overall the transformed image i got from nnAudio and PyCBC are not exactly same to each other
", "votes": null, "replies": [ { "id": 1384467, "author_name": "zhangeng", "author_url": "", "post_date": "07/11/2021 22:23:05", "content": "I use nnaudio just like you do. Pycbc didn't try…
", "votes": null, "replies": [] }, { "id": 1386231, "author_name": "atharvaingle", "author_url": "", "post_date": "07/13/2021 10:50:46", "content": "Can you pls tell what hop_length you used for different sized images?
hop_length=64 unchange
\nResize in data enhancement Library
Thanks. It may seem a dumb question, but I am finding it really hard to digest this. Let's say you have the wave shape as 12288 after stacking all the three signals horizontally. After applying Q-transform with hop_length=32 you get the shape as (69, 385) which again has 12288 data points. But when I resize it to 224 * 224 (total data_points = 50176), it perfectly resizes. So, how does a 12288 data_point wave gets resized to any shape we want (for example 224 * 224, 320 * 320, etc). It would be really helpful if someone answers this. I have also attached a reference screenshot for the same.
I can only say that I am a 3-channel image, but the specific meaning of 3-channel is? Think about it for yourself. If you think about it, it's very simple
", "votes": null, "replies": [] } ] }, { "id": 1385206, "author_name": "aliabdin1", "author_url": "", "post_date": "07/12/2021 14:15:59", "content": "There are mutliple reason why this could happen:
\n1.Basically bigger images contain more information which can be used to train a NN. the EfficientNet paper concludes and investigates this:
\n\n\nIt is also well-recognized that bigger input image size will help accuracy with the overhead of more FLOPS
\n
See: https://arxiv.org/pdf/1905.11946.pdf
\n2.Do you use augmentations to train your model? How much did you change your batch_size? Batchsize increase can also lead to an increase in accuracy due to better calculated batchnormalisation values. A batchsize decrease lets to your NN weights being updated more often and potentially lead into an increase in accuracy. You also would have to adapt your learning rate etc. when adjusting the batchsize.
", "votes": null, "replies": [ { "id": 1385733, "author_name": "zhangeng", "author_url": "", "post_date": "07/13/2021 01:47:20", "content": "Thanks! Use only cutmix batch_Size = 256 - > 128 - > 64 - > 32 LR unchanged
", "votes": null, "replies": [] } ] }, { "id": 1385625, "author_name": "cpmpml", "author_url": "", "post_date": "07/12/2021 21:05:33", "content": "You also change batch size. Are you sure image size is the reason for better performance?
", "votes": null, "replies": [ { "id": 1385727, "author_name": "zhangeng", "author_url": "", "post_date": "07/13/2021 01:39:23", "content": "Sorry, this need to be confirmed by experiments.Due to CUDA memory limitation, the larger the image, the smaller the batch_ Size, from batch_Size = 256 - > 128 - > 64 - > 32, batch may also affect the recognition rate, so next, I'm going to test the impact on recognition rate under the same batch size
", "votes": null, "replies": [] } ] }, { "id": 1387446, "author_name": "saurabhbagchi", "author_url": "", "post_date": "07/14/2021 07:33:19", "content": "Hi @zhangeng ,
\nBigger efficientnet backbone is also improving the results, have discussed the same in my discussion thread here.
\nhttps://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/252754
\nThanks and Regards,
\nOld Monk
He does not change the used model during his test.
", "votes": null, "replies": [] }, { "id": 1387458, "author_name": "saurabhbagchi", "author_url": "", "post_date": "07/14/2021 07:47:29", "content": "Yes @aliabdin1, my thoughts are we can create a table of different efficient net models and image sizes (coupled with batch sizes perhaps), that might help us to identify where the optimum lies.
", "votes": null, "replies": [] }, { "id": 1387460, "author_name": "zhangeng", "author_url": "", "post_date": "07/14/2021 07:49:28", "content": "@saurabhbagchi Hello,Old Monk!
\nThanks, I will look at it now, but in my experiments, the model has not been changed. It may be that more complex and deeper models perform better. This is almost certain.
For the same computer, the same batch size, the same seed, and different image_size, the data has been published
", "votes": null, "replies": [ { "id": 1390701, "author_name": "samshipengs", "author_url": "", "post_date": "07/17/2021 02:10:19", "content": "thanks for sharing them, just wondering seems like your one epoch for example size 128 takes around 3mins, is that right? did you perform some optimization so that each epoch takes such short time? My experiment with 128 size took me around 50mins per epoch (or even more with kaggles kernel without caching q-transformed image files), and would hardware also be a factor? (if you dont mind, would also appreciate sharing hardware info)
", "votes": null, "replies": [] }, { "id": 1390714, "author_name": "zhangeng", "author_url": "", "post_date": "07/17/2021 03:19:04", "content": "@samshipengs verification takes 3 minutes and training takes 10 minutes. During data preprocessing, it is saved to the local computer, which is rtx6000.
\ntrain:
\nepoch iteration lr train/loss val/loss val/metric elapsed_time
\n1 7000 0.000279904 0.521752 0.444988 0.857384 595.367
\n2 14000 0.000176047 0.476475 0.433594 0.860317 1223.32
\n3 21000 5.35821e-05 0.464922 0.428673 0.864507 1843.21
\n4 28000 3e-10 0.454592 0.426412 0.864342 2468.73
@zhangeng Thank you for sharing your results. I confirmed the same trend - bigger image / better result. Btw, your training is quite fast. Judge from the iteration count and batch size (875 * 64 * 8 = 448000), are you running it on 8 x RTX6000 with DDP or something?
", "votes": null, "replies": [] }, { "id": 1391815, "author_name": "zhangeng", "author_url": "", "post_date": "07/18/2021 05:37:04", "content": "@analokamus 1x RTX6000 num_workers=8 or16?I can't remember clearly😅. In the last month, I mainly completed another project (alien), and pin_memory=True
", "votes": null, "replies": [] } ] }, { "id": 1391861, "author_name": "cpmpml", "author_url": "", "post_date": "07/18/2021 06:59:00", "content": "With your updated data, 256 seems as good as 512.
", "votes": null, "replies": [ { "id": 1391878, "author_name": "zhangeng", "author_url": "", "post_date": "07/18/2021 07:21:41", "content": "A little mistake
\nimage_size=256 cv=0.86712 lb=0.870
\nimage_size=384 cv=0.86812 lb=0.872
\nupdate:
\nWith your updated data, 384 seems as good as 512
Hi @zhangeng , thanks for sharing this, can you share the source from where you implemented tf_efficientnetv2_b1?
", "votes": null, "replies": [ { "id": 1465342, "author_name": "zhangeng", "author_url": "", "post_date": "08/11/2021 02:43:34", "content": "!pip install timm
", "votes": null, "replies": [] } ] }, { "id": 1458500, "author_name": "anjum48", "author_url": "", "post_date": "08/07/2021 22:54:25", "content": "What is controlling your image size? Are you enlarging your spectrograms using bilinear interpolation from a smaller size?
", "votes": null, "replies": [ { "id": 1479409, "author_name": "dannywu375", "author_url": "", "post_date": "08/18/2021 13:04:27", "content": "I have the same question. If so (if interpolation is used), what makes you want to try this?
\nSince interpolation doesn't add new information.
The point is that image size itself also affects model performance. For example, EfficientNet was able to achieve SOTA by scaling the input image size, depth, and width.
\nThus it is plausible that small image will result in worse performance.
@analokamus
\nI think what you said is right. In addition, I use the default interpolation algorithm of OpenCV without any change, but you also remind me whether this interpolation algorithm needs to be improved when the image does not lose any information?
Hey All,
\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.
\nThank you all,
\nRegards,
\nChris