Hi there. As the title states, I'm curious to see what aspect ratios others are using.
\nI know it has been (tentatively) shown that a 2:1 aspect ratio (appropriately cropped) may be the most efficient method of capturing relevant information.
\nHowever, others have suggested that the margins might contain useable information to potentially recover ghost methyl groups.
\nPersonally, I'm still just using a padded-to-square input image (384x384x3). This is probably not ideal, but I wanted to query other Kagglers to see their thought process. I plan on running a minor benchmarking comparison in the near future as well.
Hi there. As the title states, I'm curious to see what aspect ratios others are using.
\nI know it has been (tentatively) shown that a 2:1 aspect ratio (appropriately cropped) may be the most efficient method of capturing relevant information.
\nHowever, others have suggested that the margins might contain useable information to potentially recover ghost methyl groups.
\nPersonally, I'm still just using a padded-to-square input image (384x384x3). This is probably not ideal, but I wanted to query other Kagglers to see their thought process. I plan on running a minor benchmarking comparison in the near future as well.
In my first experiments, I was using the 1.72 ratio as stated in Mark's TF records notebook. Then I noticed that this ratio is calculated before the cropping. Then I started to use the cropped ratio (2.17) based on the resolution analysis notebook.
\nI am not sure about the effect of optimizing the ratio on the model performance, but it decreases the training time for sure.
", "rawMarkdown": "In my first experiments, I was using the 1.72 ratio as stated in [ Mark's TF records notebook](https://www.kaggle.com/markwijkhuizen/advanced-image-cleaning-and-tfrecord-generation). Then I noticed that this ratio is calculated before the cropping. Then I started to use the cropped ratio (2.17) based on the [resolution analysis notebook](https://www.kaggle.com/michaelwolff/bms-inchi-cropped-img-sizes-for-best-resolution).\n\nI am not sure about the effect of optimizing the ratio on the model performance, but it decreases the training time for sure.", "votes": null }, { "id": "1281257", "postDate": "04/22/2021 18:44:16", "content": "Awesome. Thanks for the info. I was a bit uncertain if rectangular input images would be ok to use… but from my research, and the feedback from everyone on Kaggle, it seems that square is just a choice dictated by simplicity and that I can change over my network to using rectangular inputs.
\nThanks again!
", "rawMarkdown": "Awesome. Thanks for the info. I was a bit uncertain if rectangular input images would be ok to use... but from my research, and the feedback from everyone on Kaggle, it seems that square is just a choice dictated by simplicity and that I can change over my network to using rectangular inputs.\n\nThanks again!", "votes": null }, { "id": "1282373", "postDate": "04/23/2021 21:15:38", "content": "Hi,
\nI also switched from Mark's ratio of 1.75 to 2.0 (512x256) and got a slight improvement on the LB of 0.4. That was with the 'old' attention model (version 3) in his notebook with LB scores of around 14.
I also read that the input res should be tuned to the encoder model you want to use. For example EfficientNet B0 has a 'natural' resolution of 224. I.e. that is the resolution the network would expect if you used the whole pretrained model for image classification.
\nI got that from these two sites, here and here, also with the resolutions of the higher models. I'm not quite clear on what the possible side effects are when you use it with a higher resolution (without the final classification layer).
\nAn advantage of squares is that you can try 90 degree rotation augmentation.
", "rawMarkdown": "Hi, \nI also switched from Mark's ratio of 1.75 to 2.0 (512x256) and got a slight improvement on the LB of 0.4. That was with the 'old' attention model (version 3) in his notebook with LB scores of around 14.\n\nI also read that the input res should be tuned to the encoder model you want to use. For example EfficientNet B0 has a 'natural' resolution of 224. I.e. that is the resolution the network would expect if you used the whole pretrained model for image classification.\n\nI got that from these two sites, [here](https://patrick-llgc.github.io/Learning-Deep-Learning/paper_notes/efficientnet.html) and [here](https://keras.io/examples/vision/image_classification_efficientnet_fine_tuning/), also with the resolutions of the higher models. I'm not quite clear on what the possible side effects are when you use it with a higher resolution (without the final classification layer).\n\nAn advantage of squares is that you can try 90 degree rotation augmentation.", "votes": null }, { "id": "1282403", "postDate": "04/23/2021 22:20:09", "content": "From what I see, Efficientnet rebuilds the architecture based on the given image resolution. Do you know if the same is possible for other pre-trained models like ResNet? My other question is what happens to the pre-trained weights when the architecture is rebuilt by the new resolution?
", "rawMarkdown": "From what I see, Efficientnet rebuilds the architecture based on the given image resolution. Do you know if the same is possible for other pre-trained models like ResNet? My other question is what happens to the pre-trained weights when the architecture is rebuilt by the new resolution?", "votes": null }, { "id": "1282424", "postDate": "04/23/2021 23:11:36", "content": "To be honest, I have not had the time or compute available to experiment with different aspect ratios. However I have found a benefit from increasing image size:
\n224x224 => 3.15 LB
\n320x320 => 2.43 LB
\n384x384 => 2.13 LB
the above results are based on transformer or attention-lstm?
", "rawMarkdown": "the above results are based on transformer or attention-lstm?", "votes": null }, { "id": "1283063", "postDate": "04/24/2021 14:45:18", "content": "I am ignoring right now aspect ratio and resize to a quadratic image size. So no padding involved. Weeks ago I have tried to respect the aspect ratio, but the result was worse. Intuitively, if you keep the aspect ratio and pad, your effective resolution will be smaller. Here it seems really bigger resolution -> better score.
", "rawMarkdown": "I am ignoring right now aspect ratio and resize to a quadratic image size. So no padding involved. Weeks ago I have tried to respect the aspect ratio, but the result was worse. Intuitively, if you keep the aspect ratio and pad, your effective resolution will be smaller. Here it seems really bigger resolution -> better score.", "votes": null }, { "id": "1283099", "postDate": "04/24/2021 15:24:01", "content": "Even if original aspect ratio is not preserved, image shape other than quadratic probably is beneficial.
\nIf you have an image that is twice as wide as tall, and you reshape it to a quadratic image, then you loose more information (~×2 as much?) horizontally. But if you reshape it to an image with the same area as the quadratic one but that is a bit wider and less tall, then you preserve more information in the end with the same amount of computation.
", "rawMarkdown": "Even if original aspect ratio is not preserved, image shape other than quadratic probably is beneficial.\n\nIf you have an image that is twice as wide as tall, and you reshape it to a quadratic image, then you loose more information (~×2 as much?) horizontally. But if you reshape it to an image with the same area as the quadratic one but that is a bit wider and less tall, then you preserve more information in the end with the same amount of computation.", "votes": null }, { "id": "1283111", "postDate": "04/24/2021 15:36:47", "content": "If you want rotatation as data augmentation, then you will have also to pad.
", "rawMarkdown": "If you want rotatation as data augmentation, then you will have also to pad.", "votes": null }, { "id": "1283197", "postDate": "04/24/2021 16:57:41", "content": "I think it uses the same pretrained weights independent of the resolution. Also, the number of parameters in the encoder is independent of the resolution (I just checked). The parameters of a CNN are mostly in the convolutional kernels, which can be applied to different input sizes without changing anything. Just the output dimension scales with the resolution, e.g. EffNet B0: 224x224 produces 7x7 features, 512x256 produces 16x8 features.
", "rawMarkdown": "I think it uses the same pretrained weights independent of the resolution. Also, the number of parameters in the encoder is independent of the resolution (I just checked). The parameters of a CNN are mostly in the convolutional kernels, which can be applied to different input sizes without changing anything. Just the output dimension scales with the resolution, e.g. EffNet B0: 224x224 produces 7x7 features, 512x256 produces 16x8 features.", "votes": null }, { "id": "1283261", "postDate": "04/24/2021 18:39:13", "content": "Thanks, I got it now! It scans the image with the same weights, but the only change is the number of operations based on the given size.
", "rawMarkdown": "Thanks, I got it now! It scans the image with the same weights, but the only change is the number of operations based on the given size.", "votes": null }, { "id": "1283278", "postDate": "04/24/2021 18:59:53", "content": "Sorry, that I don't understand. Why do you need to pad to rotate? If you resize to quadratic shape, like you described, you could just rotate then, no?
\nI wouldn't have expected that distorting the non quadratic images to a quadratic shape would work out better in the end. For example angles are not conserved in this transformation. Thanks for this hint, very interesting…
", "rawMarkdown": "Sorry, that I don't understand. Why do you need to pad to rotate? If you resize to quadratic shape, like you described, you could just rotate then, no?\n\nI wouldn't have expected that distorting the non quadratic images to a quadratic shape would work out better in the end. For example angles are not conserved in this transformation. Thanks for this hint, very interesting...", "votes": null }, { "id": "1283349", "postDate": "04/24/2021 21:07:38", "content": "Transformer
", "rawMarkdown": "Transformer", "votes": null }, { "id": "1283467", "postDate": "04/25/2021 00:44:12", "content": "the giant vision transformer from timm models is:
\nvit_huge_patch14_224_in21k
\n patch_size=14, embed_dim=1280, depth=32, num_heads=16, representation_size=1280
i wonder if anyone has the resource to try this
", "rawMarkdown": "the giant vision transformer from timm models is:\n\n vit_huge_patch14_224_in21k\n patch_size=14, embed_dim=1280, depth=32, num_heads=16, representation_size=1280\n\ni wonder if anyone has the resource to try this ~~(effective input size is 1280x1280)~~", "votes": null }, { "id": "1283471", "postDate": "04/25/2021 00:47:10", "content": "if you analyze the error by size, you will find that for the longest seq (largest image), results improve as input size increase
\nfor the shortest seq (smallest images), the results peak at the best size of about 320. it decreases after this
\nthere is no best size that fits all. i am talking about just resizing the input image without considering the aspect ratio.
", "rawMarkdown": "if you analyze the error by size, you will find that for the longest seq (largest image), results improve as input size increase\n\nfor the shortest seq (smallest images), the results peak at the best size of about 320. it decreases after this\n\nthere is no best size that fits all. i am talking about just resizing the input image without considering the aspect ratio.", "votes": null }, { "id": "1283478", "postDate": "04/25/2021 01:00:04", "content": "How did you make your experiments in order to get these results?
", "rawMarkdown": "How did you make your experiments in order to get these results?", "votes": null }, { "id": "1283480", "postDate": "04/25/2021 01:03:52", "content": "\"Weeks ago I have tried to respect the aspect ratio, but the result was worse.\"
\ni updated some results with preserved const aspect ratio when resizing
\n(see excel table in the post)
https://www.kaggle.com/c/bms-molecular-translation/discussion/231190
\nCV definitely worsens. But LB could be similar or better
", "rawMarkdown": "\"Weeks ago I have tried to respect the aspect ratio, but the result was worse.\"\n\n\ni updated some results with preserved const aspect ratio when resizing\n(see excel table in the post)\n\nhttps://www.kaggle.com/c/bms-molecular-translation/discussion/231190\n\nCV definitely worsens. But LB could be similar or better", "votes": null }, { "id": "1283499", "postDate": "04/25/2021 01:40:29", "content": "I thought _224_ means that your input size is 224x224.
you can correct. i misread the meaning of representative size.
\nin the paper \"For ImageNet results in Table 2, we fine-tuned at higher resolution:
\n512 for ViT-L/16 and 518 for ViT-H/14,\" i suppose higher resolution models are not released.
Update: increasing even more makes performance worse.
\n512x512 => 2.36 LB
\n(all other configuration the same as above experiments)
breakdown your LB score by seq length. each length has its optimum score for different image size,
", "rawMarkdown": "breakdown your LB score by seq length. each length has its optimum score for different image size,", "votes": null }, { "id": "1304427", "postDate": "05/12/2021 15:58:33", "content": "If you are still thinking about the aspect ratio: with ignoring aspect ratio and using a quadratic image size, single model is able to get LB0.75.
", "rawMarkdown": "If you are still thinking about the aspect ratio: with ignoring aspect ratio and using a quadratic image size, single model is able to get LB0.75.", "votes": null }, { "id": "1304833", "postDate": "05/12/2021 23:30:05", "content": "Single model with beam search + normalization ?
", "rawMarkdown": "Single model with beam search + normalization ?", "votes": null }, { "id": "1304861", "postDate": "05/13/2021 00:21:17", "content": "this is something that puzzles me.
\ni would think that in theory, keeping the aspect ratio would be better. But in experiments, just resizing it to square gives the best results.
if you keep aspect ratio:
\nif you ignore aspect ratio:
\non a side note, i note that there is only one type of rotation in train images (e.g. only +90 degree. -90 is absent)
\nIs that the same for test? I look through several test images, it seems that it follows the same rule as train (but there are too many test images … and i didn't check all)
", "rawMarkdown": "on a side note, i note that there is only one type of rotation in train images (e.g. only +90 degree. -90 is absent)\n\nIs that the same for test? I look through several test images, it seems that it follows the same rule as train (but there are too many test images ... and i didn't check all)", "votes": null }, { "id": "1305502", "postDate": "05/13/2021 10:21:24", "content": "@nofreewill no beam search.
", "rawMarkdown": "nofreewill no beam search.", "votes": null }, { "id": "1305509", "postDate": "05/13/2021 10:29:36", "content": "I tested quadratic vs ratio 2 image resolutions with the same number of pixels with LSTMs and Efficientnet B0 encoder. There ratio 2 seems to work best, then quadratic with original aspect ratio and worst, quadratic with resize ignoring aspect ratio.
\nCould it be that quadratic is better for transformers, but not LSTMs?
@tugstugi wow, that's impressive. Thanks!
", "rawMarkdown": "tugstugi wow, that's impressive. Thanks!", "votes": null }, { "id": "1305573", "postDate": "05/13/2021 11:22:16", "content": "I tested quadratic vs ratio 2 image resolutions with the same number of pixels with LSTMs and Efficientnet B0 encoder. There ratio 2 seems to work best, then quadratic with original aspect ratio and worst, quadratic with resize ignoring aspect ratio.\nCould it be that quadratic is better for transformers, but not LSTMs?\nI run some small experiments early on with LSTM andresnet34
\nCV: 6.01 with preserving aspect ratio
\nCV:604 with not preserving aspect ratio
I observed only small difference but as @hengck23 pointed out for me it felt more logical to preserve aspect ratio.
\nmy best LSTM has cv of 1.30 and LB of 1.30 using image resizing with preserving aspect ratio. and transformer based model with preserving aspect ratio has CV of 0.88, and LB of 0.87. I believe Its possible for my model to reach CV of 0.78-0.80 with slightly modified architecture and preserving aspect ratio.
I know @nofreewill will ask these questions, so I am answering in advance, no beam search, single_model with normalization
Thanks for the answer! :D
", "rawMarkdown": "Thanks for the answer! :D", "votes": null }, { "id": "1306508", "postDate": "05/13/2021 20:28:45", "content": "@drhabib Have you ever compared:
\nI think squared works better because your image is large in the height direction which is better for all the not so 'short in height' images. And increasing 384384 to 384768 won't help because it is already large enough in the width direction for most cases. And the few exceptions don't make a difference considering that some images might be stretched too much.
\nBUT I don't think squared is the key - it's more about finding the best expected height & width (for your model) given the train images.
To be honest, I have not had the time or compute available to experiment with different aspect ratios. However I have found a benefit from increasing image size:
Can you pls tell how you came up with those numbers then ? Did you trained on subset of data or is it something else ?
", "rawMarkdown": "`To be honest, I have not had the time or compute available to experiment with different aspect ratios. However I have found a benefit from increasing image size:`\n\nCan you pls tell how you came up with those numbers then ? Did you trained on subset of data or is it something else ?", "votes": null }, { "id": "1309733", "postDate": "05/16/2021 08:41:26", "content": "He didn't experiment with different aspect ratios (height/width).
\nHe experimented with different image sizes that all had aspect ratio of 1.
Oh I see, didn't read the comment carefully. Thanks for the clarification.
", "rawMarkdown": "Oh I see, didn't read the comment carefully. Thanks for the clarification.", "votes": null } ], "comments": [ { "id": 1281243, "author_name": "sorkun", "author_url": "", "post_date": "04/22/2021 18:32:01", "content": "In my first experiments, I was using the 1.72 ratio as stated in Mark's TF records notebook. Then I noticed that this ratio is calculated before the cropping. Then I started to use the cropped ratio (2.17) based on the resolution analysis notebook.
\nI am not sure about the effect of optimizing the ratio on the model performance, but it decreases the training time for sure.
", "votes": null, "replies": [ { "id": 1281257, "author_name": "dschettler8845", "author_url": "", "post_date": "04/22/2021 18:44:16", "content": "Awesome. Thanks for the info. I was a bit uncertain if rectangular input images would be ok to use… but from my research, and the feedback from everyone on Kaggle, it seems that square is just a choice dictated by simplicity and that I can change over my network to using rectangular inputs.
\nThanks again!
", "votes": null, "replies": [] }, { "id": 1282373, "author_name": "michaelwolff", "author_url": "", "post_date": "04/23/2021 21:15:38", "content": "Hi,
\nI also switched from Mark's ratio of 1.75 to 2.0 (512x256) and got a slight improvement on the LB of 0.4. That was with the 'old' attention model (version 3) in his notebook with LB scores of around 14.
I also read that the input res should be tuned to the encoder model you want to use. For example EfficientNet B0 has a 'natural' resolution of 224. I.e. that is the resolution the network would expect if you used the whole pretrained model for image classification.
\nI got that from these two sites, here and here, also with the resolutions of the higher models. I'm not quite clear on what the possible side effects are when you use it with a higher resolution (without the final classification layer).
\nAn advantage of squares is that you can try 90 degree rotation augmentation.
", "votes": null, "replies": [] }, { "id": 1282403, "author_name": "sorkun", "author_url": "", "post_date": "04/23/2021 22:20:09", "content": "From what I see, Efficientnet rebuilds the architecture based on the given image resolution. Do you know if the same is possible for other pre-trained models like ResNet? My other question is what happens to the pre-trained weights when the architecture is rebuilt by the new resolution?
", "votes": null, "replies": [] }, { "id": 1283197, "author_name": "michaelwolff", "author_url": "", "post_date": "04/24/2021 16:57:41", "content": "I think it uses the same pretrained weights independent of the resolution. Also, the number of parameters in the encoder is independent of the resolution (I just checked). The parameters of a CNN are mostly in the convolutional kernels, which can be applied to different input sizes without changing anything. Just the output dimension scales with the resolution, e.g. EffNet B0: 224x224 produces 7x7 features, 512x256 produces 16x8 features.
", "votes": null, "replies": [] }, { "id": 1283261, "author_name": "sorkun", "author_url": "", "post_date": "04/24/2021 18:39:13", "content": "Thanks, I got it now! It scans the image with the same weights, but the only change is the number of operations based on the given size.
", "votes": null, "replies": [] } ] }, { "id": 1282424, "author_name": "talktocharles", "author_url": "", "post_date": "04/23/2021 23:11:36", "content": "To be honest, I have not had the time or compute available to experiment with different aspect ratios. However I have found a benefit from increasing image size:
\n224x224 => 3.15 LB
\n320x320 => 2.43 LB
\n384x384 => 2.13 LB
the above results are based on transformer or attention-lstm?
", "votes": null, "replies": [ { "id": 1283349, "author_name": "talktocharles", "author_url": "", "post_date": "04/24/2021 21:07:38", "content": "Transformer
", "votes": null, "replies": [] } ] }, { "id": 1283478, "author_name": "sorkun", "author_url": "", "post_date": "04/25/2021 01:00:04", "content": "How did you make your experiments in order to get these results?
", "votes": null, "replies": [] }, { "id": 1288650, "author_name": "talktocharles", "author_url": "", "post_date": "04/30/2021 07:51:03", "content": "Update: increasing even more makes performance worse.
\n512x512 => 2.36 LB
\n(all other configuration the same as above experiments)
breakdown your LB score by seq length. each length has its optimum score for different image size,
", "votes": null, "replies": [] }, { "id": 1309721, "author_name": "atharvaingle", "author_url": "", "post_date": "05/16/2021 08:27:48", "content": "To be honest, I have not had the time or compute available to experiment with different aspect ratios. However I have found a benefit from increasing image size:
Can you pls tell how you came up with those numbers then ? Did you trained on subset of data or is it something else ?
", "votes": null, "replies": [] }, { "id": 1309733, "author_name": "nofreewill", "author_url": "", "post_date": "05/16/2021 08:41:26", "content": "He didn't experiment with different aspect ratios (height/width).
\nHe experimented with different image sizes that all had aspect ratio of 1.
Oh I see, didn't read the comment carefully. Thanks for the clarification.
", "votes": null, "replies": [] } ] }, { "id": 1283063, "author_name": "tugstugi", "author_url": "", "post_date": "04/24/2021 14:45:18", "content": "I am ignoring right now aspect ratio and resize to a quadratic image size. So no padding involved. Weeks ago I have tried to respect the aspect ratio, but the result was worse. Intuitively, if you keep the aspect ratio and pad, your effective resolution will be smaller. Here it seems really bigger resolution -> better score.
", "votes": null, "replies": [ { "id": 1283099, "author_name": "nofreewill", "author_url": "", "post_date": "04/24/2021 15:24:01", "content": "Even if original aspect ratio is not preserved, image shape other than quadratic probably is beneficial.
\nIf you have an image that is twice as wide as tall, and you reshape it to a quadratic image, then you loose more information (~×2 as much?) horizontally. But if you reshape it to an image with the same area as the quadratic one but that is a bit wider and less tall, then you preserve more information in the end with the same amount of computation.
", "votes": null, "replies": [] }, { "id": 1283111, "author_name": "tugstugi", "author_url": "", "post_date": "04/24/2021 15:36:47", "content": "If you want rotatation as data augmentation, then you will have also to pad.
", "votes": null, "replies": [] }, { "id": 1283278, "author_name": "michaelwolff", "author_url": "", "post_date": "04/24/2021 18:59:53", "content": "Sorry, that I don't understand. Why do you need to pad to rotate? If you resize to quadratic shape, like you described, you could just rotate then, no?
\nI wouldn't have expected that distorting the non quadratic images to a quadratic shape would work out better in the end. For example angles are not conserved in this transformation. Thanks for this hint, very interesting…
", "votes": null, "replies": [] }, { "id": 1283471, "author_name": "hengck23", "author_url": "", "post_date": "04/25/2021 00:47:10", "content": "if you analyze the error by size, you will find that for the longest seq (largest image), results improve as input size increase
\nfor the shortest seq (smallest images), the results peak at the best size of about 320. it decreases after this
\nthere is no best size that fits all. i am talking about just resizing the input image without considering the aspect ratio.
", "votes": null, "replies": [] }, { "id": 1283480, "author_name": "hengck23", "author_url": "", "post_date": "04/25/2021 01:03:52", "content": "\"Weeks ago I have tried to respect the aspect ratio, but the result was worse.\"
\ni updated some results with preserved const aspect ratio when resizing
\n(see excel table in the post)
https://www.kaggle.com/c/bms-molecular-translation/discussion/231190
\nCV definitely worsens. But LB could be similar or better
", "votes": null, "replies": [] } ] }, { "id": 1283467, "author_name": "hengck23", "author_url": "", "post_date": "04/25/2021 00:44:12", "content": "the giant vision transformer from timm models is:
\nvit_huge_patch14_224_in21k
\n patch_size=14, embed_dim=1280, depth=32, num_heads=16, representation_size=1280
i wonder if anyone has the resource to try this
", "votes": null, "replies": [ { "id": 1283499, "author_name": "tugstugi", "author_url": "", "post_date": "04/25/2021 01:40:29", "content": "I thought _224_ means that your input size is 224x224.
you can correct. i misread the meaning of representative size.
\nin the paper \"For ImageNet results in Table 2, we fine-tuned at higher resolution:
\n512 for ViT-L/16 and 518 for ViT-H/14,\" i suppose higher resolution models are not released.
If you are still thinking about the aspect ratio: with ignoring aspect ratio and using a quadratic image size, single model is able to get LB0.75.
", "votes": null, "replies": [ { "id": 1304833, "author_name": "nofreewill", "author_url": "", "post_date": "05/12/2021 23:30:05", "content": "Single model with beam search + normalization ?
", "votes": null, "replies": [] }, { "id": 1304861, "author_name": "hengck23", "author_url": "", "post_date": "05/13/2021 00:21:17", "content": "this is something that puzzles me.
\ni would think that in theory, keeping the aspect ratio would be better. But in experiments, just resizing it to square gives the best results.
if you keep aspect ratio:
\nif you ignore aspect ratio:
\non a side note, i note that there is only one type of rotation in train images (e.g. only +90 degree. -90 is absent)
\nIs that the same for test? I look through several test images, it seems that it follows the same rule as train (but there are too many test images … and i didn't check all)
", "votes": null, "replies": [] }, { "id": 1305502, "author_name": "tugstugi", "author_url": "", "post_date": "05/13/2021 10:21:24", "content": "@nofreewill no beam search.
", "votes": null, "replies": [] }, { "id": 1305509, "author_name": "michaelwolff", "author_url": "", "post_date": "05/13/2021 10:29:36", "content": "I tested quadratic vs ratio 2 image resolutions with the same number of pixels with LSTMs and Efficientnet B0 encoder. There ratio 2 seems to work best, then quadratic with original aspect ratio and worst, quadratic with resize ignoring aspect ratio.
\nCould it be that quadratic is better for transformers, but not LSTMs?
@tugstugi wow, that's impressive. Thanks!
", "votes": null, "replies": [] }, { "id": 1305573, "author_name": "drhabib", "author_url": "", "post_date": "05/13/2021 11:22:16", "content": "I tested quadratic vs ratio 2 image resolutions with the same number of pixels with LSTMs and Efficientnet B0 encoder. There ratio 2 seems to work best, then quadratic with original aspect ratio and worst, quadratic with resize ignoring aspect ratio.\nCould it be that quadratic is better for transformers, but not LSTMs?\nI run some small experiments early on with LSTM andresnet34
\nCV: 6.01 with preserving aspect ratio
\nCV:604 with not preserving aspect ratio
I observed only small difference but as @hengck23 pointed out for me it felt more logical to preserve aspect ratio.
\nmy best LSTM has cv of 1.30 and LB of 1.30 using image resizing with preserving aspect ratio. and transformer based model with preserving aspect ratio has CV of 0.88, and LB of 0.87. I believe Its possible for my model to reach CV of 0.78-0.80 with slightly modified architecture and preserving aspect ratio.
I know @nofreewill will ask these questions, so I am answering in advance, no beam search, single_model with normalization
Thanks for the answer! :D
", "votes": null, "replies": [] }, { "id": 1306508, "author_name": "cepheidq", "author_url": "", "post_date": "05/13/2021 20:28:45", "content": "@drhabib Have you ever compared:
\nI think squared works better because your image is large in the height direction which is better for all the not so 'short in height' images. And increasing 384384 to 384768 won't help because it is already large enough in the width direction for most cases. And the few exceptions don't make a difference considering that some images might be stretched too much.
\nBUT I don't think squared is the key - it's more about finding the best expected height & width (for your model) given the train images.