Many thanks to the hosts and congrats to all participants! It was a very interesting task. I enjoyed working with the dataset and I think organizers made very good choices related to the background and proportion of small/large digits. So this dataset may serve as a good foundation for benchmarks.
\nMy solution is based on a classic multiclass classification setup with an extensive tuning and some tricks helping to preserve as much of the signal from the small digits as possible. I trained on a competition dataset only. Neither MNIST data, nor any knowledge about background was utilized because such constrained setup is very much inline with my interests. I deployed the EfficientNet-B7 backbone and 2-stage training procedure. In the 1st stage I trained the model with 1024x1024 resolution, and then in the 2nd stage I continued training on a larger resolution of 1536x1536 and smaller learning rate. TPUv3 allowed a global batch size of 16 for both stages. The learning rates were 5e-4 and 1e-4 correspondingly with a reduction on plateau. As the only augmentation I used inversion (255 - image) which is natural to this dataset. CV and LB were very close (single fold): 0.87 for stage-1 and 0.94 for stage-2.
\nI have published my training code:
\nhttps://www.kaggle.com/code/vecxoz/tricks-for-large-images
Keep learning, keep kaggling!
", "messages": [ { "id": "1756375", "postDate": "04/15/2022 12:58:53", "content": "Many thanks to the hosts and congrats to all participants! It was a very interesting task. I enjoyed working with the dataset and I think organizers made very good choices related to the background and proportion of small/large digits. So this dataset may serve as a good foundation for benchmarks.
\nMy solution is based on a classic multiclass classification setup with an extensive tuning and some tricks helping to preserve as much of the signal from the small digits as possible. I trained on a competition dataset only. Neither MNIST data, nor any knowledge about background was utilized because such constrained setup is very much inline with my interests. I deployed the EfficientNet-B7 backbone and 2-stage training procedure. In the 1st stage I trained the model with 1024x1024 resolution, and then in the 2nd stage I continued training on a larger resolution of 1536x1536 and smaller learning rate. TPUv3 allowed a global batch size of 16 for both stages. The learning rates were 5e-4 and 1e-4 correspondingly with a reduction on plateau. As the only augmentation I used inversion (255 - image) which is natural to this dataset. CV and LB were very close (single fold): 0.87 for stage-1 and 0.94 for stage-2.
\nI have published my training code:
\nhttps://www.kaggle.com/code/vecxoz/tricks-for-large-images
Keep learning, keep kaggling!
", "rawMarkdown": "Many thanks to the hosts and congrats to all participants! It was a very interesting task. I enjoyed working with the dataset and I think organizers made very good choices related to the background and proportion of small/large digits. So this dataset may serve as a good foundation for benchmarks.\n\nMy solution is based on a classic multiclass classification setup with an extensive tuning and some tricks helping to preserve as much of the signal from the small digits as possible. I trained on a competition dataset only. Neither MNIST data, nor any knowledge about background was utilized because such constrained setup is very much inline with my interests. I deployed the EfficientNet-B7 backbone and 2-stage training procedure. In the 1st stage I trained the model with 1024x1024 resolution, and then in the 2nd stage I continued training on a larger resolution of 1536x1536 and smaller learning rate. TPUv3 allowed a global batch size of 16 for both stages. The learning rates were 5e-4 and 1e-4 correspondingly with a reduction on plateau. As the only augmentation I used inversion (255 - image) which is natural to this dataset. CV and LB were very close (single fold): 0.87 for stage-1 and 0.94 for stage-2.\n\nI have published my training code:\nhttps://www.kaggle.com/code/vecxoz/tricks-for-large-images\n\nKeep learning, keep kaggling!", "votes": null }, { "id": "1756381", "postDate": "04/15/2022 13:08:15", "content": "Quite detailed, I didn't know about INTEGER_ACCURATE and didn't even think about anti-aliasing, I mainly restricted myself to NEAREST_NEIGHBOURS and didn't get that good results. Thanks so much!!
Did you also try to use stride=1 in the first conv layer?
\nAnd did you infer on full scale images?
Thank you!
\nI did not try stride=1, though it was on my todo list. Did you try?
\nYes, I ran inference using different larger resolutions - the scores were very low.
Thanks @vecxoz for participating and achieving a nice rank in the competition. Your solution, although simple, complies with the innovation track requirements. Among the various solutions that might form part of the innovation, the simple baseline with proper training, is also going to be one and you have really implemented that baseline very well. In this regard, we would like to ask you to participate in the innovation track if interested. If you agree, we will need more details on the process flow as we move ahead in compiling the paper.
", "rawMarkdown": "Thanks @vecxoz for participating and achieving a nice rank in the competition. Your solution, although simple, complies with the innovation track requirements. Among the various solutions that might form part of the innovation, the simple baseline with proper training, is also going to be one and you have really implemented that baseline very well. In this regard, we would like to ask you to participate in the innovation track if interested. If you agree, we will need more details on the process flow as we move ahead in compiling the paper.", "votes": null }, { "id": "1756536", "postDate": "04/15/2022 15:36:09", "content": "wow I never thought that the most basic classification model can actually achieve more than 90% acc, wonderful work!
", "rawMarkdown": "wow I never thought that the most basic classification model can actually achieve more than 90% acc, wonderful work!", "votes": null }, { "id": "1756653", "postDate": "04/15/2022 18:12:24", "content": "I tried with yolov5 but then training got very slow and I couldn't continue further experiments.
", "rawMarkdown": "I tried with yolov5 but then training got very slow and I couldn't continue further experiments.", "votes": null }, { "id": "1757069", "postDate": "04/16/2022 08:09:07", "content": "Thank you very much! Unfortunately, I will not be able to participate due to a very tight schedule for the near future. Many thanks for the invitation, I appreciate it.
", "rawMarkdown": "Thank you very much! Unfortunately, I will not be able to participate due to a very tight schedule for the near future. Many thanks for the invitation, I appreciate it.", "votes": null }, { "id": "1757071", "postDate": "04/16/2022 08:12:59", "content": "Thanks! Pushing baselines to the limits is my passion.
", "rawMarkdown": "Thanks! Pushing baselines to the limits is my passion.", "votes": null }, { "id": "1758162", "postDate": "04/17/2022 11:42:11", "content": "Hi @vecxoz, great notebook! Absolutely outstanding tips! Thank you - I have learned a lot from your notebook.
\n Congratulations! Your score is outstanding using only one stage solution - I am really impressed.
Thank you!
", "rawMarkdown": "Thank you!", "votes": null }, { "id": "1758627", "postDate": "04/17/2022 21:46:15", "content": "Great work! I'm really interested in how you were able to preserve so much information from the small digits.
", "rawMarkdown": "Great work! I'm really interested in how you were able to preserve so much information from the small digits.", "votes": null } ], "comments": [ { "id": 1756381, "author_name": "mightyrains", "author_url": "", "post_date": "04/15/2022 13:08:15", "content": "Quite detailed, I didn't know about INTEGER_ACCURATE and didn't even think about anti-aliasing, I mainly restricted myself to NEAREST_NEIGHBOURS and didn't get that good results. Thanks so much!!
Did you also try to use stride=1 in the first conv layer?
\nAnd did you infer on full scale images?
Thank you!
\nI did not try stride=1, though it was on my todo list. Did you try?
\nYes, I ran inference using different larger resolutions - the scores were very low.
I tried with yolov5 but then training got very slow and I couldn't continue further experiments.
", "votes": null, "replies": [] } ] }, { "id": 1756488, "author_name": "dkgupta90", "author_url": "", "post_date": "04/15/2022 15:01:48", "content": "Thanks @vecxoz for participating and achieving a nice rank in the competition. Your solution, although simple, complies with the innovation track requirements. Among the various solutions that might form part of the innovation, the simple baseline with proper training, is also going to be one and you have really implemented that baseline very well. In this regard, we would like to ask you to participate in the innovation track if interested. If you agree, we will need more details on the process flow as we move ahead in compiling the paper.
", "votes": null, "replies": [ { "id": 1757069, "author_name": "vecxoz", "author_url": "", "post_date": "04/16/2022 08:09:07", "content": "Thank you very much! Unfortunately, I will not be able to participate due to a very tight schedule for the near future. Many thanks for the invitation, I appreciate it.
", "votes": null, "replies": [] } ] }, { "id": 1756536, "author_name": "haqishen", "author_url": "", "post_date": "04/15/2022 15:36:09", "content": "wow I never thought that the most basic classification model can actually achieve more than 90% acc, wonderful work!
", "votes": null, "replies": [ { "id": 1757071, "author_name": "vecxoz", "author_url": "", "post_date": "04/16/2022 08:12:59", "content": "Thanks! Pushing baselines to the limits is my passion.
", "votes": null, "replies": [] } ] }, { "id": 1758162, "author_name": "remekkinas", "author_url": "", "post_date": "04/17/2022 11:42:11", "content": "Hi @vecxoz, great notebook! Absolutely outstanding tips! Thank you - I have learned a lot from your notebook.
\n Congratulations! Your score is outstanding using only one stage solution - I am really impressed.
Thank you!
", "votes": null, "replies": [] } ] }, { "id": 1758627, "author_name": "", "author_url": "", "post_date": "04/17/2022 21:46:15", "content": "Great work! I'm really interested in how you were able to preserve so much information from the small digits.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1756375": "Many thanks to the hosts and congrats to all participants! It was a very interesting task. I enjoyed working with the dataset and I think organizers made very good choices related to the background and proportion of small/large digits. So this dataset may serve as a good foundation for benchmarks.\n\nMy solution is based on a classic multiclass classification setup with an extensive tuning and some tricks helping to preserve as much of the signal from the small digits as possible. I trained on a competition dataset only. Neither MNIST data, nor any knowledge about background was utilized because such constrained setup is very much inline with my interests. I deployed the EfficientNet-B7 backbone and 2-stage training procedure. In the 1st stage I trained the model with 1024x1024 resolution, and then in the 2nd stage I continued training on a larger resolution of 1536x1536 and smaller learning rate. TPUv3 allowed a global batch size of 16 for both stages. The learning rates were 5e-4 and 1e-4 correspondingly with a reduction on plateau. As the only augmentation I used inversion (255 - image) which is natural to this dataset. CV and LB were very close (single fold): 0.87 for stage-1 and 0.94 for stage-2.\n\nI have published my training code:\nhttps://www.kaggle.com/code/vecxoz/tricks-for-large-images\n\nKeep learning, keep kaggling!", "1756381": "Quite detailed, I didn't know about `INTEGER_ACCURATE` and didn't even think about anti-aliasing, I mainly restricted myself to NEAREST_NEIGHBOURS and didn't get that good results. Thanks so much!!\n\nDid you also try to use `stride=1` in the first conv layer?\nAnd did you infer on full scale images?", "1756394": "Thank you!\nI did not try `stride=1`, though it was on my todo list. Did you try?\nYes, I ran inference using different larger resolutions - the scores were very low.", "1756488": "Thanks @vecxoz for participating and achieving a nice rank in the competition. Your solution, although simple, complies with the innovation track requirements. Among the various solutions that might form part of the innovation, the simple baseline with proper training, is also going to be one and you have really implemented that baseline very well. In this regard, we would like to ask you to participate in the innovation track if interested. If you agree, we will need more details on the process flow as we move ahead in compiling the paper.", "1756536": "wow I never thought that the most basic classification model can actually achieve more than 90% acc, wonderful work!", "1756653": "I tried with yolov5 but then training got very slow and I couldn't continue further experiments.", "1757069": "Thank you very much! Unfortunately, I will not be able to participate due to a very tight schedule for the near future. Many thanks for the invitation, I appreciate it.", "1757071": "Thanks! Pushing baselines to the limits is my passion.", "1758162": "Hi @vecxoz, great notebook! Absolutely outstanding tips! Thank you - I have learned a lot from your notebook.\n Congratulations! Your score is outstanding using only one stage solution - I am really impressed.", "1758248": "Thank you!", "1758627": "Great work! I'm really interested in how you were able to preserve so much information from the small digits." }, "source": "meta" }