Using external dataset is not allowed. That means you can't fintune object detectors for predicting the bounding boxes for the digits without putting manual effort of annotating the given dataset (Assuming there is no public available model trained on such dataset). Cost of manually annotating the whole dataset is huge.
\nSeems like a perfect opportunity to explore self supervised learning.
\nAttaching helpful resources to learn about the same:
\nUsing external dataset is not allowed. That means you can't fintune object detectors for predicting the bounding boxes for the digits without putting manual effort of annotating the given dataset (Assuming there is no public available model trained on such dataset). Cost of manually annotating the whole dataset is huge.
\nSeems like a perfect opportunity to explore self supervised learning.
\nAttaching helpful resources to learn about the same:
\nWhen you mention self-supervised are you considering building a Student-Teacher Models kind of scenario. Where the knowledge from one distill out to another. On the other hand it does make sense to annotate data without much effort but would it be accurate? What are your thoughts(As self-supervised will also lead to a probabilistic result)
", "rawMarkdown": "When you mention self-supervised are you considering building a Student-Teacher Models kind of scenario. Where the knowledge from one distill out to another. On the other hand it does make sense to annotate data without much effort but would it be accurate? What are your thoughts(As self-supervised will also lead to a probabilistic result)", "votes": null }, { "id": "1717773", "postDate": "03/10/2022 08:00:07", "content": "No. What you are talking about is called Knowledge distillation. It is completely different from self supervised learning, as in distillation the teacher model is trained on the given labelled dataset.
\nself supervised learning is helpful especially when large portion of dataset is unlabelled and only some small section is labelled.
\nYes, now that we can get labelled data easily, using self supervised learning doesn't make sense.
\nI feel results from model trained on generated data will have better results compared to the one trained using self supervised learning.
One thing you have to ensure if using generated data is that there should not be significant domain shift between generated data and the provided data.
\nCommon approaches of generating data that could cause domain shift:
How to check if the data distribution of generated dataset is same as that of provided data ?
\nThis is called covariate shift. You can read about the existing methods. One simple way is to train a CNN to classify. the data as generated or original. If the classifier is able to distinguish these two, then there is a significant change in the data distribution and the generated data is not good.
I understand what you mean. I actually mis quoted distillation instead of self supervision.
", "rawMarkdown": "I understand what you mean. I actually mis quoted distillation instead of self supervision.", "votes": null } ], "comments": [ { "id": 1717389, "author_name": "shivkumarganesh", "author_url": "", "post_date": "03/09/2022 22:21:55", "content": "When you mention self-supervised are you considering building a Student-Teacher Models kind of scenario. Where the knowledge from one distill out to another. On the other hand it does make sense to annotate data without much effort but would it be accurate? What are your thoughts(As self-supervised will also lead to a probabilistic result)
", "votes": null, "replies": [ { "id": 1717773, "author_name": "harshraj22", "author_url": "", "post_date": "03/10/2022 08:00:07", "content": "No. What you are talking about is called Knowledge distillation. It is completely different from self supervised learning, as in distillation the teacher model is trained on the given labelled dataset.
\nself supervised learning is helpful especially when large portion of dataset is unlabelled and only some small section is labelled.
\nYes, now that we can get labelled data easily, using self supervised learning doesn't make sense.
\nI feel results from model trained on generated data will have better results compared to the one trained using self supervised learning.
One thing you have to ensure if using generated data is that there should not be significant domain shift between generated data and the provided data.
\nCommon approaches of generating data that could cause domain shift:
How to check if the data distribution of generated dataset is same as that of provided data ?
\nThis is called covariate shift. You can read about the existing methods. One simple way is to train a CNN to classify. the data as generated or original. If the classifier is able to distinguish these two, then there is a significant change in the data distribution and the generated data is not good.
I understand what you mean. I actually mis quoted distillation instead of self supervision.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1717350": "Using external dataset is not allowed. That means you can't fintune object detectors for predicting the bounding boxes for the digits without putting manual effort of annotating the given dataset (Assuming there is no public available model trained on such dataset). Cost of manually annotating the whole dataset is huge.\n\nSeems like a perfect opportunity to explore self supervised learning.\n\nAttaching helpful resources to learn about the same:\n- [Introductory youtube video](https://youtube.com/playlist?list=PLd9i_xMMzZF7QiPZNF7zblpTksTt0DDy6) by Anuj Shah, about using self supervised learning for classification", "1717389": "When you mention self-supervised are you considering building a Student-Teacher Models kind of scenario. Where the knowledge from one distill out to another. On the other hand it does make sense to annotate data without much effort but would it be accurate? What are your thoughts(As self-supervised will also lead to a probabilistic result)", "1717773": "No. What you are talking about is called Knowledge distillation. It is completely different from self supervised learning, as in distillation the teacher model is trained on the given labelled dataset.\n\nself supervised learning is helpful especially when large portion of dataset is unlabelled and only some small section is labelled.\n\nYes, now that we can get labelled data easily, using self supervised learning doesn't make sense.\nI feel results from model trained on generated data will have better results compared to the one trained using self supervised learning. \n\nOne thing you have to ensure if using generated data is that there should not be significant domain shift between generated data and the provided data.\nCommon approaches of generating data that could cause domain shift:\n- Keeping bounding boxes of same size.. The dataset has digits of different size.\n- Manually annotating only a few samples per digit\n- using augmentations like flip, random crop, center crop etc\n\nHow to check if the data distribution of generated dataset is same as that of provided data ?\nThis is called covariate shift. You can read about the existing methods. One simple way is to train a CNN to classify. the data as generated or original. If the classifier is able to distinguish these two, then there is a significant change in the data distribution and the generated data is not good.", "1717811": "I understand what you mean. I actually mis quoted distillation instead of self supervision." }, "source": "meta" }