{
  "id": 65122,
  "title": "Help requested to understand this specific requirement on model eligibility",
  "url": "/competitions/inclusive-images-challenge/discussion/65122",
  "author_name": "",
  "post_date": "2018-09-06T12:09:25.330892600Z",
  "votes": 13,
  "comment_count": 3,
  "views": 0,
  "content": "<p>As this following rule is new. would like to understand it better. requesting for help on the same.<br></p>\n\n<pre><code>Competitors are not permitted to warm-start their models using pretrained models, or otherwise use pretrained models in the training of their models.\n</code></pre>\n\n<p>Does this mean, <br>\n 1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)  (or) <br> \n2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed. (or) <br>\n3.  Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed <br>\n4.  Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.<br>\n5. (not sure if this will be useful)  there may not be any restriction on word embeddings if one wants to use in their models. <br></p>\n\n<p>Thank you</p>",
  "messages": [
    {
      "id": "382461",
      "postDate": "09/06/2018 12:09:25",
      "content": "<p>As this following rule is new. would like to understand it better. requesting for help on the same.<br></p>\n\n<pre><code>Competitors are not permitted to warm-start their models using pretrained models, or otherwise use pretrained models in the training of their models.\n</code></pre>\n\n<p>Does this mean, <br>\n 1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)  (or) <br> \n2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed. (or) <br>\n3.  Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed <br>\n4.  Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.<br>\n5. (not sure if this will be useful)  there may not be any restriction on word embeddings if one wants to use in their models. <br></p>\n\n<p>Thank you</p>",
      "rawMarkdown": "As this following rule is new. would like to understand it better. requesting for help on the same.<br>\n \n\n    Competitors are not permitted to warm-start their models using pretrained models, or otherwise use pretrained models in the training of their models.\n\nDoes this mean, <br>\n 1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)  (or) <br> \n2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed. (or) <br>\n3.  Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed <br>\n4.  Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.<br>\n5. (not sure if this will be useful)  there may not be any restriction on word embeddings if one wants to use in their models. <br>\n\nThank you",
      "votes": null
    },
    {
      "id": "382506",
      "postDate": "09/06/2018 13:44:32",
      "content": "<p>Hi Kishore,</p>\n\n<p>Thanks for the great question!  This is a critical point in the rules and I'm glad you called it out.</p>\n\n<p>The background of the rule is that we wish to make sure that competitors are exploring methods of correcting such issues other than simply adding more data from other sources, which is an excellent approach but one that doesn’t move the research field forward in the areas of handling distributional skew.  There are a variety of ways that pretrained models could be used as a “back door” that effectively would allow competitors to get around this provision, so we’ve made the rules on this pretty strict.</p>\n\n<p>I can give more detail on your specific questions:</p>\n\n<p><em>1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)</em>\nYou are free to use these model architectures, but not allowed to use pretrained weights for the architectures.  You would need to train them from scratch.</p>\n\n<p><em>2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed.</em>\nWhile the generation/extraction of bottleneck features is fine on its own, it would be against the rules to do so with a pretrained model.</p>\n\n<p><em>3. Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed</em> \nThis approach would be against the rules.</p>\n\n<p><em>4. Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.</em>\nThis would be against the rules because pretrained weights are being used to warm-start the model (and continue to persist in the frozen layers).</p>\n\n<p><em>5. (not sure if this will be useful) there may not be any restriction on word embeddings if one wants to use in their models.</em> \nUsing a pretrained word embedding would be against the rules.  Training your own word embedding would be fine; using an untrained vectorization like bag-of-words or the hashing trick would also be fine.</p>",
      "rawMarkdown": "Hi Kishore,\n\nThanks for the great question!  This is a critical point in the rules and I'm glad you called it out.\n\nThe background of the rule is that we wish to make sure that competitors are exploring methods of correcting such issues other than simply adding more data from other sources, which is an excellent approach but one that doesn’t move the research field forward in the areas of handling distributional skew.  There are a variety of ways that pretrained models could be used as a “back door” that effectively would allow competitors to get around this provision, so we’ve made the rules on this pretty strict.\n\nI can give more detail on your specific questions:\n\n*1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)*\nYou are free to use these model architectures, but not allowed to use pretrained weights for the architectures.  You would need to train them from scratch.\n\n*2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed.*\nWhile the generation/extraction of bottleneck features is fine on its own, it would be against the rules to do so with a pretrained model.\n\n*3. Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed* \nThis approach would be against the rules.\n\n*4. Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.*\nThis would be against the rules because pretrained weights are being used to warm-start the model (and continue to persist in the frozen layers).\n\n*5. (not sure if this will be useful) there may not be any restriction on word embeddings if one wants to use in their models.* \nUsing a pretrained word embedding would be against the rules.  Training your own word embedding would be fine; using an untrained vectorization like bag-of-words or the hashing trick would also be fine.",
      "votes": null
    },
    {
      "id": "382525",
      "postDate": "09/06/2018 14:29:57",
      "content": "<p>Thank you very much @JamesAtwood for the quick response and for detailed answers to each of the  queries. <br> your explanation made rules related to model training crystal clear. once again thank you.</p>",
      "rawMarkdown": "Thank you very much @JamesAtwood for the quick response and for detailed answers to each of the  queries. <br> your explanation made rules related to model training crystal clear. once again thank you.",
      "votes": null
    },
    {
      "id": "383194",
      "postDate": "09/08/2018 01:28:44",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 382506,
      "author_name": "atwoodj",
      "author_url": "",
      "post_date": "09/06/2018 13:44:32",
      "content": "<p>Hi Kishore,</p>\n\n<p>Thanks for the great question!  This is a critical point in the rules and I'm glad you called it out.</p>\n\n<p>The background of the rule is that we wish to make sure that competitors are exploring methods of correcting such issues other than simply adding more data from other sources, which is an excellent approach but one that doesn’t move the research field forward in the areas of handling distributional skew.  There are a variety of ways that pretrained models could be used as a “back door” that effectively would allow competitors to get around this provision, so we’ve made the rules on this pretty strict.</p>\n\n<p>I can give more detail on your specific questions:</p>\n\n<p><em>1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)</em>\nYou are free to use these model architectures, but not allowed to use pretrained weights for the architectures.  You would need to train them from scratch.</p>\n\n<p><em>2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed.</em>\nWhile the generation/extraction of bottleneck features is fine on its own, it would be against the rules to do so with a pretrained model.</p>\n\n<p><em>3. Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed</em> \nThis approach would be against the rules.</p>\n\n<p><em>4. Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.</em>\nThis would be against the rules because pretrained weights are being used to warm-start the model (and continue to persist in the frozen layers).</p>\n\n<p><em>5. (not sure if this will be useful) there may not be any restriction on word embeddings if one wants to use in their models.</em> \nUsing a pretrained word embedding would be against the rules.  Training your own word embedding would be fine; using an untrained vectorization like bag-of-words or the hashing trick would also be fine.</p>",
      "votes": null,
      "replies": [
        {
          "id": 382525,
          "author_name": "reachkishore",
          "author_url": "",
          "post_date": "09/06/2018 14:29:57",
          "content": "<p>Thank you very much @JamesAtwood for the quick response and for detailed answers to each of the  queries. <br> your explanation made rules related to model training crystal clear. once again thank you.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 383194,
          "author_name": "ryanzhang",
          "author_url": "",
          "post_date": "09/08/2018 01:28:44",
          "content": "",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "382461": "As this following rule is new. would like to understand it better. requesting for help on the same.<br>\n \n\n    Competitors are not permitted to warm-start their models using pretrained models, or otherwise use pretrained models in the training of their models.\n\nDoes this mean, <br>\n 1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)  (or) <br> \n2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed. (or) <br>\n3.  Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed <br>\n4.  Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.<br>\n5. (not sure if this will be useful)  there may not be any restriction on word embeddings if one wants to use in their models. <br>\n\nThank you",
    "382506": "Hi Kishore,\n\nThanks for the great question!  This is a critical point in the rules and I'm glad you called it out.\n\nThe background of the rule is that we wish to make sure that competitors are exploring methods of correcting such issues other than simply adding more data from other sources, which is an excellent approach but one that doesn’t move the research field forward in the areas of handling distributional skew.  There are a variety of ways that pretrained models could be used as a “back door” that effectively would allow competitors to get around this provision, so we’ve made the rules on this pretty strict.\n\nI can give more detail on your specific questions:\n\n*1. we can't use publicly available pretrained models at all( Imagenet models like Resnet,inception etc)*\nYou are free to use these model architectures, but not allowed to use pretrained weights for the architectures.  You would need to train them from scratch.\n\n*2. generation/extraction of bottleneck features from pretrained models and then building models with these features as input are not allowed.*\nWhile the generation/extraction of bottleneck features is fine on its own, it would be against the rules to do so with a pretrained model.\n\n*3. Approaches like mapping between the classes of pre trained model predicted classes and current data classes and building probability based models on top of them is not allowed* \nThis approach would be against the rules.\n\n*4. Regular transfer learning techniques with pretrained weights and frozen layers are allowed as long as the base model is extended and trained on competition train data.*\nThis would be against the rules because pretrained weights are being used to warm-start the model (and continue to persist in the frozen layers).\n\n*5. (not sure if this will be useful) there may not be any restriction on word embeddings if one wants to use in their models.* \nUsing a pretrained word embedding would be against the rules.  Training your own word embedding would be fine; using an untrained vectorization like bag-of-words or the hashing trick would also be fine.",
    "382525": "Thank you very much @JamesAtwood for the quick response and for detailed answers to each of the  queries. <br> your explanation made rules related to model training crystal clear. once again thank you.",
    "383194": ""
  },
  "source": "meta"
}