{
  "id": 41044,
  "title": "how to factorise trained convolution filters?",
  "url": "/competitions/cdiscount-image-classification-challenge/discussion/41044",
  "author_name": "",
  "post_date": "2017-10-11T23:23:50.989498Z",
  "votes": null,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Most pretrained CNN has first conv layer with size 7x7. I want to factorise them to a series of 3x3 filter. Is there any code or paper that mentions how it is done?</p>\n\n<p>Mathematically it is formulated as: think of a sequence 3x3 filter weights that will approximated convolution results of a given 7x7 filter weights. It is related to separable filter, matrix reduction, etc</p>\n\n<p>I remembered i came across such works before but cannot recalled the paper title exactly.</p>\n\n<p>Thanks!</p>",
  "messages": [
    {
      "id": "230424",
      "postDate": "10/11/2017 23:23:50",
      "content": "<p>Most pretrained CNN has first conv layer with size 7x7. I want to factorise them to a series of 3x3 filter. Is there any code or paper that mentions how it is done?</p>\n\n<p>Mathematically it is formulated as: think of a sequence 3x3 filter weights that will approximated convolution results of a given 7x7 filter weights. It is related to separable filter, matrix reduction, etc</p>\n\n<p>I remembered i came across such works before but cannot recalled the paper title exactly.</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "Most pretrained CNN has first conv layer with size 7x7. I want to factorise them to a series of 3x3 filter. Is there any code or paper that mentions how it is done?\n\nMathematically it is formulated as: think of a sequence 3x3 filter weights that will approximated convolution results of a given 7x7 filter weights. It is related to separable filter, matrix reduction, etc\n\nI remembered i came across such works before but cannot recalled the paper title exactly.\n\nThanks!",
      "votes": null
    },
    {
      "id": "230506",
      "postDate": "10/12/2017 04:20:31",
      "content": "<p>I think probably you can construct a 7x7 conv from three 3x3 convs but not always the other way around? Do you mean there is a way to systematically reduce the elements of a 7x7 conv to 3x3x3 convs?</p>",
      "rawMarkdown": "I think probably you can construct a 7x7 conv from three 3x3 convs but not always the other way around? Do you mean there is a way to systematically reduce the elements of a 7x7 conv to 3x3x3 convs?",
      "votes": null
    },
    {
      "id": "230508",
      "postDate": "10/12/2017 04:25:23",
      "content": "<p>yes there is</p>",
      "rawMarkdown": "yes there is",
      "votes": null
    },
    {
      "id": "230514",
      "postDate": "10/12/2017 04:52:22",
      "content": "<p>That is to solve 27 parameters with 49 equations, hard for me to imagine... Thanks for all these exploration, learned a lot from you.</p>",
      "rawMarkdown": "That is to solve 27 parameters with 49 equations, hard for me to imagine... Thanks for all these exploration, learned a lot from you.",
      "votes": null
    },
    {
      "id": "230516",
      "postDate": "10/12/2017 05:08:52",
      "content": "<p>some matrix factorisation method should work. my backup plan is to learn a 3x3 network to mimic 7x7 network by minimizing l2 loss.</p>\n\n<p>one reference: <a href=\"https://arxiv.org/abs/1505.06798\">https://arxiv.org/abs/1505.06798</a></p>",
      "rawMarkdown": "some matrix factorisation method should work. my backup plan is to learn a 3x3 network to mimic 7x7 network by minimizing l2 loss.\n\none reference: https://arxiv.org/abs/1505.06798",
      "votes": null
    },
    {
      "id": "231755",
      "postDate": "10/15/2017 22:40:31",
      "content": "<p>is it the winograd convolution ?</p>",
      "rawMarkdown": "is it the winograd convolution ?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 230506,
      "author_name": "ybw9000",
      "author_url": "",
      "post_date": "10/12/2017 04:20:31",
      "content": "<p>I think probably you can construct a 7x7 conv from three 3x3 convs but not always the other way around? Do you mean there is a way to systematically reduce the elements of a 7x7 conv to 3x3x3 convs?</p>",
      "votes": null,
      "replies": [
        {
          "id": 230508,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "10/12/2017 04:25:23",
          "content": "<p>yes there is</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 230514,
      "author_name": "ybw9000",
      "author_url": "",
      "post_date": "10/12/2017 04:52:22",
      "content": "<p>That is to solve 27 parameters with 49 equations, hard for me to imagine... Thanks for all these exploration, learned a lot from you.</p>",
      "votes": null,
      "replies": [
        {
          "id": 230516,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "10/12/2017 05:08:52",
          "content": "<p>some matrix factorisation method should work. my backup plan is to learn a 3x3 network to mimic 7x7 network by minimizing l2 loss.</p>\n\n<p>one reference: <a href=\"https://arxiv.org/abs/1505.06798\">https://arxiv.org/abs/1505.06798</a></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 231755,
      "author_name": "rteja1113",
      "author_url": "",
      "post_date": "10/15/2017 22:40:31",
      "content": "<p>is it the winograd convolution ?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "230424": "Most pretrained CNN has first conv layer with size 7x7. I want to factorise them to a series of 3x3 filter. Is there any code or paper that mentions how it is done?\n\nMathematically it is formulated as: think of a sequence 3x3 filter weights that will approximated convolution results of a given 7x7 filter weights. It is related to separable filter, matrix reduction, etc\n\nI remembered i came across such works before but cannot recalled the paper title exactly.\n\nThanks!",
    "230506": "I think probably you can construct a 7x7 conv from three 3x3 convs but not always the other way around? Do you mean there is a way to systematically reduce the elements of a 7x7 conv to 3x3x3 convs?",
    "230508": "yes there is",
    "230514": "That is to solve 27 parameters with 49 equations, hard for me to imagine... Thanks for all these exploration, learned a lot from you.",
    "230516": "some matrix factorisation method should work. my backup plan is to learn a 3x3 network to mimic 7x7 network by minimizing l2 loss.\n\none reference: https://arxiv.org/abs/1505.06798",
    "231755": "is it the winograd convolution ?"
  },
  "source": "meta"
}