{
  "id": 134089,
  "title": "an idea from quantization network training",
  "url": "/competitions/bengaliai-cv19/discussion/134089",
  "author_name": "hengck23",
  "post_date": "2020-03-05T23:25:06.434000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>i was recently reading this paper: </p>\n\n<p><a href=\"https://github.com/amirgholami/ZeroQ\">https://github.com/amirgholami/ZeroQ</a>\nZeroQ: A Novel Zero Shot Quantization Framework</p>\n\n<p>The objective is to create a set of modified images x such that\n1.  original_net(x)=y.  original_net has  conv-bn-relu layer\n2. quant_net(x)=y_bar. quant_net has qconv-relu layer\n3. distribution loss, e.g. KL (y, y_bar) is min</p>\n\n<p>the formulation is such that gradient is back propagated all the way back to the input image, which is then updated to minimise the KL loss:. i.e modify x = x -rate*grad</p>\n\n<hr>\n\n<p>now we can apply  the same idea to learn augmentation. The objective is to create a set of modified images x, so that x matches another set of image z\n(assume x,z are split of images of the same class. idea can be extended to x,z being different class later)</p>\n\n<ol>\n<li>net(x)=f</li>\n<li>net(z)=f_bar. f and f_bar are some embedding vector</li>\n<li>distance loss, e.g. L2(f, f_bar) is min for some z</li>\n</ol>\n\n<p>back propagation :  x = x -rate*grad. we create new x such that feature of new x is closer to feature of x for some z</p>",
  "messages": [
    {
      "id": 764805,
      "postDate": "2020-03-05T23:25:06.433Z",
      "content": "<p>i was recently reading this paper: </p>\n\n<p><a href=\"https://github.com/amirgholami/ZeroQ\">https://github.com/amirgholami/ZeroQ</a>\nZeroQ: A Novel Zero Shot Quantization Framework</p>\n\n<p>The objective is to create a set of modified images x such that\n1.  original_net(x)=y.  original_net has  conv-bn-relu layer\n2. quant_net(x)=y_bar. quant_net has qconv-relu layer\n3. distribution loss, e.g. KL (y, y_bar) is min</p>\n\n<p>the formulation is such that gradient is back propagated all the way back to the input image, which is then updated to minimise the KL loss:. i.e modify x = x -rate*grad</p>\n\n<hr>\n\n<p>now we can apply  the same idea to learn augmentation. The objective is to create a set of modified images x, so that x matches another set of image z\n(assume x,z are split of images of the same class. idea can be extended to x,z being different class later)</p>\n\n<ol>\n<li>net(x)=f</li>\n<li>net(z)=f_bar. f and f_bar are some embedding vector</li>\n<li>distance loss, e.g. L2(f, f_bar) is min for some z</li>\n</ol>\n\n<p>back propagation :  x = x -rate*grad. we create new x such that feature of new x is closer to feature of x for some z</p>",
      "rawMarkdown": "i was recently reading this paper: \n\nhttps://github.com/amirgholami/ZeroQ\nZeroQ: A Novel Zero Shot Quantization Framework\n\nThe objective is to create a set of modified images x such that\n1.  original\\_net(x)=y.  original\\_net has  conv-bn-relu layer\n2. quant\\_net(x)=y\\_bar. quant\\_net has qconv-relu layer\n3. distribution loss, e.g. KL (y, y\\_bar) is min\n \nthe formulation is such that gradient is back propagated all the way back to the input image, which is then updated to minimise the KL loss:. i.e modify x = x -rate*grad\n\n---\n\nnow we can apply  the same idea to learn augmentation. The objective is to create a set of modified images x, so that x matches another set of image z\n(assume x,z are split of images of the same class. idea can be extended to x,z being different class later)\n\n1.  net(x)=f\n2.  net(z)=f\\_bar. f and f\\_bar are some embedding vector\n3. distance loss, e.g. L2(f, f\\_bar) is min for some z\n \nback propagation :  x = x -rate*grad. we create new x such that feature of new x is closer to feature of x for some z\n\n\n\n\n\n",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "764805": "i was recently reading this paper: \n\nhttps://github.com/amirgholami/ZeroQ\nZeroQ: A Novel Zero Shot Quantization Framework\n\nThe objective is to create a set of modified images x such that\n1.  original\\_net(x)=y.  original\\_net has  conv-bn-relu layer\n2. quant\\_net(x)=y\\_bar. quant\\_net has qconv-relu layer\n3. distribution loss, e.g. KL (y, y\\_bar) is min\n \nthe formulation is such that gradient is back propagated all the way back to the input image, which is then updated to minimise the KL loss:. i.e modify x = x -rate*grad\n\n---\n\nnow we can apply  the same idea to learn augmentation. The objective is to create a set of modified images x, so that x matches another set of image z\n(assume x,z are split of images of the same class. idea can be extended to x,z being different class later)\n\n1.  net(x)=f\n2.  net(z)=f\\_bar. f and f\\_bar are some embedding vector\n3. distance loss, e.g. L2(f, f\\_bar) is min for some z\n \nback propagation :  x = x -rate*grad. we create new x such that feature of new x is closer to feature of x for some z\n\n\n\n\n\n"
  }
}