{
  "id": 316041,
  "title": "Why does gem pooling remove negative values?",
  "url": "/competitions/happy-whale-and-dolphin/discussion/316041",
  "author_name": "datadote",
  "post_date": "2022-03-31T01:27:24.473000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi all,<br>\nI'm looking at the author's implementation (<a href=\"https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py\" target=\"_blank\">https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py</a>)</p>\n<p>def gem(x, p=3, eps=1e-6):<br>\n    return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)</p>\n<p>I can't understand the intuition behind the x.clamp. X.clamp removes all negative values. I can understand that a small value taken to a power can lead to numerical instability. Is the clamp mainly added for convenience, and it seems to work? The implementation doesn't strictly follow the GeM formula.</p>",
  "messages": [
    {
      "id": 1740476,
      "postDate": "2022-03-31T01:27:24.473Z",
      "content": "<p>Hi all,<br>\nI'm looking at the author's implementation (<a href=\"https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py\" target=\"_blank\">https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py</a>)</p>\n<p>def gem(x, p=3, eps=1e-6):<br>\n    return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)</p>\n<p>I can't understand the intuition behind the x.clamp. X.clamp removes all negative values. I can understand that a small value taken to a power can lead to numerical instability. Is the clamp mainly added for convenience, and it seems to work? The implementation doesn't strictly follow the GeM formula.</p>",
      "rawMarkdown": "Hi all,\nI'm looking at the author's implementation (https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py)\n\ndef gem(x, p=3, eps=1e-6):\n    return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)\n\nI can't understand the intuition behind the x.clamp. X.clamp removes all negative values. I can understand that a small value taken to a power can lead to numerical instability. Is the clamp mainly added for convenience, and it seems to work? The implementation doesn't strictly follow the GeM formula.",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1740476": "Hi all,\nI'm looking at the author's implementation (https://github.com/filipradenovic/cnnimageretrieval-pytorch/blob/master/cirtorch/layers/functional.py)\n\ndef gem(x, p=3, eps=1e-6):\n    return F.avg_pool2d(x.clamp(min=eps).pow(p), (x.size(-2), x.size(-1))).pow(1./p)\n\nI can't understand the intuition behind the x.clamp. X.clamp removes all negative values. I can understand that a small value taken to a power can lead to numerical instability. Is the clamp mainly added for convenience, and it seems to work? The implementation doesn't strictly follow the GeM formula."
  }
}