{
  "id": 165994,
  "title": "TPU High LR doesn't seem to work very well?",
  "url": "/competitions/alaska2-image-steganalysis/discussion/165994",
  "author_name": "",
  "post_date": "2020-07-11T14:19:13.888330200Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>So is anyone else experiencing LR scaling problems with TPUs? I'm finding that doubling the batch size doesn't allow me to double the learning rate or really increase it at all without losing performance. </p>\n\n<p>It seems that this dataset performs well on LR &lt; 1e-3 and anything more seems to cause learning issues. </p>",
  "messages": [
    {
      "id": "924589",
      "postDate": "07/11/2020 14:19:13",
      "content": "<p>So is anyone else experiencing LR scaling problems with TPUs? I'm finding that doubling the batch size doesn't allow me to double the learning rate or really increase it at all without losing performance. </p>\n\n<p>It seems that this dataset performs well on LR &lt; 1e-3 and anything more seems to cause learning issues. </p>",
      "rawMarkdown": "So is anyone else experiencing LR scaling problems with TPUs? I'm finding that doubling the batch size doesn't allow me to double the learning rate or really increase it at all without losing performance. \n\nIt seems that this dataset performs well on LR &lt; 1e-3 and anything more seems to cause learning issues.",
      "votes": null
    },
    {
      "id": "931812",
      "postDate": "07/16/2020 13:20:02",
      "content": "<p>Experienced the exact same thing, pika.</p>",
      "rawMarkdown": "Experienced the exact same thing, pika.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 931812,
      "author_name": "authman",
      "author_url": "",
      "post_date": "07/16/2020 13:20:02",
      "content": "<p>Experienced the exact same thing, pika.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "924589": "So is anyone else experiencing LR scaling problems with TPUs? I'm finding that doubling the batch size doesn't allow me to double the learning rate or really increase it at all without losing performance. \n\nIt seems that this dataset performs well on LR &lt; 1e-3 and anything more seems to cause learning issues.",
    "931812": "Experienced the exact same thing, pika."
  },
  "source": "meta"
}