{
  "id": 135568,
  "title": "can one use tfa optimizers with tpu?",
  "url": "/competitions/flower-classification-with-tpus/discussion/135568",
  "author_name": "",
  "post_date": "2020-03-14T16:28:20.195208500Z",
  "votes": null,
  "comment_count": 9,
  "views": 0,
  "content": "<p>I tried to use this combination to get an ranger optimizer but the training did not start:</p>\n\n<p>radam = tfa.optimizers.RectifiedAdam(lr=1e-3)\nranger = tfa.optimizers.Lookahead(radam)</p>\n\n<p>Any ideas?</p>",
  "messages": [
    {
      "id": "771791",
      "postDate": "03/14/2020 16:28:20",
      "content": "<p>I tried to use this combination to get an ranger optimizer but the training did not start:</p>\n\n<p>radam = tfa.optimizers.RectifiedAdam(lr=1e-3)\nranger = tfa.optimizers.Lookahead(radam)</p>\n\n<p>Any ideas?</p>",
      "rawMarkdown": "I tried to use this combination to get an ranger optimizer but the training did not start:\n\nradam = tfa.optimizers.RectifiedAdam(lr=1e-3)\nranger = tfa.optimizers.Lookahead(radam)\n\nAny ideas?",
      "votes": null
    },
    {
      "id": "772780",
      "postDate": "03/15/2020 22:41:59",
      "content": "<p>Can elaborate more on how the training did not start ?</p>",
      "rawMarkdown": "Can elaborate more on how the training did not start ?",
      "votes": null
    },
    {
      "id": "775208",
      "postDate": "03/16/2020 12:14:02",
      "content": "<p>After calling .fit() I had some cpu load which stopped after 1-2 minutes (prob loading training data) and then I saw no CPU, TPU activity at all for some minutes, so I canceled the run.</p>",
      "rawMarkdown": "After calling .fit() I had some cpu load which stopped after 1-2 minutes (prob loading training data) and then I saw no CPU, TPU activity at all for some minutes, so I canceled the run.",
      "votes": null
    },
    {
      "id": "775451",
      "postDate": "03/16/2020 17:36:41",
      "content": "<p>That is a weird behavior but i tried using it the same way and worked correctly.</p>",
      "rawMarkdown": "That is a weird behavior but i tried using it the same way and worked correctly.",
      "votes": null
    },
    {
      "id": "775503",
      "postDate": "03/16/2020 18:41:17",
      "content": "<p>Thx for the feedback. I will try again after tpu quota reset.</p>",
      "rawMarkdown": "Thx for the feedback. I will try again after tpu quota reset.",
      "votes": null
    },
    {
      "id": "775539",
      "postDate": "03/16/2020 19:29:13",
      "content": "<p>your welcome :D</p>",
      "rawMarkdown": "your welcome :D",
      "votes": null
    },
    {
      "id": "775681",
      "postDate": "03/16/2020 23:58:19",
      "content": "<p>When you can try again, please tell us how it goes. The first epoch can be slow because it includes the time needed to compile your model to TPU (XLA compilation).</p>",
      "rawMarkdown": "When you can try again, please tell us how it goes. The first epoch can be slow because it includes the time needed to compile your model to TPU (XLA compilation).",
      "votes": null
    },
    {
      "id": "782604",
      "postDate": "03/22/2020 13:29:05",
      "content": "<p>Ok - first epoch needed 1000 secs, every other 42 secs.\nSo it seems to work, although the first epoch needed more than twice the time compared to Adam.</p>\n\n<p>Thx for helping.</p>",
      "rawMarkdown": "Ok - first epoch needed 1000 secs, every other 42 secs.\nSo it seems to work, although the first epoch needed more than twice the time compared to Adam.\n\nThx for helping.",
      "votes": null
    },
    {
      "id": "1178648",
      "postDate": "01/31/2021 03:57:05",
      "content": "<p>Has anyone got info into this problem? I've been struggling with the same issue at the moment. The dataset loads and the cpu stops, I waited for 20 minutes and it still hasn't started. Apparently its the lookahead that causes the problem, since ive tried with rectified independently.</p>",
      "rawMarkdown": "Has anyone got info into this problem? I've been struggling with the same issue at the moment. The dataset loads and the cpu stops, I waited for 20 minutes and it still hasn't started. Apparently its the lookahead that causes the problem, since ive tried with rectified independently.",
      "votes": null
    },
    {
      "id": "1179830",
      "postDate": "01/31/2021 21:39:02",
      "content": "<p>Try training only on a sample of the images to check if its working or not.</p>",
      "rawMarkdown": "Try training only on a sample of the images to check if its working or not.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1178648,
      "author_name": "elysium1436",
      "author_url": "",
      "post_date": "01/31/2021 03:57:05",
      "content": "<p>Has anyone got info into this problem? I've been struggling with the same issue at the moment. The dataset loads and the cpu stops, I waited for 20 minutes and it still hasn't started. Apparently its the lookahead that causes the problem, since ive tried with rectified independently.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1179830,
          "author_name": "ibrahimsherify",
          "author_url": "",
          "post_date": "01/31/2021 21:39:02",
          "content": "<p>Try training only on a sample of the images to check if its working or not.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 772780,
      "author_name": "ibrahimsherify",
      "author_url": "",
      "post_date": "03/15/2020 22:41:59",
      "content": "<p>Can elaborate more on how the training did not start ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 775208,
      "author_name": "romanweilguny",
      "author_url": "",
      "post_date": "03/16/2020 12:14:02",
      "content": "<p>After calling .fit() I had some cpu load which stopped after 1-2 minutes (prob loading training data) and then I saw no CPU, TPU activity at all for some minutes, so I canceled the run.</p>",
      "votes": null,
      "replies": [
        {
          "id": 775451,
          "author_name": "ibrahimsherify",
          "author_url": "",
          "post_date": "03/16/2020 17:36:41",
          "content": "<p>That is a weird behavior but i tried using it the same way and worked correctly.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 775503,
      "author_name": "romanweilguny",
      "author_url": "",
      "post_date": "03/16/2020 18:41:17",
      "content": "<p>Thx for the feedback. I will try again after tpu quota reset.</p>",
      "votes": null,
      "replies": [
        {
          "id": 775539,
          "author_name": "ibrahimsherify",
          "author_url": "",
          "post_date": "03/16/2020 19:29:13",
          "content": "<p>your welcome :D</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 775681,
      "author_name": "mgorner",
      "author_url": "",
      "post_date": "03/16/2020 23:58:19",
      "content": "<p>When you can try again, please tell us how it goes. The first epoch can be slow because it includes the time needed to compile your model to TPU (XLA compilation).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 782604,
      "author_name": "romanweilguny",
      "author_url": "",
      "post_date": "03/22/2020 13:29:05",
      "content": "<p>Ok - first epoch needed 1000 secs, every other 42 secs.\nSo it seems to work, although the first epoch needed more than twice the time compared to Adam.</p>\n\n<p>Thx for helping.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "771791": "I tried to use this combination to get an ranger optimizer but the training did not start:\n\nradam = tfa.optimizers.RectifiedAdam(lr=1e-3)\nranger = tfa.optimizers.Lookahead(radam)\n\nAny ideas?",
    "772780": "Can elaborate more on how the training did not start ?",
    "775208": "After calling .fit() I had some cpu load which stopped after 1-2 minutes (prob loading training data) and then I saw no CPU, TPU activity at all for some minutes, so I canceled the run.",
    "775451": "That is a weird behavior but i tried using it the same way and worked correctly.",
    "775503": "Thx for the feedback. I will try again after tpu quota reset.",
    "775539": "your welcome :D",
    "775681": "When you can try again, please tell us how it goes. The first epoch can be slow because it includes the time needed to compile your model to TPU (XLA compilation).",
    "782604": "Ok - first epoch needed 1000 secs, every other 42 secs.\nSo it seems to work, although the first epoch needed more than twice the time compared to Adam.\n\nThx for helping.",
    "1178648": "Has anyone got info into this problem? I've been struggling with the same issue at the moment. The dataset loads and the cpu stops, I waited for 20 minutes and it still hasn't started. Apparently its the lookahead that causes the problem, since ive tried with rectified independently.",
    "1179830": "Try training only on a sample of the images to check if its working or not."
  },
  "source": "meta"
}