{
  "id": 118454,
  "title": "is Colab good for this competition ?",
  "url": "/competitions/tensorflow2-question-answering/discussion/118454",
  "author_name": "",
  "post_date": "2019-11-21T15:01:36.121622600Z",
  "votes": 4,
  "comment_count": 18,
  "views": 0,
  "content": "<p>Anyone trained bert model on colab successfully ?</p>",
  "messages": [
    {
      "id": "678568",
      "postDate": "11/21/2019 15:01:36",
      "content": "<p>Anyone trained bert model on colab successfully ?</p>",
      "rawMarkdown": "Anyone trained bert model on colab successfully ?",
      "votes": null
    },
    {
      "id": "680287",
      "postDate": "11/24/2019 11:43:30",
      "content": "<p>No success so far. It fails on both TPU ( resources, could be the number of parameter servers or similar ), and K80 GPU ( short on memory ). It also fails on P100.    </p>",
      "rawMarkdown": "No success so far. It fails on both TPU ( resources, could be the number of parameter servers or similar ), and K80 GPU ( short on memory ). It also fails on P100.",
      "votes": null
    },
    {
      "id": "680311",
      "postDate": "11/24/2019 12:15:40",
      "content": "<p>Yes same for me too :( <a href=\"/mosicr\">@mosicr</a></p>",
      "rawMarkdown": "Yes same for me too :( @mosicr",
      "votes": null
    },
    {
      "id": "688563",
      "postDate": "12/05/2019 18:44:17",
      "content": "<p>It works on Google cloud platform :) </p>",
      "rawMarkdown": "It works on Google cloud platform :)",
      "votes": null
    },
    {
      "id": "688705",
      "postDate": "12/06/2019 00:44:35",
      "content": "<p>I think one problem is this thing about not getting the same GPU every time, some times I can use bs 4, and other times I can't</p>",
      "rawMarkdown": "I think one problem is this thing about not getting the same GPU every time, some times I can use bs 4, and other times I can't",
      "votes": null
    },
    {
      "id": "689026",
      "postDate": "12/06/2019 11:20:04",
      "content": "<p>Colab disconnects session and bans access for a while after 12 hours of GPU usage.</p>",
      "rawMarkdown": "Colab disconnects session and bans access for a while after 12 hours of GPU usage.",
      "votes": null
    },
    {
      "id": "689139",
      "postDate": "12/06/2019 14:23:48",
      "content": "<p>what type of gpu using and cost per hour ? <a href=\"/zhangmen\">@zhangmen</a> </p>",
      "rawMarkdown": "what type of gpu using and cost per hour ? @zhangmen",
      "votes": null
    },
    {
      "id": "689303",
      "postDate": "12/06/2019 19:03:19",
      "content": "<p>what is your problem via colab, you can share.\nI trained my model epoch1 and get 0.56LB (not examine cv yet) via colab TPU</p>",
      "rawMarkdown": "what is your problem via colab, you can share.\nI trained my model epoch1 and get 0.56LB (not examine cv yet) via colab TPU",
      "votes": null
    },
    {
      "id": "689338",
      "postDate": "12/06/2019 19:44:20",
      "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> that nice, how long it run and you didn't faced any session/memory errors?</p>",
      "rawMarkdown": "httpwwwfszyc that nice, how long it run and you didn't faced any session/memory errors?",
      "votes": null
    },
    {
      "id": "689425",
      "postDate": "12/06/2019 23:12:09",
      "content": "<p>approx. 4200 sec per epoch, no session error</p>",
      "rawMarkdown": "approx. 4200 sec per epoch, no session error",
      "votes": null
    },
    {
      "id": "689523",
      "postDate": "12/07/2019 02:16:15",
      "content": "<p>nice and thanks </p>",
      "rawMarkdown": "nice and thanks",
      "votes": null
    },
    {
      "id": "709621",
      "postDate": "01/03/2020 18:21:40",
      "content": "<p>you get $300 free credit, which is enough for this competition. I've used ~$50 so far. </p>",
      "rawMarkdown": "you get $300 free credit, which is enough for this competition. I've used ~$50 so far.",
      "votes": null
    },
    {
      "id": "709934",
      "postDate": "01/04/2020 05:09:47",
      "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> are you talking about bert-base or bert-large?</p>",
      "rawMarkdown": "httpwwwfszyc are you talking about bert-base or bert-large?",
      "votes": null
    },
    {
      "id": "710258",
      "postDate": "01/04/2020 14:09:37",
      "content": "<p>base</p>",
      "rawMarkdown": "base",
      "votes": null
    },
    {
      "id": "710438",
      "postDate": "01/04/2020 18:29:01",
      "content": "<p>What about the large one? Is it unstable? I'm trying the base for now since it's faster</p>",
      "rawMarkdown": "What about the large one? Is it unstable? I'm trying the base for now since it's faster",
      "votes": null
    },
    {
      "id": "710507",
      "postDate": "01/04/2020 20:15:02",
      "content": "<p>6h per 1 milion samples</p>",
      "rawMarkdown": "6h per 1 milion samples",
      "votes": null
    },
    {
      "id": "710892",
      "postDate": "01/05/2020 11:59:38",
      "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> what batch size worked for you I tried 32 and 64 none of them work on TPU are you using 16? note: I am using free TPU tier.</p>",
      "rawMarkdown": "httpwwwfszyc what batch size worked for you I tried 32 and 64 none of them work on TPU are you using 16? note: I am using free TPU tier.",
      "votes": null
    },
    {
      "id": "711425",
      "postDate": "01/06/2020 04:35:48",
      "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> I thought TPU in Google Colab doesn't support tensorflow 2.x.x. Do you use tensorflow 1.x.x??</p>",
      "rawMarkdown": "httpwwwfszyc I thought TPU in Google Colab doesn't support tensorflow 2.x.x. Do you use tensorflow 1.x.x??",
      "votes": null
    },
    {
      "id": "711460",
      "postDate": "01/06/2020 06:12:08",
      "content": "<p><a href=\"/kitkatbar0429\">@kitkatbar0429</a> it supports TF 2.1</p>",
      "rawMarkdown": "kitkatbar0429 it supports TF 2.1",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 680287,
      "author_name": "mosicr",
      "author_url": "",
      "post_date": "11/24/2019 11:43:30",
      "content": "<p>No success so far. It fails on both TPU ( resources, could be the number of parameter servers or similar ), and K80 GPU ( short on memory ). It also fails on P100.    </p>",
      "votes": null,
      "replies": [
        {
          "id": 680311,
          "author_name": "seshurajup",
          "author_url": "",
          "post_date": "11/24/2019 12:15:40",
          "content": "<p>Yes same for me too :( <a href=\"/mosicr\">@mosicr</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 688705,
          "author_name": "dimitreoliveira",
          "author_url": "",
          "post_date": "12/06/2019 00:44:35",
          "content": "<p>I think one problem is this thing about not getting the same GPU every time, some times I can use bs 4, and other times I can't</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 688563,
      "author_name": "zhangmen",
      "author_url": "",
      "post_date": "12/05/2019 18:44:17",
      "content": "<p>It works on Google cloud platform :) </p>",
      "votes": null,
      "replies": [
        {
          "id": 689139,
          "author_name": "seshurajup",
          "author_url": "",
          "post_date": "12/06/2019 14:23:48",
          "content": "<p>what type of gpu using and cost per hour ? <a href=\"/zhangmen\">@zhangmen</a> </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 709621,
          "author_name": "zhangmen",
          "author_url": "",
          "post_date": "01/03/2020 18:21:40",
          "content": "<p>you get $300 free credit, which is enough for this competition. I've used ~$50 so far. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 689026,
      "author_name": "mosicr",
      "author_url": "",
      "post_date": "12/06/2019 11:20:04",
      "content": "<p>Colab disconnects session and bans access for a while after 12 hours of GPU usage.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 689303,
      "author_name": "httpwwwfszyc",
      "author_url": "",
      "post_date": "12/06/2019 19:03:19",
      "content": "<p>what is your problem via colab, you can share.\nI trained my model epoch1 and get 0.56LB (not examine cv yet) via colab TPU</p>",
      "votes": null,
      "replies": [
        {
          "id": 689338,
          "author_name": "seshurajup",
          "author_url": "",
          "post_date": "12/06/2019 19:44:20",
          "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> that nice, how long it run and you didn't faced any session/memory errors?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 689425,
          "author_name": "httpwwwfszyc",
          "author_url": "",
          "post_date": "12/06/2019 23:12:09",
          "content": "<p>approx. 4200 sec per epoch, no session error</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 689523,
          "author_name": "seshurajup",
          "author_url": "",
          "post_date": "12/07/2019 02:16:15",
          "content": "<p>nice and thanks </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 709934,
          "author_name": "axel81",
          "author_url": "",
          "post_date": "01/04/2020 05:09:47",
          "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> are you talking about bert-base or bert-large?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 710258,
          "author_name": "httpwwwfszyc",
          "author_url": "",
          "post_date": "01/04/2020 14:09:37",
          "content": "<p>base</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 710438,
          "author_name": "ranik40",
          "author_url": "",
          "post_date": "01/04/2020 18:29:01",
          "content": "<p>What about the large one? Is it unstable? I'm trying the base for now since it's faster</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 710507,
          "author_name": "httpwwwfszyc",
          "author_url": "",
          "post_date": "01/04/2020 20:15:02",
          "content": "<p>6h per 1 milion samples</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 710892,
          "author_name": "axel81",
          "author_url": "",
          "post_date": "01/05/2020 11:59:38",
          "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> what batch size worked for you I tried 32 and 64 none of them work on TPU are you using 16? note: I am using free TPU tier.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 711425,
          "author_name": "kitkatbar0429",
          "author_url": "",
          "post_date": "01/06/2020 04:35:48",
          "content": "<p><a href=\"/httpwwwfszyc\">@httpwwwfszyc</a> I thought TPU in Google Colab doesn't support tensorflow 2.x.x. Do you use tensorflow 1.x.x??</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 711460,
          "author_name": "axel81",
          "author_url": "",
          "post_date": "01/06/2020 06:12:08",
          "content": "<p><a href=\"/kitkatbar0429\">@kitkatbar0429</a> it supports TF 2.1</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "678568": "Anyone trained bert model on colab successfully ?",
    "680287": "No success so far. It fails on both TPU ( resources, could be the number of parameter servers or similar ), and K80 GPU ( short on memory ). It also fails on P100.",
    "680311": "Yes same for me too :( @mosicr",
    "688563": "It works on Google cloud platform :)",
    "688705": "I think one problem is this thing about not getting the same GPU every time, some times I can use bs 4, and other times I can't",
    "689026": "Colab disconnects session and bans access for a while after 12 hours of GPU usage.",
    "689139": "what type of gpu using and cost per hour ? @zhangmen",
    "689303": "what is your problem via colab, you can share.\nI trained my model epoch1 and get 0.56LB (not examine cv yet) via colab TPU",
    "689338": "httpwwwfszyc that nice, how long it run and you didn't faced any session/memory errors?",
    "689425": "approx. 4200 sec per epoch, no session error",
    "689523": "nice and thanks",
    "709621": "you get $300 free credit, which is enough for this competition. I've used ~$50 so far.",
    "709934": "httpwwwfszyc are you talking about bert-base or bert-large?",
    "710258": "base",
    "710438": "What about the large one? Is it unstable? I'm trying the base for now since it's faster",
    "710507": "6h per 1 milion samples",
    "710892": "httpwwwfszyc what batch size worked for you I tried 32 and 64 none of them work on TPU are you using 16? note: I am using free TPU tier.",
    "711425": "httpwwwfszyc I thought TPU in Google Colab doesn't support tensorflow 2.x.x. Do you use tensorflow 1.x.x??",
    "711460": "kitkatbar0429 it supports TF 2.1"
  },
  "source": "meta"
}