{
  "id": 271946,
  "title": "TPU crisis Continues...FYA Kaggle Team",
  "url": "/competitions/g2net-gravitational-wave-detection/discussion/271946",
  "author_name": "",
  "post_date": "2021-09-13T10:36:44.041632900Z",
  "votes": 12,
  "comment_count": 7,
  "views": 0,
  "content": "<p>It takes long time to start the TPUs.. I lost TPU hours left last week because it couldnt start before reset time of TPU.</p>\n<p>GPUs also dont take so long to start even though there is a traffic. </p>",
  "messages": [
    {
      "id": "1511330",
      "postDate": "09/13/2021 10:36:44",
      "content": "<p>It takes long time to start the TPUs.. I lost TPU hours left last week because it couldnt start before reset time of TPU.</p>\n<p>GPUs also dont take so long to start even though there is a traffic. </p>",
      "rawMarkdown": "It takes long time to start the TPUs.. I lost TPU hours left last week because it couldnt start before reset time of TPU.\n\nGPUs also dont take so long to start even though there is a traffic.",
      "votes": null
    },
    {
      "id": "1511337",
      "postDate": "09/13/2021 10:40:21",
      "content": "<p>There is no need to Post this again. You have already posted this. Kaggle team must be looking into this. i think its better to post this here <a href=\"https://www.kaggle.com/product-feedback\" target=\"_blank\">https://www.kaggle.com/product-feedback</a></p>",
      "rawMarkdown": "There is no need to Post this again. You have already posted this. Kaggle team must be looking into this. i think its better to post this here https://www.kaggle.com/product-feedback",
      "votes": null
    },
    {
      "id": "1511363",
      "postDate": "09/13/2021 11:12:51",
      "content": "<p>You don't know that Kaggle team is looking into it.  Have you seen a message from them?  It is right to ping them again IMHO.</p>\n<p>Now that the week end is over they will certainly look into it.</p>",
      "rawMarkdown": "You don't know that Kaggle team is looking into it.  Have you seen a message from them?  It is right to ping them again IMHO.\n\nNow that the week end is over they will certainly look into it.",
      "votes": null
    },
    {
      "id": "1511389",
      "postDate": "09/13/2021 11:38:25",
      "content": "<p><a href=\"https://www.kaggle.com/mithilsalunkhe\" target=\"_blank\">@mithilsalunkhe</a> if you check my subject it says clearly TPU crisis Continues.. which is continuation to what i posted before . Some time kaggle team feels issue is intermittent will get autoresolved but when they see it is still continuing then they might look more.. my one kernel is stuck since 3 hours.. its complete waste of time.  I think you  know very well  that almost all work by taking their time off from their regular work.</p>",
      "rawMarkdown": "mithilsalunkhe if you check my subject it says clearly TPU crisis Continues.. which is continuation to what i posted before . Some time kaggle team feels issue is intermittent will get autoresolved but when they see it is still continuing then they might look more.. my one kernel is stuck since 3 hours.. its complete waste of time.  I think you  know very well  that almost all work by taking their time off from their regular work.",
      "votes": null
    },
    {
      "id": "1511877",
      "postDate": "09/13/2021 18:37:34",
      "content": "<p>I think TPU time only counts after commit has started. At least this is what I observe - my kernel was stuck for few hours before start of commit and than it eventually started. And only after that remaining TPU quota started decreasing.</p>",
      "rawMarkdown": "I think TPU time only counts after commit has started. At least this is what I observe - my kernel was stuck for few hours before start of commit and than it eventually started. And only after that remaining TPU quota started decreasing.",
      "votes": null
    },
    {
      "id": "1512640",
      "postDate": "09/14/2021 12:51:55",
      "content": "<p>Yes, that's expected behaviour. <br>\nYou can see that your commit is queued if you open <strong>View Active Events</strong> tab in the lower-left corner of the site.<br>\nQueued notebooks don't consume the TPU quota; I've checked that a few times last week.</p>\n<p>Our TPU notebooks get queued each time these days; it looks like the TPU demand for this competition (and Google Landmark ones) is extra high.</p>",
      "rawMarkdown": "Yes, that's expected behaviour. \nYou can see that your commit is queued if you open **View Active Events** tab in the lower-left corner of the site.\nQueued notebooks don't consume the TPU quota; I've checked that a few times last week.\n\nOur TPU notebooks get queued each time these days; it looks like the TPU demand for this competition (and Google Landmark ones) is extra high.",
      "votes": null
    },
    {
      "id": "1512869",
      "postDate": "09/14/2021 16:47:44",
      "content": "<p>I was running my recommender system the notebook stopped  working even without using TPU's. Still can't figure it out what to do.</p>",
      "rawMarkdown": "I was running my recommender system the notebook stopped  working even without using TPU's. Still can't figure it out what to do.",
      "votes": null
    },
    {
      "id": "1559713",
      "postDate": "10/27/2021 07:08:42",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1511337,
      "author_name": "mithilsalunkhe",
      "author_url": "",
      "post_date": "09/13/2021 10:40:21",
      "content": "<p>There is no need to Post this again. You have already posted this. Kaggle team must be looking into this. i think its better to post this here <a href=\"https://www.kaggle.com/product-feedback\" target=\"_blank\">https://www.kaggle.com/product-feedback</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 1511363,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "09/13/2021 11:12:51",
          "content": "<p>You don't know that Kaggle team is looking into it.  Have you seen a message from them?  It is right to ping them again IMHO.</p>\n<p>Now that the week end is over they will certainly look into it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1511389,
          "author_name": "jaideepvalani",
          "author_url": "",
          "post_date": "09/13/2021 11:38:25",
          "content": "<p><a href=\"https://www.kaggle.com/mithilsalunkhe\" target=\"_blank\">@mithilsalunkhe</a> if you check my subject it says clearly TPU crisis Continues.. which is continuation to what i posted before . Some time kaggle team feels issue is intermittent will get autoresolved but when they see it is still continuing then they might look more.. my one kernel is stuck since 3 hours.. its complete waste of time.  I think you  know very well  that almost all work by taking their time off from their regular work.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1511877,
      "author_name": "vzaguskin",
      "author_url": "",
      "post_date": "09/13/2021 18:37:34",
      "content": "<p>I think TPU time only counts after commit has started. At least this is what I observe - my kernel was stuck for few hours before start of commit and than it eventually started. And only after that remaining TPU quota started decreasing.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1512640,
          "author_name": "rolandw0w",
          "author_url": "",
          "post_date": "09/14/2021 12:51:55",
          "content": "<p>Yes, that's expected behaviour. <br>\nYou can see that your commit is queued if you open <strong>View Active Events</strong> tab in the lower-left corner of the site.<br>\nQueued notebooks don't consume the TPU quota; I've checked that a few times last week.</p>\n<p>Our TPU notebooks get queued each time these days; it looks like the TPU demand for this competition (and Google Landmark ones) is extra high.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1512869,
      "author_name": "saadsikander",
      "author_url": "",
      "post_date": "09/14/2021 16:47:44",
      "content": "<p>I was running my recommender system the notebook stopped  working even without using TPU's. Still can't figure it out what to do.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1559713,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/27/2021 07:08:42",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1511330": "It takes long time to start the TPUs.. I lost TPU hours left last week because it couldnt start before reset time of TPU.\n\nGPUs also dont take so long to start even though there is a traffic.",
    "1511337": "There is no need to Post this again. You have already posted this. Kaggle team must be looking into this. i think its better to post this here https://www.kaggle.com/product-feedback",
    "1511363": "You don't know that Kaggle team is looking into it.  Have you seen a message from them?  It is right to ping them again IMHO.\n\nNow that the week end is over they will certainly look into it.",
    "1511389": "mithilsalunkhe if you check my subject it says clearly TPU crisis Continues.. which is continuation to what i posted before . Some time kaggle team feels issue is intermittent will get autoresolved but when they see it is still continuing then they might look more.. my one kernel is stuck since 3 hours.. its complete waste of time.  I think you  know very well  that almost all work by taking their time off from their regular work.",
    "1511877": "I think TPU time only counts after commit has started. At least this is what I observe - my kernel was stuck for few hours before start of commit and than it eventually started. And only after that remaining TPU quota started decreasing.",
    "1512640": "Yes, that's expected behaviour. \nYou can see that your commit is queued if you open **View Active Events** tab in the lower-left corner of the site.\nQueued notebooks don't consume the TPU quota; I've checked that a few times last week.\n\nOur TPU notebooks get queued each time these days; it looks like the TPU demand for this competition (and Google Landmark ones) is extra high.",
    "1512869": "I was running my recommender system the notebook stopped  working even without using TPU's. Still can't figure it out what to do.",
    "1559713": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
  },
  "source": "meta"
}