{
  "id": 79838,
  "title": "Check if your kernels run on the new stage 2 data",
  "url": "/competitions/quora-insincere-questions-classification/discussion/79838",
  "author_name": "",
  "post_date": "2019-02-07T22:35:26.260417500Z",
  "votes": 8,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Hi,</p>\n\n<p>Now that the competition is over, the new 376k row test set has been released. Although we can't submit predictions on the stage 2 test set, we can still run our kernels again to check if they ran within the time limit and if there were any errors.</p>",
  "messages": [
    {
      "id": "467888",
      "postDate": "02/07/2019 22:35:26",
      "content": "<p>Hi,</p>\n\n<p>Now that the competition is over, the new 376k row test set has been released. Although we can't submit predictions on the stage 2 test set, we can still run our kernels again to check if they ran within the time limit and if there were any errors.</p>",
      "rawMarkdown": "Hi,\n\nNow that the competition is over, the new 376k row test set has been released. Although we can't submit predictions on the stage 2 test set, we can still run our kernels again to check if they ran within the time limit and if there were any errors.",
      "votes": null
    },
    {
      "id": "468142",
      "postDate": "02/08/2019 11:06:18",
      "content": "<p>I tested my kernel on the new test data, and it runs in time (with 200s to spare). I'm so relieved. Phew !</p>",
      "rawMarkdown": "I tested my kernel on the new test data, and it runs in time (with 200s to spare). I'm so relieved. Phew !",
      "votes": null
    },
    {
      "id": "468190",
      "postDate": "02/08/2019 13:02:37",
      "content": "<p>Seems like kernels are really slow! Hard to tell whether a kernel will succeed or not just by commiting it ...\nedit:\nduring competition my final model had a mean(runtime) of ~ 6400 seconds.\n- stage 2 test-set (full = 375806 rows) mean(runtime) ~ 7303 seconds.\n- stage 2 test-set (subset = 56370 rows) mean(runtime) ~ 7002 seconds.</p>\n\n<p>As you can see ... even the subsetted stage 2 version is ~ 600 seconds slower compared to stage 1. </p>\n\n<p>edit #2:\ndid 5 more runs (subset = 56370 rows) \nMin. = 6892 seconds\nMean = 7006 seconds\nMax. = 7195 seconds\nI really hope my kernels won't crash! </p>",
      "rawMarkdown": "Seems like kernels are really slow! Hard to tell whether a kernel will succeed or not just by commiting it ...\nedit:\nduring competition my final model had a mean(runtime) of ~ 6400 seconds.\n- stage 2 test-set (full = 375806 rows) mean(runtime) ~ 7303 seconds.\n- stage 2 test-set (subset = 56370 rows) mean(runtime) ~ 7002 seconds.\n\nAs you can see ... even the subsetted stage 2 version is ~ 600 seconds slower compared to stage 1. \n\nedit #2:\ndid 5 more runs (subset = 56370 rows) \nMin. = 6892 seconds\nMean = 7006 seconds\nMax. = 7195 seconds\nI really hope my kernels won't crash!",
      "votes": null
    },
    {
      "id": "468211",
      "postDate": "02/08/2019 13:53:26",
      "content": "<p>Do you limit test set in the beginning to 56k rows or after processing and stuff?</p>",
      "rawMarkdown": "Do you limit test set in the beginning to 56k rows or after processing and stuff?",
      "votes": null
    },
    {
      "id": "468213",
      "postDate": "02/08/2019 13:56:09",
      "content": "<p>In the beginning. I simply read the top X rows of the test-set.</p>",
      "rawMarkdown": "In the beginning. I simply read the top X rows of the test-set.",
      "votes": null
    },
    {
      "id": "468217",
      "postDate": "02/08/2019 14:08:38",
      "content": "<p>Then it's weird. Runtimes look like expected for me when I tried it two days ago. Edit: still run fine</p>",
      "rawMarkdown": "Then it's weird. Runtimes look like expected for me when I tried it two days ago. Edit: still run fine",
      "votes": null
    },
    {
      "id": "468662",
      "postDate": "02/09/2019 12:03:22",
      "content": "<p>I did some further investigations because my stage 2 runtime is pretty bad ...\nSeems like kaggle updated some packages inbetween stages (i'm using R by the way).</p>\n\n<p>Stage 1:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11224/stage1.PNG\" alt=\"some stage 1 package info\">\nStage 2:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11225/stage2.PNG\" alt=\"some stage 2 package info\"></p>",
      "rawMarkdown": "I did some further investigations because my stage 2 runtime is pretty bad ...\nSeems like kaggle updated some packages inbetween stages (i'm using R by the way).\n\nStage 1:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11224/stage1.PNG\" alt=\"some stage 1 package info\">\nStage 2:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11225/stage2.PNG\" alt=\"some stage 2 package info\">",
      "votes": null
    },
    {
      "id": "469014",
      "postDate": "02/10/2019 10:02:10",
      "content": "<p>Thanks <a href=\"/bkkaggle\">@bkkaggle</a>, I did not realize the stage-2 test data was released until I saw your post.</p>\n\n<p>Anyway, I ran both my kernels and they finished with time to spare. My Keras kernel is really close but the pytorch kernel had over 600 seconds left. Another point for using pytorch in my next computer vision competition :-)</p>",
      "rawMarkdown": "Thanks @bkkaggle, I did not realize the stage-2 test data was released until I saw your post.\n\nAnyway, I ran both my kernels and they finished with time to spare. My Keras kernel is really close but the pytorch kernel had over 600 seconds left. Another point for using pytorch in my next computer vision competition :-)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 468142,
      "author_name": "tarunpaparaju",
      "author_url": "",
      "post_date": "02/08/2019 11:06:18",
      "content": "<p>I tested my kernel on the new test data, and it runs in time (with 200s to spare). I'm so relieved. Phew !</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 468190,
      "author_name": "springmanndaniel",
      "author_url": "",
      "post_date": "02/08/2019 13:02:37",
      "content": "<p>Seems like kernels are really slow! Hard to tell whether a kernel will succeed or not just by commiting it ...\nedit:\nduring competition my final model had a mean(runtime) of ~ 6400 seconds.\n- stage 2 test-set (full = 375806 rows) mean(runtime) ~ 7303 seconds.\n- stage 2 test-set (subset = 56370 rows) mean(runtime) ~ 7002 seconds.</p>\n\n<p>As you can see ... even the subsetted stage 2 version is ~ 600 seconds slower compared to stage 1. </p>\n\n<p>edit #2:\ndid 5 more runs (subset = 56370 rows) \nMin. = 6892 seconds\nMean = 7006 seconds\nMax. = 7195 seconds\nI really hope my kernels won't crash! </p>",
      "votes": null,
      "replies": [
        {
          "id": 468211,
          "author_name": "philippsinger",
          "author_url": "",
          "post_date": "02/08/2019 13:53:26",
          "content": "<p>Do you limit test set in the beginning to 56k rows or after processing and stuff?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 468213,
          "author_name": "springmanndaniel",
          "author_url": "",
          "post_date": "02/08/2019 13:56:09",
          "content": "<p>In the beginning. I simply read the top X rows of the test-set.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 468217,
          "author_name": "philippsinger",
          "author_url": "",
          "post_date": "02/08/2019 14:08:38",
          "content": "<p>Then it's weird. Runtimes look like expected for me when I tried it two days ago. Edit: still run fine</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 468662,
      "author_name": "springmanndaniel",
      "author_url": "",
      "post_date": "02/09/2019 12:03:22",
      "content": "<p>I did some further investigations because my stage 2 runtime is pretty bad ...\nSeems like kaggle updated some packages inbetween stages (i'm using R by the way).</p>\n\n<p>Stage 1:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11224/stage1.PNG\" alt=\"some stage 1 package info\">\nStage 2:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11225/stage2.PNG\" alt=\"some stage 2 package info\"></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 469014,
      "author_name": "sheriytm",
      "author_url": "",
      "post_date": "02/10/2019 10:02:10",
      "content": "<p>Thanks <a href=\"/bkkaggle\">@bkkaggle</a>, I did not realize the stage-2 test data was released until I saw your post.</p>\n\n<p>Anyway, I ran both my kernels and they finished with time to spare. My Keras kernel is really close but the pytorch kernel had over 600 seconds left. Another point for using pytorch in my next computer vision competition :-)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "467888": "Hi,\n\nNow that the competition is over, the new 376k row test set has been released. Although we can't submit predictions on the stage 2 test set, we can still run our kernels again to check if they ran within the time limit and if there were any errors.",
    "468142": "I tested my kernel on the new test data, and it runs in time (with 200s to spare). I'm so relieved. Phew !",
    "468190": "Seems like kernels are really slow! Hard to tell whether a kernel will succeed or not just by commiting it ...\nedit:\nduring competition my final model had a mean(runtime) of ~ 6400 seconds.\n- stage 2 test-set (full = 375806 rows) mean(runtime) ~ 7303 seconds.\n- stage 2 test-set (subset = 56370 rows) mean(runtime) ~ 7002 seconds.\n\nAs you can see ... even the subsetted stage 2 version is ~ 600 seconds slower compared to stage 1. \n\nedit #2:\ndid 5 more runs (subset = 56370 rows) \nMin. = 6892 seconds\nMean = 7006 seconds\nMax. = 7195 seconds\nI really hope my kernels won't crash!",
    "468211": "Do you limit test set in the beginning to 56k rows or after processing and stuff?",
    "468213": "In the beginning. I simply read the top X rows of the test-set.",
    "468217": "Then it's weird. Runtimes look like expected for me when I tried it two days ago. Edit: still run fine",
    "468662": "I did some further investigations because my stage 2 runtime is pretty bad ...\nSeems like kaggle updated some packages inbetween stages (i'm using R by the way).\n\nStage 1:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11224/stage1.PNG\" alt=\"some stage 1 package info\">\nStage 2:\n<img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/468662/11225/stage2.PNG\" alt=\"some stage 2 package info\">",
    "469014": "Thanks @bkkaggle, I did not realize the stage-2 test data was released until I saw your post.\n\nAnyway, I ran both my kernels and they finished with time to spare. My Keras kernel is really close but the pytorch kernel had over 600 seconds left. Another point for using pytorch in my next computer vision competition :-)"
  },
  "source": "meta"
}