{
  "id": 250371,
  "title": "how much time does it take to unzip this data",
  "url": "/competitions/g2net-gravitational-wave-detection/discussion/250371",
  "author_name": "Mithil Salunkhe",
  "post_date": "2021-07-02T11:42:56.362000",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I wanted to ask how much time does it take to extract the train data with the !unzip command ?</p>",
  "messages": [
    {
      "id": 1373292,
      "postDate": "2021-07-02T11:42:56.363Z",
      "content": "<p>I wanted to ask how much time does it take to extract the train data with the !unzip command ?</p>",
      "rawMarkdown": " I wanted to ask how much time does it take to extract the train data with the !unzip command ?",
      "votes": 3
    },
    {
      "id": 1563261,
      "postDate": "2021-10-28T06:52:10.147Z",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
    },
    {
      "id": 1376221,
      "postDate": "2021-07-05T01:00:29.080Z",
      "content": "<p>[beginner alert!] How are guys planning to process this data? I mean can you actually write python scripts/notebooks and treat it like you would any other dataset? Is working with this large dataset possible on a regular laptop (i7, 16GB RAM, SSD)? Thanks </p>",
      "rawMarkdown": "[beginner alert!] How are guys planning to process this data? I mean can you actually write python scripts/notebooks and treat it like you would any other dataset? Is working with this large dataset possible on a regular laptop (i7, 16GB RAM, SSD)? Thanks ",
      "replies": [
        {
          "id": 1377392,
          "postDate": "2021-07-05T19:33:16.453Z",
          "content": "<p>Laura's post: <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250244\" target=\"_blank\">https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250244</a><br>\nYaroslav's post: <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250130\" target=\"_blank\">https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250130</a></p>\n<p>Those two posts were extremely helpful in understanding and getting started with how to process the data. </p>\n<p>While working with a large dataset is possible, it would take a decent amount of time (even longer if it's just done on CPU locally). My recommendation would be to start off with the Kaggle kernel or google colab notebooks which offer free compute resources and learning and improving. It would be the best way to start off, and later on you can either invest in online compute resources (g cloud, azure, colab pro, aws, etc.) or have a local setup. Regarding this competition specifically Bojan's reply in <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250008\" target=\"_blank\">this post</a> provides a great explanation!</p>",
          "rawMarkdown": "Laura's post: https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250244\nYaroslav's post: https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250130\n\nThose two posts were extremely helpful in understanding and getting started with how to process the data. \n\nWhile working with a large dataset is possible, it would take a decent amount of time (even longer if it's just done on CPU locally). My recommendation would be to start off with the Kaggle kernel or google colab notebooks which offer free compute resources and learning and improving. It would be the best way to start off, and later on you can either invest in online compute resources (g cloud, azure, colab pro, aws, etc.) or have a local setup. Regarding this competition specifically Bojan's reply in [this post](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250008) provides a great explanation!",
          "votes": 2
        },
        {
          "id": 1377553,
          "postDate": "2021-07-06T00:36:46.270Z",
          "content": "<p>I've read posts, thanks for the reply</p>",
          "rawMarkdown": "I've read posts, thanks for the reply"
        }
      ]
    },
    {
      "id": 1374623,
      "postDate": "2021-07-03T13:18:10.407Z",
      "content": "<p>Hi!<br>\nIt took less than 10 minutes to unzip all the data on intel core i7 machine with an SSD.<br>\nThe unzipped test folder weights 22G, train weights 54G.</p>",
      "rawMarkdown": "Hi!\nIt took less than 10 minutes to unzip all the data on intel core i7 machine with an SSD.\nThe unzipped test folder weights 22G, train weights 54G."
    },
    {
      "id": 1376106,
      "postDate": "2021-07-04T19:17:42.633Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1563261,
      "author_name": "ChristopherZerafa",
      "author_url": "",
      "post_date": "2021-10-28T06:52:10.147000",
      "content": "<p>Hey,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1376221,
      "author_name": "Sid Patel",
      "author_url": "",
      "post_date": "2021-07-05T01:00:29.080000",
      "content": "<p>[beginner alert!] How are guys planning to process this data? I mean can you actually write python scripts/notebooks and treat it like you would any other dataset? Is working with this large dataset possible on a regular laptop (i7, 16GB RAM, SSD)? Thanks </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1377392,
          "author_name": "Anand Sunderrajan",
          "author_url": "",
          "post_date": "2021-07-05T19:33:16.453000",
          "content": "<p>Laura's post: <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250244\" target=\"_blank\">https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250244</a><br>\nYaroslav's post: <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250130\" target=\"_blank\">https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250130</a></p>\n<p>Those two posts were extremely helpful in understanding and getting started with how to process the data. </p>\n<p>While working with a large dataset is possible, it would take a decent amount of time (even longer if it's just done on CPU locally). My recommendation would be to start off with the Kaggle kernel or google colab notebooks which offer free compute resources and learning and improving. It would be the best way to start off, and later on you can either invest in online compute resources (g cloud, azure, colab pro, aws, etc.) or have a local setup. Regarding this competition specifically Bojan's reply in <a href=\"https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/250008\" target=\"_blank\">this post</a> provides a great explanation!</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1377553,
          "author_name": "Sid Patel",
          "author_url": "",
          "post_date": "2021-07-06T00:36:46.270000",
          "content": "<p>I've read posts, thanks for the reply</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1374623,
      "author_name": "Andrey",
      "author_url": "",
      "post_date": "2021-07-03T13:18:10.407000",
      "content": "<p>Hi!<br>\nIt took less than 10 minutes to unzip all the data on intel core i7 machine with an SSD.<br>\nThe unzipped test folder weights 22G, train weights 54G.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1376106,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-07-04T19:17:42.633000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1373292": " I wanted to ask how much time does it take to extract the train data with the !unzip command ?",
    "1563261": "Hey,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
    "1376221": "[beginner alert!] How are guys planning to process this data? I mean can you actually write python scripts/notebooks and treat it like you would any other dataset? Is working with this large dataset possible on a regular laptop (i7, 16GB RAM, SSD)? Thanks ",
    "1374623": "Hi!\nIt took less than 10 minutes to unzip all the data on intel core i7 machine with an SSD.\nThe unzipped test folder weights 22G, train weights 54G.",
    "1376106": ""
  }
}