{
  "id": 258889,
  "title": "Tips for training on a large  amount of data on budget ",
  "url": "/competitions/g2net-gravitational-wave-detection/discussion/258889",
  "author_name": "",
  "post_date": "2021-08-03T13:17:34.278406600Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Step 1 - Get a colab Pro sub<br>\nStep 2 - Launch 4 or 5 sessions on which you can train each fold <br>\nStep 3 - ????<br>\nStep 4 - Profit<br>\n Some code snippets to help you <br>\n1.donwload Train Data <code>\n!nvidia-smi data_link = \"Link in the comments.Its updated every few days so i will keep posting them\"\n!wget \"$data_link\" -O train.zip \n!unzip -qq /content/train.zip -d /content/Train \n!rm /content/train.zip</code><br>\n2.Spilt the fold and see which to train on `skf = KFold(n_splits=4, random_state=42, shuffle=True)<br>\nfold = 0<br>\nfor train, test in skf.split(train_idx, y=y):</p>\n<pre><code>if fold ==0 #which fold you want to train on:\n    your code here\n\nfold += 1`\n</code></pre>\n<p>Suggestion are welcome <br>\nThe training time  is 2 hours per epoch with Efn. B7 <br>\nimage size = 69(nice),193,1<br>\nCQT1992v2 Transform</p>",
  "messages": [
    {
      "id": "1432439",
      "postDate": "08/03/2021 13:17:34",
      "content": "<p>Step 1 - Get a colab Pro sub<br>\nStep 2 - Launch 4 or 5 sessions on which you can train each fold <br>\nStep 3 - ????<br>\nStep 4 - Profit<br>\n Some code snippets to help you <br>\n1.donwload Train Data <code>\n!nvidia-smi data_link = \"Link in the comments.Its updated every few days so i will keep posting them\"\n!wget \"$data_link\" -O train.zip \n!unzip -qq /content/train.zip -d /content/Train \n!rm /content/train.zip</code><br>\n2.Spilt the fold and see which to train on `skf = KFold(n_splits=4, random_state=42, shuffle=True)<br>\nfold = 0<br>\nfor train, test in skf.split(train_idx, y=y):</p>\n<pre><code>if fold ==0 #which fold you want to train on:\n    your code here\n\nfold += 1`\n</code></pre>\n<p>Suggestion are welcome <br>\nThe training time  is 2 hours per epoch with Efn. B7 <br>\nimage size = 69(nice),193,1<br>\nCQT1992v2 Transform</p>",
      "rawMarkdown": "Step 1 - Get a colab Pro sub\nStep 2 - Launch 4 or 5 sessions on which you can train each fold \nStep 3 - ????\nStep 4 - Profit\n Some code snippets to help you \n1.donwload Train Data `\n!nvidia-smi data_link = \"Link in the comments.Its updated every few days so i will keep posting them\"\n!wget \"$data_link\" -O train.zip \n!unzip -qq /content/train.zip -d /content/Train \n!rm /content/train.zip`\n2.Spilt the fold and see which to train on `skf = KFold(n_splits=4, random_state=42, shuffle=True)\nfold = 0\nfor train, test in skf.split(train_idx, y=y):\n\n\tif fold ==0 #which fold you want to train on:\n\t\tyour code here\n\n\tfold += 1`\nSuggestion are welcome \nThe training time  is 2 hours per epoch with Efn. B7 \nimage size = 69(nice),193,1\nCQT1992v2 Transform",
      "votes": null
    },
    {
      "id": "1432505",
      "postDate": "08/03/2021 13:20:30",
      "content": "<p>Link for today <code>\"data_link = \"https://storage.googleapis.com/kaggle-competitions-data/kaggle-v2/23249/2399555/compressed/train.zip?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&amp;Expires=1628218497&amp;Signature=RTGPN1hhO%2BvEHvwIvWIWa9OCTXZTbyV1Vr38iqgO0v%2FM2FGb2Mb7urBJSNIloyfpkF32bybInqE746EAjmJvyLAfx%2B9yw7n7TxAisdCrmV4ONAX3J4owzoYC2frr79bvvhNlOQY6wdk22Y3oay0EkeZ4UEWPZB1e97%2Fz9jmKtcxr9fUGxFI6G5QmapnYAphZSu9SHzBNTjGunns3GI4l8Za1ws2mRIZ7m0%2FeUI6Z1r%2BiiZOK2CYNGJAwqMhSR8NHbDph1Ca6VBYoC4nS%2F44r81QvZJ0AYPc%2B4DJ%2FszMlSpXvQlUrmS554Fsdd9651GJnlYE0gX58mK%2BZL6zlnwK18Q%3D%3D&amp;response-content-disposition=attachment%3B+filename%3Dtrain.zip\"\n\"</code></p>",
      "rawMarkdown": "Link for today `\"data_link = \"https://storage.googleapis.com/kaggle-competitions-data/kaggle-v2/23249/2399555/compressed/train.zip?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&Expires=1628218497&Signature=RTGPN1hhO%2BvEHvwIvWIWa9OCTXZTbyV1Vr38iqgO0v%2FM2FGb2Mb7urBJSNIloyfpkF32bybInqE746EAjmJvyLAfx%2B9yw7n7TxAisdCrmV4ONAX3J4owzoYC2frr79bvvhNlOQY6wdk22Y3oay0EkeZ4UEWPZB1e97%2Fz9jmKtcxr9fUGxFI6G5QmapnYAphZSu9SHzBNTjGunns3GI4l8Za1ws2mRIZ7m0%2FeUI6Z1r%2BiiZOK2CYNGJAwqMhSR8NHbDph1Ca6VBYoC4nS%2F44r81QvZJ0AYPc%2B4DJ%2FszMlSpXvQlUrmS554Fsdd9651GJnlYE0gX58mK%2BZL6zlnwK18Q%3D%3D&response-content-disposition=attachment%3B+filename%3Dtrain.zip\"\n\"`",
      "votes": null
    },
    {
      "id": "1561296",
      "postDate": "10/27/2021 13:12:57",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "rawMarkdown": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1432505,
      "author_name": "mithilsalunkhe",
      "author_url": "",
      "post_date": "08/03/2021 13:20:30",
      "content": "<p>Link for today <code>\"data_link = \"https://storage.googleapis.com/kaggle-competitions-data/kaggle-v2/23249/2399555/compressed/train.zip?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&amp;Expires=1628218497&amp;Signature=RTGPN1hhO%2BvEHvwIvWIWa9OCTXZTbyV1Vr38iqgO0v%2FM2FGb2Mb7urBJSNIloyfpkF32bybInqE746EAjmJvyLAfx%2B9yw7n7TxAisdCrmV4ONAX3J4owzoYC2frr79bvvhNlOQY6wdk22Y3oay0EkeZ4UEWPZB1e97%2Fz9jmKtcxr9fUGxFI6G5QmapnYAphZSu9SHzBNTjGunns3GI4l8Za1ws2mRIZ7m0%2FeUI6Z1r%2BiiZOK2CYNGJAwqMhSR8NHbDph1Ca6VBYoC4nS%2F44r81QvZJ0AYPc%2B4DJ%2FszMlSpXvQlUrmS554Fsdd9651GJnlYE0gX58mK%2BZL6zlnwK18Q%3D%3D&amp;response-content-disposition=attachment%3B+filename%3Dtrain.zip\"\n\"</code></p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1561296,
      "author_name": "zerafachris",
      "author_url": "",
      "post_date": "10/27/2021 13:12:57",
      "content": "<p>Hey All,</p>\n<p>Thank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey <a href=\"https://forms.gle/QP9L16niPexozyhu5\" target=\"_blank\">https://forms.gle/QP9L16niPexozyhu5</a>.</p>\n<p>Thank you all,</p>\n<p>Regards,<br>\nChris</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1432439": "Step 1 - Get a colab Pro sub\nStep 2 - Launch 4 or 5 sessions on which you can train each fold \nStep 3 - ????\nStep 4 - Profit\n Some code snippets to help you \n1.donwload Train Data `\n!nvidia-smi data_link = \"Link in the comments.Its updated every few days so i will keep posting them\"\n!wget \"$data_link\" -O train.zip \n!unzip -qq /content/train.zip -d /content/Train \n!rm /content/train.zip`\n2.Spilt the fold and see which to train on `skf = KFold(n_splits=4, random_state=42, shuffle=True)\nfold = 0\nfor train, test in skf.split(train_idx, y=y):\n\n\tif fold ==0 #which fold you want to train on:\n\t\tyour code here\n\n\tfold += 1`\nSuggestion are welcome \nThe training time  is 2 hours per epoch with Efn. B7 \nimage size = 69(nice),193,1\nCQT1992v2 Transform",
    "1432505": "Link for today `\"data_link = \"https://storage.googleapis.com/kaggle-competitions-data/kaggle-v2/23249/2399555/compressed/train.zip?GoogleAccessId=web-data@kaggle-161607.iam.gserviceaccount.com&Expires=1628218497&Signature=RTGPN1hhO%2BvEHvwIvWIWa9OCTXZTbyV1Vr38iqgO0v%2FM2FGb2Mb7urBJSNIloyfpkF32bybInqE746EAjmJvyLAfx%2B9yw7n7TxAisdCrmV4ONAX3J4owzoYC2frr79bvvhNlOQY6wdk22Y3oay0EkeZ4UEWPZB1e97%2Fz9jmKtcxr9fUGxFI6G5QmapnYAphZSu9SHzBNTjGunns3GI4l8Za1ws2mRIZ7m0%2FeUI6Z1r%2BiiZOK2CYNGJAwqMhSR8NHbDph1Ca6VBYoC4nS%2F44r81QvZJ0AYPc%2B4DJ%2FszMlSpXvQlUrmS554Fsdd9651GJnlYE0gX58mK%2BZL6zlnwK18Q%3D%3D&response-content-disposition=attachment%3B+filename%3Dtrain.zip\"\n\"`",
    "1561296": "Hey All,\n\nThank you all for taking part in our competition. The participation has been overwhelmingly positive. We are currently conducting a survey to gauge the demographic and outreach achieved. Kindly spare 2min and fill in this survey https://forms.gle/QP9L16niPexozyhu5.\n\nThank you all,\n\nRegards,\nChris"
  },
  "source": "meta"
}