{
  "id": 410436,
  "title": "Some errors and ideas",
  "url": "/competitions/vesuvius-challenge-ink-detection/discussion/410436",
  "author_name": "",
  "post_date": "2023-05-15T10:32:18.631911300Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi all. I'm new to the competition and I'm starting this one with <a href=\"https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset\" target=\"_blank\">https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset</a>.</p>\n<p>I have tried changing my validation holdout area and found that my LB score has improved slightly (to 0.16 max). I also tried eliminating some training areas (especially Train Volume 1, which repeats Test Volume), but no improvement.</p>\n<p>I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"</p>\n<p>I also think I need to use the K-fold Cross Validation as shown in <a href=\"https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation-\" target=\"_blank\">https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation-</a> with-keras.md.</p>\n<p>I think it would be a good idea to use a pre-trained U-Net to reduce the calculation time and memory.</p>\n<p>What do you think about all this?</p>",
  "messages": [
    {
      "id": "2259937",
      "postDate": "05/15/2023 10:32:18",
      "content": "<p>Hi all. I'm new to the competition and I'm starting this one with <a href=\"https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset\" target=\"_blank\">https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset</a>.</p>\n<p>I have tried changing my validation holdout area and found that my LB score has improved slightly (to 0.16 max). I also tried eliminating some training areas (especially Train Volume 1, which repeats Test Volume), but no improvement.</p>\n<p>I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"</p>\n<p>I also think I need to use the K-fold Cross Validation as shown in <a href=\"https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation-\" target=\"_blank\">https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation-</a> with-keras.md.</p>\n<p>I think it would be a good idea to use a pre-trained U-Net to reduce the calculation time and memory.</p>\n<p>What do you think about all this?</p>",
      "rawMarkdown": "Hi all. I'm new to the competition and I'm starting this one with https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset.\n\nI have tried changing my validation holdout area and found that my LB score has improved slightly (to 0.16 max). I also tried eliminating some training areas (especially Train Volume 1, which repeats Test Volume), but no improvement.\n\nI tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"\n\nI also think I need to use the K-fold Cross Validation as shown in https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation- with-keras.md.\n\nI think it would be a good idea to use a pre-trained U-Net to reduce the calculation time and memory.\n\nWhat do you think about all this?",
      "votes": null
    },
    {
      "id": "2260595",
      "postDate": "05/15/2023 18:36:41",
      "content": "<blockquote>\n  <p>I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"</p>\n</blockquote>\n<p>And strangely enough, despite the error in this version, the the submission was succeeded.</p>",
      "rawMarkdown": ">I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"\n\nAnd strangely enough, despite the error in this version, the the submission was succeeded.",
      "votes": null
    },
    {
      "id": "2261626",
      "postDate": "05/16/2023 13:17:53",
      "content": "<blockquote>\n  <p>I also think I need to use the K-fold Cross Validation</p>\n</blockquote>\n<p>Any idea how this can be done in keras, since I haven't found the right code in these contests for this?</p>",
      "rawMarkdown": ">I also think I need to use the K-fold Cross Validation\n\nAny idea how this can be done in keras, since I haven't found the right code in these contests for this?",
      "votes": null
    },
    {
      "id": "2262099",
      "postDate": "05/16/2023 18:21:20",
      "content": "<p>good job ^W^, keep going!</p>",
      "rawMarkdown": "good job ^W^, keep going!",
      "votes": null
    },
    {
      "id": "2262113",
      "postDate": "05/16/2023 18:31:59",
      "content": "<p>I think using keras to make K-fold Cross Validation is not OK if you are using the common model from here. It might cause Data Leakage unless the stride is equal to your model's input size.</p>",
      "rawMarkdown": "I think using keras to make K-fold Cross Validation is not OK if you are using the common model from here. It might cause Data Leakage unless the stride is equal to your model's input size.",
      "votes": null
    },
    {
      "id": "2262879",
      "postDate": "05/17/2023 07:27:06",
      "content": "<p>Thank you. Any idea why this might be happening?</p>",
      "rawMarkdown": "Thank you. Any idea why this might be happening?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2260595,
      "author_name": "kalakagat",
      "author_url": "",
      "post_date": "05/15/2023 18:36:41",
      "content": "<blockquote>\n  <p>I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"</p>\n</blockquote>\n<p>And strangely enough, despite the error in this version, the the submission was succeeded.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2262099,
          "author_name": "yoyobar",
          "author_url": "",
          "post_date": "05/16/2023 18:21:20",
          "content": "<p>good job ^W^, keep going!</p>",
          "votes": null,
          "replies": [
            {
              "id": 2262879,
              "author_name": "kalakagat",
              "author_url": "",
              "post_date": "05/17/2023 07:27:06",
              "content": "<p>Thank you. Any idea why this might be happening?</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2261626,
      "author_name": "kalakagat",
      "author_url": "",
      "post_date": "05/16/2023 13:17:53",
      "content": "<blockquote>\n  <p>I also think I need to use the K-fold Cross Validation</p>\n</blockquote>\n<p>Any idea how this can be done in keras, since I haven't found the right code in these contests for this?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2262113,
          "author_name": "yoyobar",
          "author_url": "",
          "post_date": "05/16/2023 18:31:59",
          "content": "<p>I think using keras to make K-fold Cross Validation is not OK if you are using the common model from here. It might cause Data Leakage unless the stride is equal to your model's input size.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2259937": "Hi all. I'm new to the competition and I'm starting this one with https://www.kaggle.com/code/fchollet/keras-starter-kit-unet-train-on-full-dataset.\n\nI have tried changing my validation holdout area and found that my LB score has improved slightly (to 0.16 max). I also tried eliminating some training areas (especially Train Volume 1, which repeats Test Volume), but no improvement.\n\nI tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"\n\nI also think I need to use the K-fold Cross Validation as shown in https://github.com/christianversloot/machine-learning-articles/blob/main/how-to-use-k-fold-cross-validation- with-keras.md.\n\nI think it would be a good idea to use a pre-trained U-Net to reduce the calculation time and memory.\n\nWhat do you think about all this?",
    "2260595": ">I tried increasing the number of slices to train from 20 to 23 (Z_DIM = 23 # number of slices in z direction), but I get the error message \"Your notebook tried to allocate more memory than is available.0\"\n\nAnd strangely enough, despite the error in this version, the the submission was succeeded.",
    "2261626": ">I also think I need to use the K-fold Cross Validation\n\nAny idea how this can be done in keras, since I haven't found the right code in these contests for this?",
    "2262099": "good job ^W^, keep going!",
    "2262113": "I think using keras to make K-fold Cross Validation is not OK if you are using the common model from here. It might cause Data Leakage unless the stride is equal to your model's input size.",
    "2262879": "Thank you. Any idea why this might be happening?"
  },
  "source": "meta"
}