{
  "id": 107018,
  "title": "Different image format in dataset is bad or not?",
  "url": "/competitions/aptos2019-blindness-detection/discussion/107018",
  "author_name": "",
  "post_date": "2019-09-01T15:50:22.616643700Z",
  "votes": null,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I want to use images from the previous competition. These images are jpeg data. On the other hand, this competetion's images are png data. I think that the same image format is beter (not sure...). So, I converted previous competition data to png. But, the data size after conversion is so large as not to import to my dataset ( up to 20GB).</p>\n\n<p>Does mixed image format in the training dataset make bad effect?\nIf not, I will use jpeg format for image of the previous conpetition (no conversion).</p>",
  "messages": [
    {
      "id": "615203",
      "postDate": "09/01/2019 15:50:22",
      "content": "<p>I want to use images from the previous competition. These images are jpeg data. On the other hand, this competetion's images are png data. I think that the same image format is beter (not sure...). So, I converted previous competition data to png. But, the data size after conversion is so large as not to import to my dataset ( up to 20GB).</p>\n\n<p>Does mixed image format in the training dataset make bad effect?\nIf not, I will use jpeg format for image of the previous conpetition (no conversion).</p>",
      "rawMarkdown": "I want to use images from the previous competition. These images are jpeg data. On the other hand, this competetion's images are png data. I think that the same image format is beter (not sure...). So, I converted previous competition data to png. But, the data size after conversion is so large as not to import to my dataset ( up to 20GB).\n\n\nDoes mixed image format in the training dataset make bad effect?\nIf not, I will use jpeg format for image of the previous conpetition (no conversion).",
      "votes": null
    },
    {
      "id": "615276",
      "postDate": "09/01/2019 18:11:13",
      "content": "<p>In the end what matters are the contents of the images, sure JPEG could introduce some artifacts, but if you choose a sufficiently high quality level I don't think it should matter too much. </p>\n\n<p>Perhaps you can crop/resize the images and save that to the dataset instead of the raw images? That should help with filesize regardless of PNG or JPEG :)</p>",
      "rawMarkdown": "In the end what matters are the contents of the images, sure JPEG could introduce some artifacts, but if you choose a sufficiently high quality level I don't think it should matter too much. \n\nPerhaps you can crop/resize the images and save that to the dataset instead of the raw images? That should help with filesize regardless of PNG or JPEG :)",
      "votes": null
    },
    {
      "id": "615482",
      "postDate": "09/02/2019 02:19:43",
      "content": "<p>Thank you.\nI use Efficientnet B5, and I already croped and resized to 456x456(PNG).  As the result of this, the sum of the data size became over 21GB. \nI'll try to use JPEG for training.</p>",
      "rawMarkdown": "Thank you.\nI use Efficientnet B5, and I already croped and resized to 456x456(PNG).  As the result of this, the sum of the data size became over 21GB. \nI'll try to use JPEG for training.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 615276,
      "author_name": "gzuidhof",
      "author_url": "",
      "post_date": "09/01/2019 18:11:13",
      "content": "<p>In the end what matters are the contents of the images, sure JPEG could introduce some artifacts, but if you choose a sufficiently high quality level I don't think it should matter too much. </p>\n\n<p>Perhaps you can crop/resize the images and save that to the dataset instead of the raw images? That should help with filesize regardless of PNG or JPEG :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 615482,
          "author_name": "dannadori",
          "author_url": "",
          "post_date": "09/02/2019 02:19:43",
          "content": "<p>Thank you.\nI use Efficientnet B5, and I already croped and resized to 456x456(PNG).  As the result of this, the sum of the data size became over 21GB. \nI'll try to use JPEG for training.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "615203": "I want to use images from the previous competition. These images are jpeg data. On the other hand, this competetion's images are png data. I think that the same image format is beter (not sure...). So, I converted previous competition data to png. But, the data size after conversion is so large as not to import to my dataset ( up to 20GB).\n\n\nDoes mixed image format in the training dataset make bad effect?\nIf not, I will use jpeg format for image of the previous conpetition (no conversion).",
    "615276": "In the end what matters are the contents of the images, sure JPEG could introduce some artifacts, but if you choose a sufficiently high quality level I don't think it should matter too much. \n\nPerhaps you can crop/resize the images and save that to the dataset instead of the raw images? That should help with filesize regardless of PNG or JPEG :)",
    "615482": "Thank you.\nI use Efficientnet B5, and I already croped and resized to 456x456(PNG).  As the result of this, the sum of the data size became over 21GB. \nI'll try to use JPEG for training."
  },
  "source": "meta"
}