{
  "id": 199459,
  "title": "Stratified TF Records",
  "url": "/competitions/cassava-leaf-disease-classification/discussion/199459",
  "author_name": "Alberto Benayas",
  "post_date": "2020-11-25T19:34:45.821000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Hi all, </p>\n<p>I have uploaded the training dataset in TF Records format. I know these files already exist out there, but I have divided the dataset into 5 stratified folds.<br>\nI have also created the datasets in 5 different resolutions.</p>\n<p><a href=\"https://www.kaggle.com/benayas/cassava-data-512\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-512</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-384\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-384</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-256\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-256</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-224\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-224</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-192\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-192</a></p>\n<p>A snippet to read data from these tfrec:</p>\n<pre><code>def read_tfrecord(example, labeled=True, one_hot=False, dim=512):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.int64),\n        \"image_name\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = tf.io.decode_raw(image, out_type=tf.uint8)\n    image = tf.cast(image, tf.float32) / 255.0\n    image = tf.reshape(image, (dim,dim, 3))\n    image = tf.image.resize(image, (dim,dim))\n\n    if labeled:\n        label = tf.cast(example['target'], tf.int32)\n        if one_hot:\n            label = tf.one_hot(label, depth=5)\n        return image, label\n    idnum = example['image_name']\n    return image, idnum\n</code></pre>",
  "messages": [
    {
      "id": 1091136,
      "postDate": "2020-11-25T19:34:45.823Z",
      "content": "<p>Hi all, </p>\n<p>I have uploaded the training dataset in TF Records format. I know these files already exist out there, but I have divided the dataset into 5 stratified folds.<br>\nI have also created the datasets in 5 different resolutions.</p>\n<p><a href=\"https://www.kaggle.com/benayas/cassava-data-512\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-512</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-384\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-384</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-256\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-256</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-224\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-224</a><br>\n<a href=\"https://www.kaggle.com/benayas/cassava-data-192\" target=\"_blank\">https://www.kaggle.com/benayas/cassava-data-192</a></p>\n<p>A snippet to read data from these tfrec:</p>\n<pre><code>def read_tfrecord(example, labeled=True, one_hot=False, dim=512):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.int64),\n        \"image_name\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = tf.io.decode_raw(image, out_type=tf.uint8)\n    image = tf.cast(image, tf.float32) / 255.0\n    image = tf.reshape(image, (dim,dim, 3))\n    image = tf.image.resize(image, (dim,dim))\n\n    if labeled:\n        label = tf.cast(example['target'], tf.int32)\n        if one_hot:\n            label = tf.one_hot(label, depth=5)\n        return image, label\n    idnum = example['image_name']\n    return image, idnum\n</code></pre>",
      "rawMarkdown": "Hi all, \n\nI have uploaded the training dataset in TF Records format. I know these files already exist out there, but I have divided the dataset into 5 stratified folds.\nI have also created the datasets in 5 different resolutions.\n\n[https://www.kaggle.com/benayas/cassava-data-512](https://www.kaggle.com/benayas/cassava-data-512)\n[https://www.kaggle.com/benayas/cassava-data-384](https://www.kaggle.com/benayas/cassava-data-384)\n[https://www.kaggle.com/benayas/cassava-data-256](https://www.kaggle.com/benayas/cassava-data-256)\n[https://www.kaggle.com/benayas/cassava-data-224](https://www.kaggle.com/benayas/cassava-data-224)\n[https://www.kaggle.com/benayas/cassava-data-192](https://www.kaggle.com/benayas/cassava-data-192)\n\nA snippet to read data from these tfrec:\n\n```\ndef read_tfrecord(example, labeled=True, one_hot=False, dim=512):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.int64),\n        \"image_name\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = tf.io.decode_raw(image, out_type=tf.uint8)\n    image = tf.cast(image, tf.float32) / 255.0\n    image = tf.reshape(image, (dim,dim, 3))\n    image = tf.image.resize(image, (dim,dim))\n\n    if labeled:\n        label = tf.cast(example['target'], tf.int32)\n        if one_hot:\n            label = tf.one_hot(label, depth=5)\n        return image, label\n    idnum = example['image_name']\n    return image, idnum\n```\n",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1091136": "Hi all, \n\nI have uploaded the training dataset in TF Records format. I know these files already exist out there, but I have divided the dataset into 5 stratified folds.\nI have also created the datasets in 5 different resolutions.\n\n[https://www.kaggle.com/benayas/cassava-data-512](https://www.kaggle.com/benayas/cassava-data-512)\n[https://www.kaggle.com/benayas/cassava-data-384](https://www.kaggle.com/benayas/cassava-data-384)\n[https://www.kaggle.com/benayas/cassava-data-256](https://www.kaggle.com/benayas/cassava-data-256)\n[https://www.kaggle.com/benayas/cassava-data-224](https://www.kaggle.com/benayas/cassava-data-224)\n[https://www.kaggle.com/benayas/cassava-data-192](https://www.kaggle.com/benayas/cassava-data-192)\n\nA snippet to read data from these tfrec:\n\n```\ndef read_tfrecord(example, labeled=True, one_hot=False, dim=512):\n    tfrecord_format = {\n        \"image\": tf.io.FixedLenFeature([], tf.string),\n        \"target\": tf.io.FixedLenFeature([], tf.int64),\n        \"image_name\": tf.io.FixedLenFeature([], tf.string)\n    }\n    example = tf.io.parse_single_example(example, tfrecord_format)\n    image = tf.io.decode_raw(image, out_type=tf.uint8)\n    image = tf.cast(image, tf.float32) / 255.0\n    image = tf.reshape(image, (dim,dim, 3))\n    image = tf.image.resize(image, (dim,dim))\n\n    if labeled:\n        label = tf.cast(example['target'], tf.int32)\n        if one_hot:\n            label = tf.one_hot(label, depth=5)\n        return image, label\n    idnum = example['image_name']\n    return image, idnum\n```\n"
  }
}