{
  "id": 316402,
  "title": "Is it possible to load not tfrecords data with TPU?",
  "url": "/competitions/happy-whale-and-dolphin/discussion/316402",
  "author_name": "",
  "post_date": "2022-04-01T18:32:56.111124600Z",
  "votes": 3,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I have tried ImageDataGenerator /tf.keras.utils.image_dataset_from_directory / from dataframe .<br>\nAlso they are not supported tf.py_function therefore I can't map images with labels.</p>\n<p>Ok, I found the solution using .from_tensor_slice I can load images and labels simultaneously, but question about work in map function is still and what about third party augmentations?</p>",
  "messages": [
    {
      "id": "1742342",
      "postDate": "04/01/2022 18:32:56",
      "content": "<p>I have tried ImageDataGenerator /tf.keras.utils.image_dataset_from_directory / from dataframe .<br>\nAlso they are not supported tf.py_function therefore I can't map images with labels.</p>\n<p>Ok, I found the solution using .from_tensor_slice I can load images and labels simultaneously, but question about work in map function is still and what about third party augmentations?</p>",
      "rawMarkdown": "I have tried ImageDataGenerator /tf.keras.utils.image_dataset_from_directory / from dataframe .\nAlso they are not supported tf.py_function therefore I can't map images with labels.\n\nOk, I found the solution using .from_tensor_slice I can load images and labels simultaneously, but question about work in map function is still and what about third party augmentations?",
      "votes": null
    },
    {
      "id": "1742463",
      "postDate": "04/01/2022 22:43:14",
      "content": "<p>I think TPU (with TensorFlow as opposed to PyTorch) requires TF data dataset but does not require TFRecords. (And as you point out, TF data dataset can be built directly from images and targets).</p>\n<p>In the beginning, TF data dataset could not use third party augmentations. I believe it still cannot. That is why many kagglers have posted code to perform different augmentations like rotation, cutmix, coarse dropout, mixup, sheer, scaling, cropping etc in various Kaggle forums and notebooks.</p>",
      "rawMarkdown": "I think TPU (with TensorFlow as opposed to PyTorch) requires TF data dataset but does not require TFRecords. (And as you point out, TF data dataset can be built directly from images and targets).\n\nIn the beginning, TF data dataset could not use third party augmentations. I believe it still cannot. That is why many kagglers have posted code to perform different augmentations like rotation, cutmix, coarse dropout, mixup, sheer, scaling, cropping etc in various Kaggle forums and notebooks.",
      "votes": null
    },
    {
      "id": "1742498",
      "postDate": "04/01/2022 23:53:25",
      "content": "<p>A number of the augmentations you mention or a version of them,  are now in the <a href=\"https://www.tensorflow.org/addons/api_docs/python/tfa/image\" target=\"_blank\">TFA (TensorFlow Add-ons)\n</a></p>\n<p>Of course I greatly value those you have written Chris, and I still use them when I can.</p>",
      "rawMarkdown": "A number of the augmentations you mention or a version of them,  are now in the [TFA (TensorFlow Add-ons)\n](https://www.tensorflow.org/addons/api_docs/python/tfa/image)\n\nOf course I greatly value those you have written Chris, and I still use them when I can.",
      "votes": null
    },
    {
      "id": "1742502",
      "postDate": "04/02/2022 00:06:30",
      "content": "<p>Thanks for the info. Can TF data dataset accept functions from TFA? In the beginning it could not. Has this changed?</p>",
      "rawMarkdown": "Thanks for the info. Can TF data dataset accept functions from TFA? In the beginning it could not. Has this changed?",
      "votes": null
    },
    {
      "id": "1742509",
      "postDate": "04/02/2022 00:15:36",
      "content": "<p>I'm using rotate and a blur function from TFA for this comp.</p>\n<p><strong>Blur</strong><br>\n<code>image = tfa.image.mean_filter2d(image, filter_shape=5)</code></p>\n<p><strong>Random Rotate</strong></p>\n<pre><code>def random_rotate(image, angle=20):\n    \"\"\"\n    Randomly rotates an image within the bounds (-angle, angle)\n    \"\"\"\n    rotate_prob = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    max_angle   = angle*math.pi/180\n    rotation    = tf.random.uniform(shape=[], minval=-max_angle, maxval=max_angle, dtype=tf.float32)\n\n    image = tfa.image.rotate(image, rotation, interpolation = \"BILINEAR\")\n    image = tf.image.central_crop(image, central_fraction=0.90)\n    image = tf.image.resize(image, [config.IMAGE_SIZE[0],config.IMAGE_SIZE[1]])\n    return image\n</code></pre>\n<p><code>image = random_rotate(image, angle=10)</code></p>",
      "rawMarkdown": "I'm using rotate and a blur function from TFA for this comp.\n\n**Blur**\n`image = tfa.image.mean_filter2d(image, filter_shape=5)`\n\n**Random Rotate**\n\n```\ndef random_rotate(image, angle=20):\n    \"\"\"\n    Randomly rotates an image within the bounds (-angle, angle)\n    \"\"\"\n    rotate_prob = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    max_angle   = angle*math.pi/180\n    rotation    = tf.random.uniform(shape=[], minval=-max_angle, maxval=max_angle, dtype=tf.float32)\n    \n    image = tfa.image.rotate(image, rotation, interpolation = \"BILINEAR\")\n    image = tf.image.central_crop(image, central_fraction=0.90)\n    image = tf.image.resize(image, [config.IMAGE_SIZE[0],config.IMAGE_SIZE[1]])\n    return image\n\n```\n\n`image = random_rotate(image, angle=10)`",
      "votes": null
    },
    {
      "id": "1742537",
      "postDate": "04/02/2022 02:07:48",
      "content": "<p>Hi, does ratetion augumentation help ? I tried it before, but got lower cv and lb score.</p>",
      "rawMarkdown": "Hi, does ratetion augumentation help ? I tried it before, but got lower cv and lb score.",
      "votes": null
    },
    {
      "id": "1742540",
      "postDate": "04/02/2022 02:33:55",
      "content": "<p>I think so. It's only a +-10 degrees max @ only 20% chance of even happening. I had 10% and I wanted a bit stronger augs to help curb overfit. I did get gains from this and some other aug changes but of course I muddied the waters by also changing some model parameters and layers. All up, that batch of changes took my best LB single fold from 0.779 to 0.787.  Unfortunately at this stage of the comp I don't have luxury of time and resources to test one thing at a time. I know I can do it more quickly at lower resolutions etc but in the end you have to try it at the resolution and model size you intend to use and that's taking 17hrs per run…</p>\n<p>I'm now struggling to stay in silver zone …<br>\nIt's getting crowded here.<br>\nI think the only reason I'm still in the silver zone is that the people above me keep teaming up… taking them out of the position list. lol</p>",
      "rawMarkdown": "I think so. It's only a +-10 degrees max @ only 20% chance of even happening. I had 10% and I wanted a bit stronger augs to help curb overfit. I did get gains from this and some other aug changes but of course I muddied the waters by also changing some model parameters and layers. All up, that batch of changes took my best LB single fold from 0.779 to 0.787.  Unfortunately at this stage of the comp I don't have luxury of time and resources to test one thing at a time. I know I can do it more quickly at lower resolutions etc but in the end you have to try it at the resolution and model size you intend to use and that's taking 17hrs per run...\n\nI'm now struggling to stay in silver zone ...\nIt's getting crowded here.\nI think the only reason I'm still in the silver zone is that the people above me keep teaming up... taking them out of the position list. lol",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1742463,
      "author_name": "cdeotte",
      "author_url": "",
      "post_date": "04/01/2022 22:43:14",
      "content": "<p>I think TPU (with TensorFlow as opposed to PyTorch) requires TF data dataset but does not require TFRecords. (And as you point out, TF data dataset can be built directly from images and targets).</p>\n<p>In the beginning, TF data dataset could not use third party augmentations. I believe it still cannot. That is why many kagglers have posted code to perform different augmentations like rotation, cutmix, coarse dropout, mixup, sheer, scaling, cropping etc in various Kaggle forums and notebooks.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1742498,
          "author_name": "mutantspore",
          "author_url": "",
          "post_date": "04/01/2022 23:53:25",
          "content": "<p>A number of the augmentations you mention or a version of them,  are now in the <a href=\"https://www.tensorflow.org/addons/api_docs/python/tfa/image\" target=\"_blank\">TFA (TensorFlow Add-ons)\n</a></p>\n<p>Of course I greatly value those you have written Chris, and I still use them when I can.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1742502,
          "author_name": "cdeotte",
          "author_url": "",
          "post_date": "04/02/2022 00:06:30",
          "content": "<p>Thanks for the info. Can TF data dataset accept functions from TFA? In the beginning it could not. Has this changed?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1742509,
          "author_name": "mutantspore",
          "author_url": "",
          "post_date": "04/02/2022 00:15:36",
          "content": "<p>I'm using rotate and a blur function from TFA for this comp.</p>\n<p><strong>Blur</strong><br>\n<code>image = tfa.image.mean_filter2d(image, filter_shape=5)</code></p>\n<p><strong>Random Rotate</strong></p>\n<pre><code>def random_rotate(image, angle=20):\n    \"\"\"\n    Randomly rotates an image within the bounds (-angle, angle)\n    \"\"\"\n    rotate_prob = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    max_angle   = angle*math.pi/180\n    rotation    = tf.random.uniform(shape=[], minval=-max_angle, maxval=max_angle, dtype=tf.float32)\n\n    image = tfa.image.rotate(image, rotation, interpolation = \"BILINEAR\")\n    image = tf.image.central_crop(image, central_fraction=0.90)\n    image = tf.image.resize(image, [config.IMAGE_SIZE[0],config.IMAGE_SIZE[1]])\n    return image\n</code></pre>\n<p><code>image = random_rotate(image, angle=10)</code></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1742537,
          "author_name": "rainfalllove",
          "author_url": "",
          "post_date": "04/02/2022 02:07:48",
          "content": "<p>Hi, does ratetion augumentation help ? I tried it before, but got lower cv and lb score.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1742540,
          "author_name": "mutantspore",
          "author_url": "",
          "post_date": "04/02/2022 02:33:55",
          "content": "<p>I think so. It's only a +-10 degrees max @ only 20% chance of even happening. I had 10% and I wanted a bit stronger augs to help curb overfit. I did get gains from this and some other aug changes but of course I muddied the waters by also changing some model parameters and layers. All up, that batch of changes took my best LB single fold from 0.779 to 0.787.  Unfortunately at this stage of the comp I don't have luxury of time and resources to test one thing at a time. I know I can do it more quickly at lower resolutions etc but in the end you have to try it at the resolution and model size you intend to use and that's taking 17hrs per run…</p>\n<p>I'm now struggling to stay in silver zone …<br>\nIt's getting crowded here.<br>\nI think the only reason I'm still in the silver zone is that the people above me keep teaming up… taking them out of the position list. lol</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1742342": "I have tried ImageDataGenerator /tf.keras.utils.image_dataset_from_directory / from dataframe .\nAlso they are not supported tf.py_function therefore I can't map images with labels.\n\nOk, I found the solution using .from_tensor_slice I can load images and labels simultaneously, but question about work in map function is still and what about third party augmentations?",
    "1742463": "I think TPU (with TensorFlow as opposed to PyTorch) requires TF data dataset but does not require TFRecords. (And as you point out, TF data dataset can be built directly from images and targets).\n\nIn the beginning, TF data dataset could not use third party augmentations. I believe it still cannot. That is why many kagglers have posted code to perform different augmentations like rotation, cutmix, coarse dropout, mixup, sheer, scaling, cropping etc in various Kaggle forums and notebooks.",
    "1742498": "A number of the augmentations you mention or a version of them,  are now in the [TFA (TensorFlow Add-ons)\n](https://www.tensorflow.org/addons/api_docs/python/tfa/image)\n\nOf course I greatly value those you have written Chris, and I still use them when I can.",
    "1742502": "Thanks for the info. Can TF data dataset accept functions from TFA? In the beginning it could not. Has this changed?",
    "1742509": "I'm using rotate and a blur function from TFA for this comp.\n\n**Blur**\n`image = tfa.image.mean_filter2d(image, filter_shape=5)`\n\n**Random Rotate**\n\n```\ndef random_rotate(image, angle=20):\n    \"\"\"\n    Randomly rotates an image within the bounds (-angle, angle)\n    \"\"\"\n    rotate_prob = tf.random.uniform([], 0, 1.0, dtype=tf.float32)\n    max_angle   = angle*math.pi/180\n    rotation    = tf.random.uniform(shape=[], minval=-max_angle, maxval=max_angle, dtype=tf.float32)\n    \n    image = tfa.image.rotate(image, rotation, interpolation = \"BILINEAR\")\n    image = tf.image.central_crop(image, central_fraction=0.90)\n    image = tf.image.resize(image, [config.IMAGE_SIZE[0],config.IMAGE_SIZE[1]])\n    return image\n\n```\n\n`image = random_rotate(image, angle=10)`",
    "1742537": "Hi, does ratetion augumentation help ? I tried it before, but got lower cv and lb score.",
    "1742540": "I think so. It's only a +-10 degrees max @ only 20% chance of even happening. I had 10% and I wanted a bit stronger augs to help curb overfit. I did get gains from this and some other aug changes but of course I muddied the waters by also changing some model parameters and layers. All up, that batch of changes took my best LB single fold from 0.779 to 0.787.  Unfortunately at this stage of the comp I don't have luxury of time and resources to test one thing at a time. I know I can do it more quickly at lower resolutions etc but in the end you have to try it at the resolution and model size you intend to use and that's taking 17hrs per run...\n\nI'm now struggling to stay in silver zone ...\nIt's getting crowded here.\nI think the only reason I'm still in the silver zone is that the people above me keep teaming up... taking them out of the position list. lol"
  },
  "source": "meta"
}