{
  "id": 177017,
  "title": "How to store label in tfrecord.",
  "url": "/competitions/landmark-recognition-2020/discussion/177017",
  "author_name": "",
  "post_date": "2020-08-24T13:49:55.664277200Z",
  "votes": 1,
  "comment_count": 2,
  "views": 0,
  "content": "<p>there are 81313 classes. I am trying to store label like this in tfrecord .<br>\nlabel = tf.keras.utils.to_categorical(dataset.label, num_classes=81313)</p>\n<p>with tf.io.TFRecordWriter(record_file) as writer:<br>\n  for i in range(len(dataset)):<br>\n    path = dataset.path[dataset.index[i]]<br>\n    label = dataset.label[dataset.index[i]]<br>\n    image_string = tf.io.read_file(path)<br>\n    tf_example = image_example(image_string, label)<br>\n    writer.write(tf_example.SerializeToString())</p>\n<p>is this right format?</p>",
  "messages": [
    {
      "id": "983672",
      "postDate": "08/24/2020 13:49:55",
      "content": "<p>there are 81313 classes. I am trying to store label like this in tfrecord .<br>\nlabel = tf.keras.utils.to_categorical(dataset.label, num_classes=81313)</p>\n<p>with tf.io.TFRecordWriter(record_file) as writer:<br>\n  for i in range(len(dataset)):<br>\n    path = dataset.path[dataset.index[i]]<br>\n    label = dataset.label[dataset.index[i]]<br>\n    image_string = tf.io.read_file(path)<br>\n    tf_example = image_example(image_string, label)<br>\n    writer.write(tf_example.SerializeToString())</p>\n<p>is this right format?</p>",
      "rawMarkdown": "there are 81313 classes. I am trying to store label like this in tfrecord .\nlabel = tf.keras.utils.to_categorical(dataset.label, num_classes=81313)\n\nwith tf.io.TFRecordWriter(record_file) as writer:\n  for i in range(len(dataset)):\n    path = dataset.path[dataset.index[i]]\n    label = dataset.label[dataset.index[i]]\n    image_string = tf.io.read_file(path)\n    tf_example = image_example(image_string, label)\n    writer.write(tf_example.SerializeToString())\n\nis this right format?",
      "votes": null
    },
    {
      "id": "988428",
      "postDate": "08/28/2020 03:38:52",
      "content": "<p>Sup, you can just store them as a normal vector and then in you data pipeline use tf.one_hot() for example. This will save storage space. Also I would suggest that you encode the target variable from 0 to 81312.</p>\n<p>Cheers</p>",
      "rawMarkdown": "Sup, you can just store them as a normal vector and then in you data pipeline use tf.one_hot() for example. This will save storage space. Also I would suggest that you encode the target variable from 0 to 81312.\n\nCheers",
      "votes": null
    },
    {
      "id": "988610",
      "postDate": "08/28/2020 06:43:41",
      "content": "<p>Thanks !!!</p>",
      "rawMarkdown": "Thanks !!!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 988428,
      "author_name": "ragnar123",
      "author_url": "",
      "post_date": "08/28/2020 03:38:52",
      "content": "<p>Sup, you can just store them as a normal vector and then in you data pipeline use tf.one_hot() for example. This will save storage space. Also I would suggest that you encode the target variable from 0 to 81312.</p>\n<p>Cheers</p>",
      "votes": null,
      "replies": [
        {
          "id": 988610,
          "author_name": "dky7376",
          "author_url": "",
          "post_date": "08/28/2020 06:43:41",
          "content": "<p>Thanks !!!</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "983672": "there are 81313 classes. I am trying to store label like this in tfrecord .\nlabel = tf.keras.utils.to_categorical(dataset.label, num_classes=81313)\n\nwith tf.io.TFRecordWriter(record_file) as writer:\n  for i in range(len(dataset)):\n    path = dataset.path[dataset.index[i]]\n    label = dataset.label[dataset.index[i]]\n    image_string = tf.io.read_file(path)\n    tf_example = image_example(image_string, label)\n    writer.write(tf_example.SerializeToString())\n\nis this right format?",
    "988428": "Sup, you can just store them as a normal vector and then in you data pipeline use tf.one_hot() for example. This will save storage space. Also I would suggest that you encode the target variable from 0 to 81312.\n\nCheers",
    "988610": "Thanks !!!"
  },
  "source": "meta"
}