{
  "id": 316481,
  "title": "Want to mix multiple tfrecords with folding",
  "url": "/competitions/happy-whale-and-dolphin/discussion/316481",
  "author_name": "",
  "post_date": "2022-04-02T08:24:17.486250500Z",
  "votes": 6,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I want to make mixed dataset. For example, mix original data and cropped data.<br>\nTherefore, I'd like to split tfrecords by fold like this .<br>\nBut I'm really beginner of TF and spend hard time.<br>\nDo you have any good sample or good functions to do like this?<br>\n If you have, please help me！</p>\n<p>What I want to do<br>\n↓↓</p>\n<pre><code>folds_dictionary = {\"image1\":0,\"image2\":2,\"image3\":1 ...}\n# mapping image(file name to folds)\n\ntfrec1 = tf.data.TFRecordDataset(\"dataset1.tfrec\")\n# tfrec1_image1,tfrec1_image2,tfrec1_image3, ...\n\ntfrec2 = tf.data.TFRecordDataset(\"dataset2.tfrec\")\n# tfrec2_image1,tfrec2_image2,tfrec2_image3, ...\n\ndef mix_func(tfrec1,tfrec2,folds_dictionary):\n    # some operations(I have no good idea...)\n    return mixed_datasets\n # [ fold0_tfrec (tfrec1_image1,tfrec2_image1) ,fold1_tfrec (tfrec1_image3,tfrec2_image3) ,fold2_tfrec (tfrec1_image2,tfrec2_image2)] .... \n</code></pre>",
  "messages": [
    {
      "id": "1742732",
      "postDate": "04/02/2022 08:24:17",
      "content": "<p>I want to make mixed dataset. For example, mix original data and cropped data.<br>\nTherefore, I'd like to split tfrecords by fold like this .<br>\nBut I'm really beginner of TF and spend hard time.<br>\nDo you have any good sample or good functions to do like this?<br>\n If you have, please help me！</p>\n<p>What I want to do<br>\n↓↓</p>\n<pre><code>folds_dictionary = {\"image1\":0,\"image2\":2,\"image3\":1 ...}\n# mapping image(file name to folds)\n\ntfrec1 = tf.data.TFRecordDataset(\"dataset1.tfrec\")\n# tfrec1_image1,tfrec1_image2,tfrec1_image3, ...\n\ntfrec2 = tf.data.TFRecordDataset(\"dataset2.tfrec\")\n# tfrec2_image1,tfrec2_image2,tfrec2_image3, ...\n\ndef mix_func(tfrec1,tfrec2,folds_dictionary):\n    # some operations(I have no good idea...)\n    return mixed_datasets\n # [ fold0_tfrec (tfrec1_image1,tfrec2_image1) ,fold1_tfrec (tfrec1_image3,tfrec2_image3) ,fold2_tfrec (tfrec1_image2,tfrec2_image2)] .... \n</code></pre>",
      "rawMarkdown": "I want to make mixed dataset. For example, mix original data and cropped data.\nTherefore, I'd like to split tfrecords by fold like this .\nBut I'm really beginner of TF and spend hard time.\nDo you have any good sample or good functions to do like this?\n If you have, please help me！\n\nWhat I want to do\n↓↓\n\n```python:mix.py\n\n\nfolds_dictionary = {\"image1\":0,\"image2\":2,\"image3\":1 ...}\n# mapping image(file name to folds)\n\ntfrec1 = tf.data.TFRecordDataset(\"dataset1.tfrec\")\n# tfrec1_image1,tfrec1_image2,tfrec1_image3, ...\n\ntfrec2 = tf.data.TFRecordDataset(\"dataset2.tfrec\")\n# tfrec2_image1,tfrec2_image2,tfrec2_image3, ...\n\ndef mix_func(tfrec1,tfrec2,folds_dictionary):\n    # some operations(I have no good idea...)\n    return mixed_datasets\n # [ fold0_tfrec (tfrec1_image1,tfrec2_image1) ,fold1_tfrec (tfrec1_image3,tfrec2_image3) ,fold2_tfrec (tfrec1_image2,tfrec2_image2)] .... \n\n\n```",
      "votes": null
    },
    {
      "id": "1743635",
      "postDate": "04/03/2022 06:52:24",
      "content": "<p>The simplest way to do that is to take images from the tf records, mixed them as you want to and create new tf records files. You can use a tutorial like <a href=\"https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880\" target=\"_blank\">https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880</a> for creating the tf records. </p>",
      "rawMarkdown": "The simplest way to do that is to take images from the tf records, mixed them as you want to and create new tf records files. You can use a tutorial like https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880 for creating the tf records.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1743635,
      "author_name": "vladvdv",
      "author_url": "",
      "post_date": "04/03/2022 06:52:24",
      "content": "<p>The simplest way to do that is to take images from the tf records, mixed them as you want to and create new tf records files. You can use a tutorial like <a href=\"https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880\" target=\"_blank\">https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880</a> for creating the tf records. </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1742732": "I want to make mixed dataset. For example, mix original data and cropped data.\nTherefore, I'd like to split tfrecords by fold like this .\nBut I'm really beginner of TF and spend hard time.\nDo you have any good sample or good functions to do like this?\n If you have, please help me！\n\nWhat I want to do\n↓↓\n\n```python:mix.py\n\n\nfolds_dictionary = {\"image1\":0,\"image2\":2,\"image3\":1 ...}\n# mapping image(file name to folds)\n\ntfrec1 = tf.data.TFRecordDataset(\"dataset1.tfrec\")\n# tfrec1_image1,tfrec1_image2,tfrec1_image3, ...\n\ntfrec2 = tf.data.TFRecordDataset(\"dataset2.tfrec\")\n# tfrec2_image1,tfrec2_image2,tfrec2_image3, ...\n\ndef mix_func(tfrec1,tfrec2,folds_dictionary):\n    # some operations(I have no good idea...)\n    return mixed_datasets\n # [ fold0_tfrec (tfrec1_image1,tfrec2_image1) ,fold1_tfrec (tfrec1_image3,tfrec2_image3) ,fold2_tfrec (tfrec1_image2,tfrec2_image2)] .... \n\n\n```",
    "1743635": "The simplest way to do that is to take images from the tf records, mixed them as you want to and create new tf records files. You can use a tutorial like https://www.kaggle.com/code/dschettler8845/create-tfrecords-512x880 for creating the tf records."
  },
  "source": "meta"
}