Hi there. Not sure if this will help anyone but I made 3 things and thought to share them.
\nA TFRecord Dataset containing 384x384 images of the entire train dataset (60000 held for Validation) and the entire test dataset. No rotation or preprocessing other than inverting from black to white, padding the images to be square, and resizing is done.
A Notebook showing how to easily create the above-mentioned dataset. It should be easy to modify and will allow you to use Kaggle resources to create whatever dataset you'd like.
A Notebook showing how to create a tf.data.Dataset from the tfrecords and display a few for a sanity check.
I am aware that there are some very basic checks (if h>w then rotate) that should be performed. This should be easy to incorporate into the notebooks listed above and I probably will update them with a FLAG to allow for such a modification in the near future. Other configuration changes such as inverting, or performing CV2 style repairs (as in this notebook by @markwijkhuizen ).
\nThanks for reading. All the best!
", "messages": [ { "id": "1265068", "postDate": "04/06/2021 15:45:58", "content": "Hi there. Not sure if this will help anyone but I made 3 things and thought to share them.
\nA TFRecord Dataset containing 384x384 images of the entire train dataset (60000 held for Validation) and the entire test dataset. No rotation or preprocessing other than inverting from black to white, padding the images to be square, and resizing is done.
A Notebook showing how to easily create the above-mentioned dataset. It should be easy to modify and will allow you to use Kaggle resources to create whatever dataset you'd like.
A Notebook showing how to create a tf.data.Dataset from the tfrecords and display a few for a sanity check.
I am aware that there are some very basic checks (if h>w then rotate) that should be performed. This should be easy to incorporate into the notebooks listed above and I probably will update them with a FLAG to allow for such a modification in the near future. Other configuration changes such as inverting, or performing CV2 style repairs (as in this notebook by @markwijkhuizen ).
\nThanks for reading. All the best!
", "rawMarkdown": "Hi there. Not sure if this will help anyone but I made 3 things and thought to share them.\n\n---\n\n1. A [**TFRecord Dataset**](https://www.kaggle.com/dschettler8845/bms-tfrecord-dataset-384x384) containing **`384x384`** images of the entire train dataset (**`60000`** held for Validation) and the entire test dataset. No rotation or preprocessing other than inverting from black to white, padding the images to be square, and resizing is done.\n\n2. A [**Notebook**](https://www.kaggle.com/dschettler8845/bms-simple-tfrecord-creation) showing how to easily create the above-mentioned dataset. It should be easy to modify and will allow you to use Kaggle resources to create whatever dataset you'd like.\n\n3. A [**Notebook**](https://www.kaggle.com/dschettler8845/open-darien-bms-tfrecords) showing how to create a **`tf.data.Dataset`** from the tfrecords and display a few for a sanity check.\n\n---\n\nI am aware that there are some very basic checks (if h>w then rotate) that should be performed. This should be easy to incorporate into the notebooks listed above and I probably will update them with a FLAG to allow for such a modification in the near future. Other configuration changes such as inverting, or performing CV2 style repairs (as in [**this notebook**](https://www.kaggle.com/markwijkhuizen/advanced-image-cleaning-and-tfrecord-generation) by @markwijkhuizen ).\n\n---\n\nThanks for reading. All the best!", "votes": null }, { "id": "1268480", "postDate": "04/09/2021 12:44:32", "content": "Thanks a lot for sharing. Can you tell how to inference with tensorflow baseline if used?
", "rawMarkdown": "Thanks a lot for sharing. Can you tell how to inference with tensorflow baseline if used?", "votes": null }, { "id": "1268485", "postDate": "04/09/2021 12:50:26", "content": "Hi there. Youโre quite welcome!
\nIโm currently working on a really beginner friendly notebook to guide people through TPU training and inference using this dataset. I hope to post it within a week or so.
", "rawMarkdown": "Hi there. Youโre quite welcome!\n\nIโm currently working on a really beginner friendly notebook to guide people through TPU training and inference using this dataset. I hope to post it within a week or so.", "votes": null }, { "id": "1268708", "postDate": "04/09/2021 17:06:22", "content": "Waiting for your TPU kernels ๐! It would be super awesome if your kernel released with Y. Nakama tokenizer then it would be super awesome for me ๐
", "rawMarkdown": "Waiting for your TPU kernels ๐! It would be super awesome if your kernel released with Y. Nakama tokenizer then it would be super awesome for me ๐", "votes": null } ], "comments": [ { "id": 1268480, "author_name": "aifahim", "author_url": "", "post_date": "04/09/2021 12:44:32", "content": "Thanks a lot for sharing. Can you tell how to inference with tensorflow baseline if used?
", "votes": null, "replies": [ { "id": 1268485, "author_name": "dschettler8845", "author_url": "", "post_date": "04/09/2021 12:50:26", "content": "Hi there. Youโre quite welcome!
\nIโm currently working on a really beginner friendly notebook to guide people through TPU training and inference using this dataset. I hope to post it within a week or so.
", "votes": null, "replies": [] }, { "id": 1268708, "author_name": "aifahim", "author_url": "", "post_date": "04/09/2021 17:06:22", "content": "Waiting for your TPU kernels ๐! It would be super awesome if your kernel released with Y. Nakama tokenizer then it would be super awesome for me ๐
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1265068": "Hi there. Not sure if this will help anyone but I made 3 things and thought to share them.\n\n---\n\n1. A [**TFRecord Dataset**](https://www.kaggle.com/dschettler8845/bms-tfrecord-dataset-384x384) containing **`384x384`** images of the entire train dataset (**`60000`** held for Validation) and the entire test dataset. No rotation or preprocessing other than inverting from black to white, padding the images to be square, and resizing is done.\n\n2. A [**Notebook**](https://www.kaggle.com/dschettler8845/bms-simple-tfrecord-creation) showing how to easily create the above-mentioned dataset. It should be easy to modify and will allow you to use Kaggle resources to create whatever dataset you'd like.\n\n3. A [**Notebook**](https://www.kaggle.com/dschettler8845/open-darien-bms-tfrecords) showing how to create a **`tf.data.Dataset`** from the tfrecords and display a few for a sanity check.\n\n---\n\nI am aware that there are some very basic checks (if h>w then rotate) that should be performed. This should be easy to incorporate into the notebooks listed above and I probably will update them with a FLAG to allow for such a modification in the near future. Other configuration changes such as inverting, or performing CV2 style repairs (as in [**this notebook**](https://www.kaggle.com/markwijkhuizen/advanced-image-cleaning-and-tfrecord-generation) by @markwijkhuizen ).\n\n---\n\nThanks for reading. All the best!", "1268480": "Thanks a lot for sharing. Can you tell how to inference with tensorflow baseline if used?", "1268485": "Hi there. Youโre quite welcome!\n\nIโm currently working on a really beginner friendly notebook to guide people through TPU training and inference using this dataset. I hope to post it within a week or so.", "1268708": "Waiting for your TPU kernels ๐! It would be super awesome if your kernel released with Y. Nakama tokenizer then it would be super awesome for me ๐" }, "source": "meta" }