{ "id": 243428, "title": "is there any 273x256x6 tf.io.TFRecordWriter dataset?", "url": "/competitions/seti-breakthrough-listen/discussion/243428", "author_name": "assign", "post_date": "2021-06-02T14:04:22.026000", "votes": 1, "comment_count": 4, "views": 0, "content": "

I try to make dataset, but kaggle disk space is much smaller than 120GB ( 273x256x6 * 32bit)

\n

is there tf record dataset with shape 273x256x6 ??

\n

or any full feature dataset?

", "messages": [ { "id": 1334069, "postDate": "2021-06-03T09:04:21.950Z", "content": "

Yes, there is one create by @xhlulu
\nTraining:
\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-train

\n

Notebook:
\nhttps://www.kaggle.com/xhlulu/seti-create-training-tf-records

\n

How to use:
\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras (V9)

\n

https://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras?scriptVersionId=63177972

\n

Test-tfrecords:

\n

https://www.kaggle.com/xhlulu/seti-tfrecords-test

\n

Note: In seti-tfrecords-test tfrecords, there is no ids. I would recommend to use test *.npy provided by kaggle for inference and submission.

\n

You can also use these tfrecords on colab by giving gcs bucket path as tfrecord file path.

", "rawMarkdown": "Yes, there is one create by @xhlulu \nTraining:\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-train\n\nNotebook:\nhttps://www.kaggle.com/xhlulu/seti-create-training-tf-records\n\nHow to use:\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras (V9)\n\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras?scriptVersionId=63177972\n\nTest-tfrecords: \n\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-test\n\nNote: In seti-tfrecords-test tfrecords, there is no ids. I would recommend to use test *.npy provided by kaggle for inference and submission. \n\nYou can also use these tfrecords on colab by giving gcs bucket path as tfrecord file path.", "votes": 1 }, { "id": 1333529, "postDate": "2021-06-02T20:22:23.020Z", "content": "

Hi, what about 255x255x6? I have created one here

", "rawMarkdown": "Hi, what about 255x255x6? I have created one [here](https://www.kaggle.com/soheild91/seti-tfrec-6ch) ", "votes": 1, "replies": [ { "id": 1333642, "postDate": "2021-06-02T23:49:23.097Z", "content": "

hmmm… thank you !
\nDoes it encoded??
\nI can't understand how dataset files were just 3GB

", "rawMarkdown": "hmmm... thank you !\nDoes it encoded??\nI can't understand how dataset files were just 3GB" }, { "id": 1333887, "postDate": "2021-06-03T05:58:43.447Z", "content": "

yes, I cast the arrays to int32 to reduce the size. I had a problem with the 20GB limited size of the notebook. right now I am trying to create a DS with float32. will share it with you when it is done.

", "rawMarkdown": "yes, I cast the arrays to int32 to reduce the size. I had a problem with the 20GB limited size of the notebook. right now I am trying to create a DS with float32. will share it with you when it is done.", "votes": 1 } ] }, { "id": 1333143, "postDate": "2021-06-02T14:04:22.027Z", "content": "

I try to make dataset, but kaggle disk space is much smaller than 120GB ( 273x256x6 * 32bit)

\n

is there tf record dataset with shape 273x256x6 ??

\n

or any full feature dataset?

", "rawMarkdown": "I try to make dataset, but kaggle disk space is much smaller than 120GB ( 273x256x6 * 32bit)\n\nis there tf record dataset with shape 273x256x6 ??\n\nor any full feature dataset?", "votes": 1 } ], "comments": [ { "id": 1334069, "author_name": "Krishna kant singh", "author_url": "", "post_date": "2021-06-03T09:04:21.950000", "content": "

Yes, there is one create by @xhlulu
\nTraining:
\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-train

\n

Notebook:
\nhttps://www.kaggle.com/xhlulu/seti-create-training-tf-records

\n

How to use:
\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras (V9)

\n

https://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras?scriptVersionId=63177972

\n

Test-tfrecords:

\n

https://www.kaggle.com/xhlulu/seti-tfrecords-test

\n

Note: In seti-tfrecords-test tfrecords, there is no ids. I would recommend to use test *.npy provided by kaggle for inference and submission.

\n

You can also use these tfrecords on colab by giving gcs bucket path as tfrecord file path.

", "votes": 1, "replies": [] }, { "id": 1333529, "author_name": "Soheil★Star", "author_url": "", "post_date": "2021-06-02T20:22:23.020000", "content": "

Hi, what about 255x255x6? I have created one here

", "votes": 1, "replies": [ { "id": 1333642, "author_name": "assign", "author_url": "", "post_date": "2021-06-02T23:49:23.097000", "content": "

hmmm… thank you !
\nDoes it encoded??
\nI can't understand how dataset files were just 3GB

", "votes": 0, "replies": [] }, { "id": 1333887, "author_name": "Soheil★Star", "author_url": "", "post_date": "2021-06-03T05:58:43.447000", "content": "

yes, I cast the arrays to int32 to reduce the size. I had a problem with the 20GB limited size of the notebook. right now I am trying to create a DS with float32. will share it with you when it is done.

", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1334069": "Yes, there is one create by @xhlulu \nTraining:\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-train\n\nNotebook:\nhttps://www.kaggle.com/xhlulu/seti-create-training-tf-records\n\nHow to use:\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras (V9)\n\nhttps://www.kaggle.com/xhlulu/seti-bidirectional-gru-in-keras?scriptVersionId=63177972\n\nTest-tfrecords: \n\nhttps://www.kaggle.com/xhlulu/seti-tfrecords-test\n\nNote: In seti-tfrecords-test tfrecords, there is no ids. I would recommend to use test *.npy provided by kaggle for inference and submission. \n\nYou can also use these tfrecords on colab by giving gcs bucket path as tfrecord file path.", "1333529": "Hi, what about 255x255x6? I have created one [here](https://www.kaggle.com/soheild91/seti-tfrec-6ch) ", "1333143": "I try to make dataset, but kaggle disk space is much smaller than 120GB ( 273x256x6 * 32bit)\n\nis there tf record dataset with shape 273x256x6 ??\n\nor any full feature dataset?" } }