I'm kind of curious to see if anyone has done much with the TensorFlow example that was provided and the TFRecord set that they also provide?
\n\nI haven't worked directly with TensorFlow that much and have been using this competition to get more familiar with it. Most if it has been pretty straightforward and the TPUs available via Colab are pretty fun to work with, but I'm having trouble with the test predictions.
\n\nI'd get like 20 or 30% accuracy on my validation set, then 0% on my test submission. I then went back and predicted my full training set and only got like 33 right, which feels impossible given the validation set accuracy. I think I've debugged it down to the dataset output not iterating in a stable, predictable order.
\n\nI've tried setting every parameter in the input_fn in input.py that should make the ordering predictable, but it hasn't seemed to help. My fork of the code is here: https://github.com/suresk/rxrx1-utils/tree/master/rxrx
\n\nI guess I'm curious to see if there is something simple I'm missing or if someone else has an approach that works?
", "messages": [ { "id": "598902", "postDate": "08/14/2019 07:55:22", "content": "I'm kind of curious to see if anyone has done much with the TensorFlow example that was provided and the TFRecord set that they also provide?
\n\nI haven't worked directly with TensorFlow that much and have been using this competition to get more familiar with it. Most if it has been pretty straightforward and the TPUs available via Colab are pretty fun to work with, but I'm having trouble with the test predictions.
\n\nI'd get like 20 or 30% accuracy on my validation set, then 0% on my test submission. I then went back and predicted my full training set and only got like 33 right, which feels impossible given the validation set accuracy. I think I've debugged it down to the dataset output not iterating in a stable, predictable order.
\n\nI've tried setting every parameter in the input_fn in input.py that should make the ordering predictable, but it hasn't seemed to help. My fork of the code is here: https://github.com/suresk/rxrx1-utils/tree/master/rxrx
\n\nI guess I'm curious to see if there is something simple I'm missing or if someone else has an approach that works?
", "rawMarkdown": "I'm kind of curious to see if anyone has done much with the TensorFlow example that was provided and the TFRecord set that they also provide?\n\nI haven't worked directly with TensorFlow that much and have been using this competition to get more familiar with it. Most if it has been pretty straightforward and the TPUs available via Colab are pretty fun to work with, but I'm having trouble with the test predictions.\n\nI'd get like 20 or 30% accuracy on my validation set, then 0% on my test submission. I then went back and predicted my full training set and only got like 33 right, which feels impossible given the validation set accuracy. I think I've debugged it down to the dataset output not iterating in a stable, predictable order.\n\nI've tried setting every parameter in the input_fn in input.py that should make the ordering predictable, but it hasn't seemed to help. My fork of the code is here: [https://github.com/suresk/rxrx1-utils/tree/master/rxrx](https://github.com/suresk/rxrx1-utils/tree/master/rxrx)\n\nI guess I'm curious to see if there is something simple I'm missing or if someone else has an approach that works?", "votes": null }, { "id": "599014", "postDate": "08/14/2019 11:40:26", "content": "How/Where do you generate your submission file?\nTFRecords don't guarantee the order, as you've seen, but you can get/generate the id_code's from the tf features like in input.parse(), or use the images directly to predict (instead of tfrecords)
", "rawMarkdown": "How/Where do you generate your submission file?\nTFRecords don't guarantee the order, as you've seen, but you can get/generate the id_code's from the tf features like in input.parse(), or use the images directly to predict (instead of tfrecords)", "votes": null } ], "comments": [ { "id": 599014, "author_name": "hmendonca", "author_url": "", "post_date": "08/14/2019 11:40:26", "content": "How/Where do you generate your submission file?\nTFRecords don't guarantee the order, as you've seen, but you can get/generate the id_code's from the tf features like in input.parse(), or use the images directly to predict (instead of tfrecords)
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "598902": "I'm kind of curious to see if anyone has done much with the TensorFlow example that was provided and the TFRecord set that they also provide?\n\nI haven't worked directly with TensorFlow that much and have been using this competition to get more familiar with it. Most if it has been pretty straightforward and the TPUs available via Colab are pretty fun to work with, but I'm having trouble with the test predictions.\n\nI'd get like 20 or 30% accuracy on my validation set, then 0% on my test submission. I then went back and predicted my full training set and only got like 33 right, which feels impossible given the validation set accuracy. I think I've debugged it down to the dataset output not iterating in a stable, predictable order.\n\nI've tried setting every parameter in the input_fn in input.py that should make the ordering predictable, but it hasn't seemed to help. My fork of the code is here: [https://github.com/suresk/rxrx1-utils/tree/master/rxrx](https://github.com/suresk/rxrx1-utils/tree/master/rxrx)\n\nI guess I'm curious to see if there is something simple I'm missing or if someone else has an approach that works?", "599014": "How/Where do you generate your submission file?\nTFRecords don't guarantee the order, as you've seen, but you can get/generate the id_code's from the tf features like in input.parse(), or use the images directly to predict (instead of tfrecords)" }, "source": "meta" }