Dear participants,
\nTLDR; Please redownload and retrain your layout models!
\nA few participants have raised a question if the data in the NLP layout collections is corrupted. We investigated this and found that the NLP collections are indeed corrupted. In particular, the configurations and the runtimes are not paired correctly due to a bug in our data pipeline. Some data in XLA layout collections is also affected.
\nWe have just uploaded all data for the layout collections. Please redownload the data and retrain your model on all layout collections. All previous submissions have been rescored. Note that submissions with the previous buggy data will likely have scores equivalent to a random model. The good news is that if your model works generally well for the XLA layout collections using the old data, and you retrain the same model on the updated data, your score should improve significantly due to this fix!
\nFor continuity, we have moved the old dataset here.
\nWe sincerely apologize for our mistake and the inconvenience caused to you all. Please bear with us since this is the first time we have run this very unique competition! Unfortunately, we won’t be able to extend the deadline because we would like to invite the winners to present at the ML for Systems workshop at NeurIPS.
\nFinally, thank you so much for your interest and participation in this competition.
\nBest regards,
\nCompetition Hosts
Dear participants,
\nTLDR; Please redownload and retrain your layout models!
\nA few participants have raised a question if the data in the NLP layout collections is corrupted. We investigated this and found that the NLP collections are indeed corrupted. In particular, the configurations and the runtimes are not paired correctly due to a bug in our data pipeline. Some data in XLA layout collections is also affected.
\nWe have just uploaded all data for the layout collections. Please redownload the data and retrain your model on all layout collections. All previous submissions have been rescored. Note that submissions with the previous buggy data will likely have scores equivalent to a random model. The good news is that if your model works generally well for the XLA layout collections using the old data, and you retrain the same model on the updated data, your score should improve significantly due to this fix!
\nFor continuity, we have moved the old dataset here.
\nWe sincerely apologize for our mistake and the inconvenience caused to you all. Please bear with us since this is the first time we have run this very unique competition! Unfortunately, we won’t be able to extend the deadline because we would like to invite the winners to present at the ML for Systems workshop at NeurIPS.
\nFinally, thank you so much for your interest and participation in this competition.
\nBest regards,
\nCompetition Hosts
Thank you! I was seriously questioning the nature of my reality……
", "rawMarkdown": "Thank you! I was seriously questioning the nature of my reality......", "votes": 6 }, { "id": 2476948, "postDate": "2023-10-11T02:47:42.280Z", "content": "So if I download the dataset from tensorflow.org right now, I will also get the updated data?
", "rawMarkdown": "So if I download the dataset from tensorflow.org right now, I will also get the updated data?", "votes": 1, "replies": [ { "id": 2477014, "postDate": "2023-10-11T04:03:34.067Z", "content": "I think the data is updated here in the competition aswell
", "rawMarkdown": "I think the data is updated here in the competition aswell", "replies": [ { "id": 2479996, "postDate": "2023-10-13T04:05:27.177Z", "content": "Yes, both sources of data were updated.
", "rawMarkdown": "Yes, both sources of data were updated." } ] } ] }, { "id": 2459954, "postDate": "2023-09-28T13:44:13.743Z", "content": "Is it possible to elaborate on which files were corrupted? It is important for us to make conclusions and to understand what models to revisit. Were the validation files corrupted also?
", "rawMarkdown": "Is it possible to elaborate on which files were corrupted? It is important for us to make conclusions and to understand what models to revisit. Were the validation files corrupted also?", "votes": 1, "replies": [ { "id": 2460762, "postDate": "2023-09-29T05:10:17.320Z", "content": "Most files in npz/layout/nlp, and some files in npz/layout/xla are corrupted. npz/tile/xla files are okay.
Thx a lot!
", "rawMarkdown": "Thx a lot!" }, { "id": 2512311, "postDate": "2023-11-04T11:03:12.500Z", "content": "Thx, this help me a lot
", "rawMarkdown": "Thx, this help me a lot " }, { "id": 2503565, "postDate": "2023-10-29T07:59:05.207Z", "content": "If I download now, the data should be ok right?
", "rawMarkdown": "If I download now, the data should be ok right?", "replies": [ { "id": 2504166, "postDate": "2023-10-29T17:34:50.410Z", "content": "That's right.
", "rawMarkdown": "That's right." } ] }, { "id": 2503331, "postDate": "2023-10-29T01:30:59.503Z", "content": "Thank you! I am sure this helped many including me!
", "rawMarkdown": "Thank you! I am sure this helped many including me!" }, { "id": 2482121, "postDate": "2023-10-14T17:21:04.207Z", "content": "Interesting work
", "rawMarkdown": "Interesting work" }, { "id": 2515750, "postDate": "2023-11-07T07:58:43.187Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2515711, "postDate": "2023-11-07T06:41:18.180Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2522743, "postDate": "2023-11-13T02:43:41.117Z", "content": "Thanks for the update!
", "rawMarkdown": "Thanks for the update!" }, { "id": 2515535, "postDate": "2023-11-07T02:20:37.827Z", "content": "Thank you for your share!
", "rawMarkdown": "Thank you for your share!" }, { "id": 2459906, "postDate": "2023-09-28T13:07:24.807Z", "content": "Thanks, that's helpful !
", "rawMarkdown": "Thanks, that's helpful !\n " } ], "comments": [ { "id": 2458981, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2023-09-28T00:19:27.700000", "content": "Thank you! I was seriously questioning the nature of my reality……
", "votes": 6, "replies": [] }, { "id": 2476948, "author_name": "Chitra077", "author_url": "", "post_date": "2023-10-11T02:47:42.280000", "content": "So if I download the dataset from tensorflow.org right now, I will also get the updated data?
", "votes": 1, "replies": [ { "id": 2477014, "author_name": "Arun", "author_url": "", "post_date": "2023-10-11T04:03:34.067000", "content": "I think the data is updated here in the competition aswell
", "votes": 0, "replies": [ { "id": 2479996, "author_name": "Mangpo Phothilimthana", "author_url": "", "post_date": "2023-10-13T04:05:27.177000", "content": "Yes, both sources of data were updated.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2459954, "author_name": "Roy Segalz", "author_url": "", "post_date": "2023-09-28T13:44:13.743000", "content": "Is it possible to elaborate on which files were corrupted? It is important for us to make conclusions and to understand what models to revisit. Were the validation files corrupted also?
", "votes": 1, "replies": [ { "id": 2460762, "author_name": "Mangpo Phothilimthana", "author_url": "", "post_date": "2023-09-29T05:10:17.320000", "content": "Most files in npz/layout/nlp, and some files in npz/layout/xla are corrupted. npz/tile/xla files are okay.
Thx a lot!
", "votes": 0, "replies": [] }, { "id": 2512311, "author_name": "Reussuuu", "author_url": "", "post_date": "2023-11-04T11:03:12.500000", "content": "Thx, this help me a lot
", "votes": 0, "replies": [] }, { "id": 2503565, "author_name": "NextChen", "author_url": "", "post_date": "2023-10-29T07:59:05.207000", "content": "If I download now, the data should be ok right?
", "votes": 0, "replies": [ { "id": 2504166, "author_name": "Mangpo Phothilimthana", "author_url": "", "post_date": "2023-10-29T17:34:50.410000", "content": "That's right.
", "votes": 0, "replies": [] } ] }, { "id": 2503331, "author_name": "Vyaas Valsaraj", "author_url": "", "post_date": "2023-10-29T01:30:59.503000", "content": "Thank you! I am sure this helped many including me!
", "votes": 0, "replies": [] }, { "id": 2482121, "author_name": "Aikesha Keutayeva", "author_url": "", "post_date": "2023-10-14T17:21:04.207000", "content": "Interesting work
", "votes": 0, "replies": [] }, { "id": 2515750, "author_name": "", "author_url": "", "post_date": "2023-11-07T07:58:43.187000", "content": "", "votes": 0, "replies": [] }, { "id": 2515711, "author_name": "", "author_url": "", "post_date": "2023-11-07T06:41:18.180000", "content": "", "votes": 0, "replies": [] }, { "id": 2522743, "author_name": "Snooy", "author_url": "", "post_date": "2023-11-13T02:43:41.117000", "content": "Thanks for the update!
", "votes": 0, "replies": [] }, { "id": 2515535, "author_name": "mayibo_hyh", "author_url": "", "post_date": "2023-11-07T02:20:37.827000", "content": "Thank you for your share!
", "votes": 0, "replies": [] }, { "id": 2459906, "author_name": "MiHu", "author_url": "", "post_date": "2023-09-28T13:07:24.807000", "content": "Thanks, that's helpful !
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2458958": "Dear participants,\n\nTLDR; Please redownload and retrain your layout models!\n\nA few participants have raised a [question](https://www.kaggle.com/competitions/predict-ai-model-runtime/discussion/438498#2454517) if the data in the NLP layout collections is corrupted. We investigated this and found that the NLP collections are indeed corrupted. In particular, the configurations and the runtimes are not paired correctly due to a bug in our data pipeline. Some data in XLA layout collections is also affected.\n\nWe have just uploaded **all data for the layout collections**. Please redownload the data and retrain your model on all layout collections. All previous submissions have been rescored. Note that submissions with the previous buggy data will likely have scores equivalent to a random model. The good news is that if your model works generally well for the XLA layout collections using the old data, and you retrain the same model on the updated data, your score should improve significantly due to this fix!\n\nFor continuity, we have moved the [old dataset here](https://www.kaggle.com/datasets/inversion/tpu-graphs-deprecated-data).\n\nWe sincerely apologize for our mistake and the inconvenience caused to you all. Please bear with us since this is the first time we have run this very unique competition! Unfortunately, we won’t be able to extend the deadline because we would like to invite the winners to present at the ML for Systems workshop at NeurIPS.\n\nFinally, thank you so much for your interest and participation in this competition. \n\nBest regards,\nCompetition Hosts\n", "2458981": "Thank you! I was seriously questioning the nature of my reality......", "2476948": "So if I download the dataset from tensorflow.org right now, I will also get the updated data?", "2459954": "Is it possible to elaborate on which files were corrupted? It is important for us to make conclusions and to understand what models to revisit. Were the validation files corrupted also?", "2514314": "Thx a lot!", "2512311": "Thx, this help me a lot ", "2503565": "If I download now, the data should be ok right?", "2503331": "Thank you! I am sure this helped many including me!", "2482121": "Interesting work", "2515750": "", "2515711": "", "2522743": "Thanks for the update!", "2515535": "Thank you for your share!", "2459906": "Thanks, that's helpful !\n " } }