I usually participate in smaller competitions where I submit a submission.csv file, and those datasets are easy to work with. However, for this competition, I wanted to test my beginner skills, but I couldn't quite grasp the submission format.
\nHere's what I thought: for each ID in the test.csv file, I assumed that there would be a corresponding hidden parquet file located at …/series_test.parquet/id=00115b9f/part-0.parquet, where I would extract time-dependent features. I wrote my code based on this assumption. However, when I ran my code, I realized that there are no hidden parquet files in the test.csv dataset as I had imagined. Doesn't the competition provide a parquet file for each ID in the test.csv file?
\nCould you please review my code and explain why it failed? I believe that some of you, at a beginner level like me, might have experienced a similar issue and will understand my confusion. I would appreciate any help or understanding you can provide. I hope my confusion will be clear once you review my code. Thank you in advance!
\nhttps://www.kaggle.com/code/iamnotindangerskyler/childmindtr
I usually participate in smaller competitions where I submit a submission.csv file, and those datasets are easy to work with. However, for this competition, I wanted to test my beginner skills, but I couldn't quite grasp the submission format.
\nHere's what I thought: for each ID in the test.csv file, I assumed that there would be a corresponding hidden parquet file located at …/series_test.parquet/id=00115b9f/part-0.parquet, where I would extract time-dependent features. I wrote my code based on this assumption. However, when I ran my code, I realized that there are no hidden parquet files in the test.csv dataset as I had imagined. Doesn't the competition provide a parquet file for each ID in the test.csv file?
\nCould you please review my code and explain why it failed? I believe that some of you, at a beginner level like me, might have experienced a similar issue and will understand my confusion. I would appreciate any help or understanding you can provide. I hope my confusion will be clear once you review my code. Thank you in advance!
\nhttps://www.kaggle.com/code/iamnotindangerskyler/childmindtr
You may consider this as a proxy for a real life situation where you don't know the actual test set and have to build a model for unseen data. The 3 rows provided to you are for syntax check only @iamnotindangerskyler
", "rawMarkdown": "You may consider this as a proxy for a real life situation where you don't know the actual test set and have to build a model for **unseen data**. The 3 rows provided to you are for syntax check only @iamnotindangerskyler", "votes": null }, { "id": "3004409", "postDate": "10/01/2024 19:34:47", "content": "We will create and submit a model that makes a prediction for the 'sii' category for each 'id' in the test.csv file. However, in this competition format, there are more test 'ids' checked in the background. I understood everything up to this point, but I thought there would be hidden parquet files for some of the test 'ids.' Isn't that a wrong assumption? There are only 2 test parquet files, no extras, right? This means that the training dataset contains less information than the test dataset, specifically regarding the features that can be extracted from the parquet files. This actually requires us to make double predictions. Did I understand correctly? Are there only 2 parquet files for the test 'ids'?
", "rawMarkdown": "We will create and submit a model that makes a prediction for the 'sii' category for each 'id' in the test.csv file. However, in this competition format, there are more test 'ids' checked in the background. I understood everything up to this point, but I thought there would be hidden parquet files for some of the test 'ids.' Isn't that a wrong assumption? There are only 2 test parquet files, no extras, right? This means that the training dataset contains less information than the test dataset, specifically regarding the features that can be extracted from the parquet files. This actually requires us to make double predictions. Did I understand correctly? Are there only 2 parquet files for the test 'ids'?", "votes": null }, { "id": "3004425", "postDate": "10/01/2024 20:09:46", "content": "The test.csv shown to you is not the test set they feed into your code.
\nIf you look carefully, you will find that 11 out of 20 id in the test set actually come from train set.
In the \"real\" test set that is hidden, I guess some of them have parquet and some don't.
\nSo in your code you can't call a specific id to get a parquet.
Below is an excellent and understandable notebook by @abdmental01 that I will recommend you to have a read. His way in extracting the parquet is elegant.
\nhttps://www.kaggle.com/code/abdmental01/cmi-best-single-model
Good luck.
", "rawMarkdown": "The test.csv shown to you is not the test set they feed into your code.\nIf you look carefully, you will find that 11 out of 20 id in the test set actually come from train set.\n\nIn the \"real\" test set that is hidden, I guess some of them have parquet and some don't.\nSo in your code you can't call a specific id to get a parquet.\n\nBelow is an excellent and understandable notebook by @abdmental01 that I will recommend you to have a read. His way in extracting the parquet is elegant.\n[https://www.kaggle.com/code/abdmental01/cmi-best-single-model](https://www.kaggle.com/code/abdmental01/cmi-best-single-model)\n\nGood luck.", "votes": null } ], "comments": [ { "id": 3004392, "author_name": "ravi20076", "author_url": "", "post_date": "10/01/2024 19:09:44", "content": "You may consider this as a proxy for a real life situation where you don't know the actual test set and have to build a model for unseen data. The 3 rows provided to you are for syntax check only @iamnotindangerskyler
", "votes": null, "replies": [ { "id": 3004409, "author_name": "iamnotindangerskyler", "author_url": "", "post_date": "10/01/2024 19:34:47", "content": "We will create and submit a model that makes a prediction for the 'sii' category for each 'id' in the test.csv file. However, in this competition format, there are more test 'ids' checked in the background. I understood everything up to this point, but I thought there would be hidden parquet files for some of the test 'ids.' Isn't that a wrong assumption? There are only 2 test parquet files, no extras, right? This means that the training dataset contains less information than the test dataset, specifically regarding the features that can be extracted from the parquet files. This actually requires us to make double predictions. Did I understand correctly? Are there only 2 parquet files for the test 'ids'?
", "votes": null, "replies": [] } ] }, { "id": 3004425, "author_name": "tomyuen", "author_url": "", "post_date": "10/01/2024 20:09:46", "content": "The test.csv shown to you is not the test set they feed into your code.
\nIf you look carefully, you will find that 11 out of 20 id in the test set actually come from train set.
In the \"real\" test set that is hidden, I guess some of them have parquet and some don't.
\nSo in your code you can't call a specific id to get a parquet.
Below is an excellent and understandable notebook by @abdmental01 that I will recommend you to have a read. His way in extracting the parquet is elegant.
\nhttps://www.kaggle.com/code/abdmental01/cmi-best-single-model
Good luck.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3004111": "I usually participate in smaller competitions where I submit a submission.csv file, and those datasets are easy to work with. However, for this competition, I wanted to test my beginner skills, but I couldn't quite grasp the submission format.\n\nHere's what I thought: for each ID in the test.csv file, I assumed that there would be a corresponding hidden parquet file located at .../series_test.parquet/id=00115b9f/part-0.parquet, where I would extract time-dependent features. I wrote my code based on this assumption. However, when I ran my code, I realized that there are no hidden parquet files in the test.csv dataset as I had imagined. Doesn't the competition provide a parquet file for each ID in the test.csv file?\n\nCould you please review my code and explain why it failed? I believe that some of you, at a beginner level like me, might have experienced a similar issue and will understand my confusion. I would appreciate any help or understanding you can provide. I hope my confusion will be clear once you review my code. Thank you in advance!\n\n[https://www.kaggle.com/code/iamnotindangerskyler/childmindtr](url)", "3004392": "You may consider this as a proxy for a real life situation where you don't know the actual test set and have to build a model for **unseen data**. The 3 rows provided to you are for syntax check only @iamnotindangerskyler", "3004409": "We will create and submit a model that makes a prediction for the 'sii' category for each 'id' in the test.csv file. However, in this competition format, there are more test 'ids' checked in the background. I understood everything up to this point, but I thought there would be hidden parquet files for some of the test 'ids.' Isn't that a wrong assumption? There are only 2 test parquet files, no extras, right? This means that the training dataset contains less information than the test dataset, specifically regarding the features that can be extracted from the parquet files. This actually requires us to make double predictions. Did I understand correctly? Are there only 2 parquet files for the test 'ids'?", "3004425": "The test.csv shown to you is not the test set they feed into your code.\nIf you look carefully, you will find that 11 out of 20 id in the test set actually come from train set.\n\nIn the \"real\" test set that is hidden, I guess some of them have parquet and some don't.\nSo in your code you can't call a specific id to get a parquet.\n\nBelow is an excellent and understandable notebook by @abdmental01 that I will recommend you to have a read. His way in extracting the parquet is elegant.\n[https://www.kaggle.com/code/abdmental01/cmi-best-single-model](https://www.kaggle.com/code/abdmental01/cmi-best-single-model)\n\nGood luck." }, "source": "meta" }