![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2F11aa73af80097227c54419e75deaf2b5%2F2024-10-19%2014.59.43.png?generation=1729317606175658&alt=media\")
From January 14th to July 12th there are approximately 124 days of market (more if symbol ids are cryptos).
\nFrom data description: 'during the final weeks of the model training phase we will be extending the public test set to include data closer to the submission deadline. Predictions on this extended set will not be scored'.
\nSo we don't know how many date_id will be in test set.
\nThis info is very important to set the validation framework properly.
\nCould you please give me an approximated number of date_id's?
From January 14th to July 12th there are approximately 124 days of market (more if symbol ids are cryptos).
\nFrom data description: 'during the final weeks of the model training phase we will be extending the public test set to include data closer to the submission deadline. Predictions on this extended set will not be scored'.
\nSo we don't know how many date_id will be in test set.
\nThis info is very important to set the validation framework properly.
\nCould you please give me an approximated number of date_id's?
@blindape just my estimation. This will be about 119~120 days for public data and about 119~120 days for private data.
\nFrom the train data, we can see the how much the row data per date_id.
\nThis is the dataframe screen shot and plot.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2F37719b84d19f14be131e294f0f9e52c7%2F2024-10-19%2014.58.47.png?generation=1729317618754294&alt=media\")
Looking at the plot, it appears to be converging upward to the right. For now, let’s try the following calculation using the very last date_id. (4.5 million row data is written in the data explanation)
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2F974c9e6dd579acf74769b3b81b3845f5%2F2024-10-19%2015.02.37.png?generation=1729317770920855&alt=media\")
Assuming there are 20 business days in a month, the number of months can be calculated using the following division.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2F38aac738bc9f616c622daef6d84f8ecd%2F2024-10-19%2015.03.45.png?generation=1729317836544287&alt=media\")
This indicates exactly 6 months. Therefore, it’s likely that there are 6 months of public data and 6 months of private data, making the test data span exactly one year. However, this is just my speculation!
", "rawMarkdown": "blindape just my estimation. This will be about 119~120 days for public data and about 119~120 days for private data.\n\nFrom the train data, we can see the how much the row data per date_id.\nThis is the dataframe screen shot and plot.\n\n\n\n\n\nLooking at the plot, it appears to be converging upward to the right. For now, let’s try the following calculation using the very last date_id. (4.5 million row data is written in the data explanation)\n\n\n\nAssuming there are 20 business days in a month, the number of months can be calculated using the following division.\n\n\n\nThis indicates exactly 6 months. Therefore, it’s likely that there are 6 months of public data and 6 months of private data, making the test data span exactly one year. However, this is just my speculation!", "votes": null }, { "id": "3022005", "postDate": "10/19/2024 06:32:42", "content": "I reached similar conclusion. Using 4.5M rows as fold length instead of computing using dates.
\nIt gives me these folds:
\n1577 - 1698
\n1455 - 1576
\n1331 - 1454
\n1210 - 1330
\n1090 - 1209
@blindape Yes, that's right! Now, I'm also using the range 1577 - 1698!
", "rawMarkdown": "blindape Yes, that's right! Now, I'm also using the range 1577 - 1698!", "votes": null }, { "id": "3051666", "postDate": "11/21/2024 14:12:35", "content": "Sorry to dig up such an old post, but I couldn't seem to find the answer on this subject.
\nHow much more data are we going to get for the forecasting phase? How much inference time should I be aiming for on the current submission, so I don't just get a timeout on final submission?
", "rawMarkdown": "Sorry to dig up such an old post, but I couldn't seem to find the answer on this subject. \n\nHow much more data are we going to get for the forecasting phase? How much inference time should I be aiming for on the current submission, so I don't just get a timeout on final submission?", "votes": null } ], "comments": [ { "id": 3021981, "author_name": "chumajin", "author_url": "", "post_date": "10/19/2024 06:05:08", "content": "@blindape just my estimation. This will be about 119~120 days for public data and about 119~120 days for private data.
\nFrom the train data, we can see the how much the row data per date_id.
\nThis is the dataframe screen shot and plot.
Looking at the plot, it appears to be converging upward to the right. For now, let’s try the following calculation using the very last date_id. (4.5 million row data is written in the data explanation)
\n
Assuming there are 20 business days in a month, the number of months can be calculated using the following division.
\n
This indicates exactly 6 months. Therefore, it’s likely that there are 6 months of public data and 6 months of private data, making the test data span exactly one year. However, this is just my speculation!
", "votes": null, "replies": [ { "id": 3022005, "author_name": "blindape", "author_url": "", "post_date": "10/19/2024 06:32:42", "content": "I reached similar conclusion. Using 4.5M rows as fold length instead of computing using dates.
\nIt gives me these folds:
\n1577 - 1698
\n1455 - 1576
\n1331 - 1454
\n1210 - 1330
\n1090 - 1209
@blindape Yes, that's right! Now, I'm also using the range 1577 - 1698!
", "votes": null, "replies": [] } ] } ] }, { "id": 3051666, "author_name": "woprime", "author_url": "", "post_date": "11/21/2024 14:12:35", "content": "Sorry to dig up such an old post, but I couldn't seem to find the answer on this subject.
\nHow much more data are we going to get for the forecasting phase? How much inference time should I be aiming for on the current submission, so I don't just get a timeout on final submission?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3021643": "From January 14th to July 12th there are approximately 124 days of market (more if symbol ids are cryptos).\nFrom data description: 'during the final weeks of the model training phase we will be extending the public test set to include data closer to the submission deadline. Predictions on this extended set will not be scored'.\nSo we don't know how many date_id will be in test set.\nThis info is very important to set the validation framework properly.\nCould you please give me an approximated number of date_id's?", "3021981": "blindape just my estimation. This will be about 119~120 days for public data and about 119~120 days for private data.\n\nFrom the train data, we can see the how much the row data per date_id.\nThis is the dataframe screen shot and plot.\n\n\n\n\n\nLooking at the plot, it appears to be converging upward to the right. For now, let’s try the following calculation using the very last date_id. (4.5 million row data is written in the data explanation)\n\n\n\nAssuming there are 20 business days in a month, the number of months can be calculated using the following division.\n\n\n\nThis indicates exactly 6 months. Therefore, it’s likely that there are 6 months of public data and 6 months of private data, making the test data span exactly one year. However, this is just my speculation!", "3022005": "I reached similar conclusion. Using 4.5M rows as fold length instead of computing using dates.\nIt gives me these folds:\n1577 - 1698\n1455 - 1576\n1331 - 1454\n1210 - 1330\n1090 - 1209", "3022016": "blindape Yes, that's right! Now, I'm also using the range 1577 - 1698!", "3051666": "Sorry to dig up such an old post, but I couldn't seem to find the answer on this subject. \n\nHow much more data are we going to get for the forecasting phase? How much inference time should I be aiming for on the current submission, so I don't just get a timeout on final submission?" }, "source": "meta" }