![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4001300%2F40bf737fbb6bd3fdf5807324c7f1e1f7%2FClipboard03.jpg?generation=1729184552987927&alt=media\")
I think the key lies in how we use the lag.
\nThe explanation mentions that all of the previous day's responder results are provided when time_id = 0.
\nlags.parquet - Values responder_{.} lagged one date_id. The evaluation API serves entirety lagged responders a date_id date_id's time_id. In other , all previous 's responders will be served step succeeding .\nBut in the lag sample (lags.parquet), we only see the time_id = 0 from the previous day. I believe that, ideally, the sample should include all time_id values on the previous day. (It's confusing otherwise.)
\n
Is it correct to assume that all time_id values from the previous day will actually be available during submission?
\nWith the new API, this part is unclear and hard to verify.
", "messages": [ { "id": "3020583", "postDate": "10/17/2024 17:02:16", "content": "I think the key lies in how we use the lag.
\nThe explanation mentions that all of the previous day's responder results are provided when time_id = 0.
\nlags.parquet - Values responder_{.} lagged one date_id. The evaluation API serves entirety lagged responders a date_id date_id's time_id. In other , all previous 's responders will be served step succeeding .\nBut in the lag sample (lags.parquet), we only see the time_id = 0 from the previous day. I believe that, ideally, the sample should include all time_id values on the previous day. (It's confusing otherwise.)
\n
Is it correct to assume that all time_id values from the previous day will actually be available during submission?
\nWith the new API, this part is unclear and hard to verify.
", "rawMarkdown": "I think the key lies in how we use the lag.\n\nThe explanation mentions that all of the previous day's responder results are provided when time_id = 0.\n~~~\nlags.parquet - Values of responder_{0...8} lagged by one date_id. The evaluation API serves the entirety of the lagged responders for a date_id on that date_id's first time_id. In other words, all of the previous date's responders will be served at the first time step of the succeeding date.\n~~~\n\nBut in the lag sample (lags.parquet), we only see the time_id = 0 from the previous day. I believe that, ideally, the sample should include all time_id values on the previous day. (It's confusing otherwise.)\n\n\n\nIs it correct to assume that all time_id values from the previous day will actually be available during submission?\n\nWith the new API, this part is unclear and hard to verify.", "votes": null }, { "id": "3020603", "postDate": "10/17/2024 17:33:14", "content": "I have verified this by submitting several tests. From what I see, the number of unique time_ids can be as large as 968. So I believe the demo lags they provide is just an confusing example.
", "rawMarkdown": "I have verified this by submitting several tests. From what I see, the number of unique time_ids can be as large as 968. So I believe the demo lags they provide is just an confusing example.", "votes": null }, { "id": "3020695", "postDate": "10/17/2024 19:50:23", "content": "I agree, the example containing only zero-valued time_id is misleading.
\nAs far as I see, we are provided with the full yesterday responders data in the lags argument each time we have time_id == 0
@lihaorocky Thank you for sharing the verification results! It's a perfect answer.
", "rawMarkdown": "lihaorocky Thank you for sharing the verification results! It's a perfect answer.", "votes": null }, { "id": "3020843", "postDate": "10/18/2024 00:20:19", "content": "@kdmitrie Thank you for the comment! That's exactly right!
", "rawMarkdown": "kdmitrie Thank you for the comment! That's exactly right!", "votes": null }, { "id": "3024809", "postDate": "10/22/2024 03:13:00", "content": "Are you using the lags? In my attempts to do so, I end up with very bad results. This isn’t exact code but I’m typing on phone… I’m doing test.join(lags_, on id_columns, left). I engineered features locally but can’t seem to create and use them in predict method.
", "rawMarkdown": "Are you using the lags? In my attempts to do so, I end up with very bad results. This isn’t exact code but I’m typing on phone… I’m doing test.join(lags_, on id_columns, left). I engineered features locally but can’t seem to create and use them in predict method.", "votes": null }, { "id": "3024816", "postDate": "10/22/2024 03:30:54", "content": "@jackvd As for my results so far, the CV has worsened locally, so I'm not using it for predictions. It seems I still need to do a bit more work.
", "rawMarkdown": "jackvd As for my results so far, the CV has worsened locally, so I'm not using it for predictions. It seems I still need to do a bit more work.", "votes": null }, { "id": "3039293", "postDate": "11/07/2024 21:47:01", "content": "Can somebody give us a small set of samples of lag and test files as they are fed from the API.This may solve lots of problems .
\nThank you.
@lihaorocky
\nJust to clarify, you mean that for each unique combination of symbol_id and date_id, we have different lags for 968 time_id values, not just for time_id=0(or lags of time_id!=0 equal to lags of time_id=0)?
i believe so, all previouse date time_id will be served, but today's time_id might not match yesterday's
", "rawMarkdown": "i believe so, all previouse date time_id will be served, but today's time_id might not match yesterday's", "votes": null }, { "id": "3057596", "postDate": "11/28/2024 10:56:37", "content": "how about the days before yesterday, do we have access to them all?
", "rawMarkdown": "how about the days before yesterday, do we have access to them all?", "votes": null }, { "id": "3077466", "postDate": "12/21/2024 02:18:47", "content": "Hi, @kdmitrie
\n\n\nwe are provided with the full yesterday responders data in the lags argument each time we have time_id == 0
\n
I am confused here, for each symbol on a single date, at each time id, there's a different responder value, how can that all be served at a single time_id == 0 ? Do you know how this works?
Can you elaborate more on what it means for today's time_id not to match yesterday's? Does it mean that each day may serve a different set of time_ids e.g. today's data contains (1, 2, 4), while tomorrow's data contains (3, 4, 5)?
Do we have a prior for what the time_ids physically stand for? Maybe each minute of trading?
no. intervals between time id are not guaranteed to be equal
", "rawMarkdown": "no. intervals between time id are not guaranteed to be equal", "votes": null }, { "id": "3079611", "postDate": "12/23/2024 22:27:02", "content": "If the previous date_id's lags are given at time_id = 0, does this mean that for all other time_ids (time_id >0), lags is None?
\nIf not None, then will the first lag of the current time_id at least be given?
I have verified this by submitting several tests. From what I see, the number of unique time_ids can be as large as 968. So I believe the demo lags they provide is just an confusing example.
", "votes": null, "replies": [ { "id": 3020695, "author_name": "kdmitrie", "author_url": "", "post_date": "10/17/2024 19:50:23", "content": "I agree, the example containing only zero-valued time_id is misleading.
\nAs far as I see, we are provided with the full yesterday responders data in the lags argument each time we have time_id == 0
@kdmitrie Thank you for the comment! That's exactly right!
", "votes": null, "replies": [] }, { "id": 3057596, "author_name": "metropolis40", "author_url": "", "post_date": "11/28/2024 10:56:37", "content": "how about the days before yesterday, do we have access to them all?
", "votes": null, "replies": [] }, { "id": 3077466, "author_name": "zhangyue199", "author_url": "", "post_date": "12/21/2024 02:18:47", "content": "Hi, @kdmitrie
\n\n\nwe are provided with the full yesterday responders data in the lags argument each time we have time_id == 0
\n
I am confused here, for each symbol on a single date, at each time id, there's a different responder value, how can that all be served at a single time_id == 0 ? Do you know how this works?
@lihaorocky Thank you for sharing the verification results! It's a perfect answer.
", "votes": null, "replies": [] }, { "id": 3024809, "author_name": "jackvd", "author_url": "", "post_date": "10/22/2024 03:13:00", "content": "Are you using the lags? In my attempts to do so, I end up with very bad results. This isn’t exact code but I’m typing on phone… I’m doing test.join(lags_, on id_columns, left). I engineered features locally but can’t seem to create and use them in predict method.
", "votes": null, "replies": [ { "id": 3024816, "author_name": "chumajin", "author_url": "", "post_date": "10/22/2024 03:30:54", "content": "@jackvd As for my results so far, the CV has worsened locally, so I'm not using it for predictions. It seems I still need to do a bit more work.
", "votes": null, "replies": [] } ] }, { "id": 3044849, "author_name": "arturkhodakovskiy", "author_url": "", "post_date": "11/13/2024 22:24:15", "content": "@lihaorocky
\nJust to clarify, you mean that for each unique combination of symbol_id and date_id, we have different lags for 968 time_id values, not just for time_id=0(or lags of time_id!=0 equal to lags of time_id=0)?
Do we have a prior for what the time_ids physically stand for? Maybe each minute of trading?
no. intervals between time id are not guaranteed to be equal
", "votes": null, "replies": [] } ] } ] }, { "id": 3039293, "author_name": "georgem", "author_url": "", "post_date": "11/07/2024 21:47:01", "content": "Can somebody give us a small set of samples of lag and test files as they are fed from the API.This may solve lots of problems .
\nThank you.
i believe so, all previouse date time_id will be served, but today's time_id might not match yesterday's
", "votes": null, "replies": [ { "id": 3078175, "author_name": "roguekk007", "author_url": "", "post_date": "12/21/2024 22:18:05", "content": "Can you elaborate more on what it means for today's time_id not to match yesterday's? Does it mean that each day may serve a different set of time_ids e.g. today's data contains (1, 2, 4), while tomorrow's data contains (3, 4, 5)?
If the previous date_id's lags are given at time_id = 0, does this mean that for all other time_ids (time_id >0), lags is None?
\nIf not None, then will the first lag of the current time_id at least be given?