![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F175917%2Ff0aae70e62741410f8da32fc21029c8e%2Fdownload%20(5).png?generation=1730109660775026&alt=media\")
Hey, there! I did EDA (Day 0 and Feature Importance analysis) and summarizing some insights from the analysis.
\nHere's the notebook: https://www.kaggle.com/code/ahsuna123/jane-street-24-day-0-eda-and-feature-importance
\nI hope the notebook helps in getting an idea of the important features and the features with the majority of missing values. Hope, it's helpful to everyone as we proceed.
Feel free to add any important insight if that seems useful. Thanks! :)
", "messages": [ { "id": 3030041, "postDate": "2024-10-28T05:11:15.677Z", "content": "Hey, there! I did EDA (Day 0 and Feature Importance analysis) and summarizing some insights from the analysis.
\nHere's the notebook: https://www.kaggle.com/code/ahsuna123/jane-street-24-day-0-eda-and-feature-importance
\nI hope the notebook helps in getting an idea of the important features and the features with the majority of missing values. Hope, it's helpful to everyone as we proceed.
Feel free to add any important insight if that seems useful. Thanks! :)
", "rawMarkdown": "Hey, there! I did EDA (Day 0 and Feature Importance analysis) and summarizing some insights from the analysis.\nHere's the notebook: https://www.kaggle.com/code/ahsuna123/jane-street-24-day-0-eda-and-feature-importance\nI hope the notebook helps in getting an idea of the important features and the features with the majority of missing values. Hope, it's helpful to everyone as we proceed. \n\n1. There are a total of 1699 days of data in train.parquet (i.e. more than four and a half years of trading data). \n2. responder_6 (in brown) most closely follows time horizon 8 (responder_8). \n3. Each trade has an associated weight and resp, which together represents a return on the trade (Can somebody clarify if this is correct? )\n4. Percentage of zero weights is: 0.00%\n5. The minimum weight is: 0.15.\n6. The maximum weight was: 10.24.\n7. resp_5 represents a more conservative strategy.\n8. The number of missing values decrease after 500 days and majority of missing values are prior to 500th day. Prior to day 250, the number of missing values exceed 8 on an average. \n9. It can be seen, these features have nan values and needs to be handled properly:\nfeature_00, feature_01, feature_02 , feature_03, feature_04, feature_21, feature_26, feature_27 and feature_31.\n10. For the first day (day 0) the top 5 most important features appear to be 58, 05, 37, 07 and 49.\n\nFeel free to add any important insight if that seems useful. Thanks! :)\n\n", "votes": 59 }, { "id": 3030230, "postDate": "2024-10-28T10:18:26.443Z", "content": "Feature 61 is interesting. It has fairly low cardinality and is constant for each date_id. It might be picking out a calendar feature.
\n
On a simple model build: the top 5 features (by out of fold shap):
\nI'm surprised to see 61 anywhere near the top 5. Given the small R^2, I'm not sure this is trustworthy.
\nFeature 34, is similarly cyclic but inverted compared to feature 61:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F175917%2F107b0e65e3de0116166cb15de361a57c%2Fdownload%20(8).png?generation=1730114469479192&alt=media\")
\n\nGiven the small R^2, I'm not sure this is trustworthy.
\n
@paddykb Can you please elaborate on this?
\nI did feature importance of my model weights (avg of 5 Folds XGB) and found 34,61,06 in the top 20. Forgive me, I haven't trained my model yet on responders. It's a basic model trained only on features columns.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9778207%2F8e382ab392070cdbf83c42c1d7b2ceb6%2FScreenshot%202024-10-29%20015841.png?generation=1730181540061798&alt=media\")
Low R^2 indicates that our models do not well capture the underlying relationships between the features and the target variable. So, feature importance metrics might reflect some noise and inflate the importance of some weak features making the ordering misleading.
\nHow is your feature importance evaluated? Tree-based models treat this as a regression problem, so anything evaluated in-sample will be optimistic.
", "rawMarkdown": "Low R^2 indicates that our models do not well capture the underlying relationships between the features and the target variable. So, feature importance metrics might reflect some noise and inflate the importance of some weak features making the ordering misleading.\n\nHow is your feature importance evaluated? Tree-based models treat this as a regression problem, so anything evaluated in-sample will be optimistic." } ] }, { "id": 3032744, "postDate": "2024-10-31T10:19:24.503Z", "content": "Not completely consistent, but the average time between peaks is ~20 days
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F175917%2Feb2afaeb4b02be507b7b8148d9f39f7f%2Fdownload%20(12).png?generation=1730370350219403&alt=media\")
\nWhy might that be? 20 trading days in a month?
Groups of features seem to be lagged or measure the same things:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F175917%2F4984b6e31682826a8e8d6e0e7814780c%2FScreenshot%202024-10-31%20at%2011.20.20.png?generation=1730373728924845&alt=media\")
Great visualization. would you mind share your plotting code snippet? I really like the ploting style.
", "rawMarkdown": "Great visualization. would you mind share your plotting code snippet? I really like the ploting style. ", "votes": 2 }, { "id": 3032832, "postDate": "2024-10-31T12:56:04.107Z", "content": "Sorry, that last one is from tableau.
", "rawMarkdown": "Sorry, that last one is from tableau.", "votes": 1 }, { "id": 3032834, "postDate": "2024-10-31T12:58:27.940Z", "content": "No problem. Thanks for the insightful plots!
", "rawMarkdown": "No problem. Thanks for the insightful plots!", "votes": 1 }, { "id": 3033032, "postDate": "2024-10-31T17:38:44.017Z", "content": "I plot the relation between these features that you mentioned in the 3d space and yet I could not come up with any meaningful analysis.
", "rawMarkdown": "I plot the relation between these features that you mentioned in the 3d space and yet I could not come up with any meaningful analysis. ", "votes": -1 }, { "id": 3033353, "postDate": "2024-11-01T02:44:54.590Z", "content": "Thank you for such a detailed analysis! @paddykb
\nI agree! Given, it's 5 days a week trade timeline. It's very much possible that it's a month timeline
@paddykb there are ~21 trading days in a month and ~252 trading days in a year
", "rawMarkdown": "@paddykb there are ~21 trading days in a month and ~252 trading days in a year", "votes": 1 } ] } ] }, { "id": 3031916, "postDate": "2024-10-30T09:50:13.003Z", "content": "Thank you very much for your insightful and thorough analysis. I believe many of the missing values may stem from the fact that some features are rolling features, calculated over 1-year (256 trading days) or 2-year (512 trading days) periods. As for features which consistently have missing values, I think they are rolling features within each day.
", "rawMarkdown": "Thank you very much for your insightful and thorough analysis. I believe many of the missing values may stem from the fact that some features are rolling features, calculated over 1-year (256 trading days) or 2-year (512 trading days) periods. As for features which consistently have missing values, I think they are rolling features within each day.", "votes": 15, "replies": [ { "id": 3031940, "postDate": "2024-10-30T10:26:33.533Z", "content": "Thanks! With your insights, lots of things make sense now :)đź‘Ť
", "rawMarkdown": "Thanks! With your insights, lots of things make sense now :)đź‘Ť", "votes": 2, "replies": [ { "id": 3031998, "postDate": "2024-10-30T12:35:09.700Z", "content": "You are very welcome.
", "rawMarkdown": "You are very welcome." }, { "id": 3032843, "postDate": "2024-10-31T13:09:15.533Z", "content": "@fengpan23 you're right. Didn't think in that direction. That was helpful! :)
", "rawMarkdown": "@fengpan23 you're right. Didn't think in that direction. That was helpful! :)", "votes": 2 } ] } ] }, { "id": 3030379, "postDate": "2024-10-28T13:20:47.213Z", "content": "I think it's likely to be >6 years of data, if we take the usual 252 trading days.
", "rawMarkdown": "I think it's likely to be >6 years of data, if we take the usual 252 trading days. ", "votes": 7, "replies": [ { "id": 3032844, "postDate": "2024-10-31T13:10:10.573Z", "content": "True that! Missed it.
", "rawMarkdown": "True that! Missed it. ", "votes": 1 } ] }, { "id": 3033034, "postDate": "2024-10-31T17:39:24.683Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4979202%2Ff57b960fbaa87bc0fcf0a702de6dc3bd%2FScreenshot%202024-10-31%20at%209.06.31PM.png?generation=1730396357458283&alt=media\")
Here is the relation between top three features in 3d space looks like
", "rawMarkdown": "Here is the relation between top three features in 3d space looks like ", "votes": 1 } ] }, { "id": 3033676, "postDate": "2024-11-01T12:25:00.057Z", "content": "I believe you're making a wrong assumption about the nature of the data - the provided data points aren't trades made within a day, but rather each day includes a continuous data for each symbol_id that is present within a day. The number of time_ids varies on a daily basis because a different set of symbol_ids may be available on a given day. However, for each symbol_id that is present, there are values for each possible time_id.
\nCan be easily checked by plotting a sum of symbol_ids over time - it stays constant within a day.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1540171%2F3ad13e8ef8c0b99dc54a732938d12d08%2FScreenshot%20from%202024-11-01%2014-22-58.png?generation=1730463795286132&alt=media\")
My theory so far is that each time_id represents a 1 min candle bar since each symbol_id has the same number of time_id count for each independent date_id. The count varies around 849 to 1000 bars for each date_id throughout the years. So this can't represent tick data but rather tick data aggregated to time bars within a certain range of trading hours. So from 7am to say 4:30pm for different years.
", "rawMarkdown": "My theory so far is that each time_id represents a 1 min candle bar since each symbol_id has the same number of time_id count for each independent date_id. The count varies around 849 to 1000 bars for each date_id throughout the years. So this can't represent tick data but rather tick data aggregated to time bars within a certain range of trading hours. So from 7am to say 4:30pm for different years.", "votes": 1 } ] }, { "id": 3030442, "postDate": "2024-10-28T14:28:19.393Z", "content": "@ahsuna123 Great notebook! I might have missed it. Was this written somewhere?
\nEach trade has associated weight resp, which together represents .\nHey! @chumajin
\nThis was derived by Carl in the last competition's timeline
@ahsuna123 Thank you! I didn’t participate in past competitions, so I didn’t know. Do you think it’s safe to assume that this can be applied to this competition as well?
", "rawMarkdown": "@ahsuna123 Thank you! I didn’t participate in past competitions, so I didn’t know. Do you think it’s safe to assume that this can be applied to this competition as well?", "votes": 1 }, { "id": 3034000, "postDate": "2024-11-01T17:20:30.297Z", "content": "Hey! @chumajin
\nI am genuinely so sorry. Thanks fo raising this point, I was wrong about the returns. I also didn't participate in the last one and thought Carl came up with that. I looked at the description of last time, it was explicitly mentioned by the hosts. There's no such mention this time. I don't think it's a good practice to assume that. Unless somebody could clarify on this point. I've updated the discussion as well.
Feature 61 is interesting. It has fairly low cardinality and is constant for each date_id. It might be picking out a calendar feature.
\n
On a simple model build: the top 5 features (by out of fold shap):
\nI'm surprised to see 61 anywhere near the top 5. Given the small R^2, I'm not sure this is trustworthy.
\nFeature 34, is similarly cyclic but inverted compared to feature 61:
\n
\n\nGiven the small R^2, I'm not sure this is trustworthy.
\n
@paddykb Can you please elaborate on this?
\nI did feature importance of my model weights (avg of 5 Folds XGB) and found 34,61,06 in the top 20. Forgive me, I haven't trained my model yet on responders. It's a basic model trained only on features columns.
\n
Low R^2 indicates that our models do not well capture the underlying relationships between the features and the target variable. So, feature importance metrics might reflect some noise and inflate the importance of some weak features making the ordering misleading.
\nHow is your feature importance evaluated? Tree-based models treat this as a regression problem, so anything evaluated in-sample will be optimistic.
", "votes": 0, "replies": [] } ] }, { "id": 3032744, "author_name": "paddykb", "author_url": "", "post_date": "2024-10-31T10:19:24.503000", "content": "Not completely consistent, but the average time between peaks is ~20 days
\n
\nWhy might that be? 20 trading days in a month?
Groups of features seem to be lagged or measure the same things:
\n
Great visualization. would you mind share your plotting code snippet? I really like the ploting style.
", "votes": 2, "replies": [] }, { "id": 3032832, "author_name": "paddykb", "author_url": "", "post_date": "2024-10-31T12:56:04.107000", "content": "Sorry, that last one is from tableau.
", "votes": 1, "replies": [] }, { "id": 3032834, "author_name": "SLi", "author_url": "", "post_date": "2024-10-31T12:58:27.940000", "content": "No problem. Thanks for the insightful plots!
", "votes": 1, "replies": [] }, { "id": 3033032, "author_name": "Farhan Kardan", "author_url": "", "post_date": "2024-10-31T17:38:44.017000", "content": "I plot the relation between these features that you mentioned in the 3d space and yet I could not come up with any meaningful analysis.
", "votes": -1, "replies": [] }, { "id": 3033353, "author_name": "AC", "author_url": "", "post_date": "2024-11-01T02:44:54.590000", "content": "Thank you for such a detailed analysis! @paddykb
\nI agree! Given, it's 5 days a week trade timeline. It's very much possible that it's a month timeline
@paddykb there are ~21 trading days in a month and ~252 trading days in a year
", "votes": 1, "replies": [] } ] } ] }, { "id": 3031916, "author_name": "WindClimber", "author_url": "", "post_date": "2024-10-30T09:50:13.003000", "content": "Thank you very much for your insightful and thorough analysis. I believe many of the missing values may stem from the fact that some features are rolling features, calculated over 1-year (256 trading days) or 2-year (512 trading days) periods. As for features which consistently have missing values, I think they are rolling features within each day.
", "votes": 15, "replies": [ { "id": 3031940, "author_name": "SLi", "author_url": "", "post_date": "2024-10-30T10:26:33.533000", "content": "Thanks! With your insights, lots of things make sense now :)đź‘Ť
", "votes": 2, "replies": [ { "id": 3031998, "author_name": "WindClimber", "author_url": "", "post_date": "2024-10-30T12:35:09.700000", "content": "You are very welcome.
", "votes": 0, "replies": [] }, { "id": 3032843, "author_name": "AC", "author_url": "", "post_date": "2024-10-31T13:09:15.533000", "content": "@fengpan23 you're right. Didn't think in that direction. That was helpful! :)
", "votes": 2, "replies": [] } ] } ] }, { "id": 3030379, "author_name": "Abstr Phil", "author_url": "", "post_date": "2024-10-28T13:20:47.213000", "content": "I think it's likely to be >6 years of data, if we take the usual 252 trading days.
", "votes": 7, "replies": [ { "id": 3032844, "author_name": "AC", "author_url": "", "post_date": "2024-10-31T13:10:10.573000", "content": "True that! Missed it.
", "votes": 1, "replies": [] } ] }, { "id": 3033034, "author_name": "Farhan Kardan", "author_url": "", "post_date": "2024-10-31T17:39:24.683000", "content": "
Here is the relation between top three features in 3d space looks like
", "votes": 1, "replies": [] } ] }, { "id": 3033676, "author_name": "Vadim Fedorov", "author_url": "", "post_date": "2024-11-01T12:25:00.057000", "content": "I believe you're making a wrong assumption about the nature of the data - the provided data points aren't trades made within a day, but rather each day includes a continuous data for each symbol_id that is present within a day. The number of time_ids varies on a daily basis because a different set of symbol_ids may be available on a given day. However, for each symbol_id that is present, there are values for each possible time_id.
\nCan be easily checked by plotting a sum of symbol_ids over time - it stays constant within a day.
\n
My theory so far is that each time_id represents a 1 min candle bar since each symbol_id has the same number of time_id count for each independent date_id. The count varies around 849 to 1000 bars for each date_id throughout the years. So this can't represent tick data but rather tick data aggregated to time bars within a certain range of trading hours. So from 7am to say 4:30pm for different years.
", "votes": 1, "replies": [] } ] }, { "id": 3030442, "author_name": "chumajin", "author_url": "", "post_date": "2024-10-28T14:28:19.393000", "content": "@ahsuna123 Great notebook! I might have missed it. Was this written somewhere?
\nEach trade has associated weight resp, which together represents .\nHey! @chumajin
\nThis was derived by Carl in the last competition's timeline
@ahsuna123 Thank you! I didn’t participate in past competitions, so I didn’t know. Do you think it’s safe to assume that this can be applied to this competition as well?
", "votes": 1, "replies": [] }, { "id": 3034000, "author_name": "AC", "author_url": "", "post_date": "2024-11-01T17:20:30.297000", "content": "Hey! @chumajin
\nI am genuinely so sorry. Thanks fo raising this point, I was wrong about the returns. I also didn't participate in the last one and thought Carl came up with that. I looked at the description of last time, it was explicitly mentioned by the hosts. There's no such mention this time. I don't think it's a good practice to assume that. Unless somebody could clarify on this point. I've updated the discussion as well.