I appreciate it's a rough position for everyone and I'm sure the hosts are trying to work out the best solution to salvage the competition. I also appreciate this is not a priority for anyone and is fundamentally a fun challenge and that much is remains true. This is also my first real competition so I have no idea of norms, if this is typical or if I am overstepping for asking this, if so I apologise.
\nBut right now I have no idea what the status of the competition is and it's crickets from hosts and the only activity I see from competitors are time series approach notebooks? Am I missing something or why are people working on time series approaches and why does that have more activity than the real compeition? Have people given up on the \"real\" competition and now just trying to ruin it further or?
\nCan we get some sort of update? Just a simple \"we are aware the persisting issues and are currently considering options\" would be greatly appreciated and if the competition is not salvageable, say that. As it stands I have no idea if it's worth finalising my submission or if I should just scrap everything I've done and forget about it.
\nThanks
", "messages": [ { "id": 3247812, "postDate": "2025-07-13T13:10:36.707Z", "content": "I appreciate it's a rough position for everyone and I'm sure the hosts are trying to work out the best solution to salvage the competition. I also appreciate this is not a priority for anyone and is fundamentally a fun challenge and that much is remains true. This is also my first real competition so I have no idea of norms, if this is typical or if I am overstepping for asking this, if so I apologise.
\nBut right now I have no idea what the status of the competition is and it's crickets from hosts and the only activity I see from competitors are time series approach notebooks? Am I missing something or why are people working on time series approaches and why does that have more activity than the real compeition? Have people given up on the \"real\" competition and now just trying to ruin it further or?
\nCan we get some sort of update? Just a simple \"we are aware the persisting issues and are currently considering options\" would be greatly appreciated and if the competition is not salvageable, say that. As it stands I have no idea if it's worth finalising my submission or if I should just scrap everything I've done and forget about it.
\nThanks
", "rawMarkdown": "I appreciate it's a rough position for everyone and I'm sure the hosts are trying to work out the best solution to salvage the competition. I also appreciate this is not a priority for anyone and is fundamentally a fun challenge and that much is remains true. This is also my first real competition so I have no idea of norms, if this is typical or if I am overstepping for asking this, if so I apologise. \n\nBut right now I have no idea what the status of the competition is and it's crickets from hosts and the only activity I see from competitors are time series approach notebooks? Am I missing something or why are people working on time series approaches and why does that have more activity than the real compeition? Have people given up on the \"real\" competition and now just trying to ruin it further or?\n\nCan we get some sort of update? Just a simple \"we are aware the persisting issues and are currently considering options\" would be greatly appreciated and if the competition is not salvageable, say that. As it stands I have no idea if it's worth finalising my submission or if I should just scrap everything I've done and forget about it.\n\nThanks", "votes": 11 }, { "id": 3247902, "postDate": "2025-07-13T16:17:30.400Z", "content": "There is no real option to salvage this.
", "rawMarkdown": "There is no real option to salvage this. ", "votes": 7, "replies": [ { "id": 3247923, "postDate": "2025-07-13T16:49:21.573Z", "content": "Agreed
\nThis should have been an API competition @lucasmorin
I think API competitions are only for featured competitions, and I think the only way is for to finish the competition and have DRW check all the notebooks to make sure people aren’t using the leaked targets.
", "rawMarkdown": "I think API competitions are only for featured competitions, and I think the only way is for to finish the competition and have DRW check all the notebooks to make sure people aren’t using the leaked targets.", "votes": 1 }, { "id": 3248005, "postDate": "2025-07-13T19:38:44.677Z", "content": "An LLM based approach can be used here perhaps and potential issues could be highlighted for easier and faster turnaround @paperxd
", "rawMarkdown": "An LLM based approach can be used here perhaps and potential issues could be highlighted for easier and faster turnaround @paperxd ", "votes": 1 }, { "id": 3248180, "postDate": "2025-07-14T05:44:21.147Z", "content": "Any feature engineering will be leaky, it's not just the targets. Removing the dates was not a step in the right direction.
", "rawMarkdown": "Any feature engineering will be leaky, it's not just the targets. Removing the dates was not a step in the right direction. ", "votes": -1 } ] }, { "id": 3247983, "postDate": "2025-07-13T18:39:00.220Z", "content": "I think I agree but it would be nice to hear that from the host.
", "rawMarkdown": "I think I agree but it would be nice to hear that from the host." } ] }, { "id": 3248017, "postDate": "2025-07-13T20:06:25.370Z", "content": "There's your question, \"what is the status and what are the rules?\" Which in my view has a simple answer, and there's a separate/implicit/other question \"can someone get around the rules without getting caught?\"
\nThe \"there is no salvaging this\" is an answer to the second question. You decide whether you agree and whether that affects your decision on finishing your own work. The proliferation of high scores on the LB seems unlikely to have much if any correlation with whether/how many people submit REAL hidden cheat code submissions.
\nI'm new to the competition, but it seems clear that the answer to the first is there's no change to the rules since the organizers update a few days ago(?). And I would assume no real chance there will be any change. Regardless of people's opinions about the second question, and gray area ways to take advantage, it's very clear that you can't use test set in any way, and that final solutions will need to submit reproducible notebooks that do not use test set or will be disqualified.
\nJust follow the rules and submit your best model is my suggestion.
", "rawMarkdown": "There's your question, \"what is the status and what are the rules?\" Which in my view has a simple answer, and there's a separate/implicit/other question \"can someone get around the rules without getting caught?\"\n\nThe \"there is no salvaging this\" is an answer to the second question. You decide whether you agree and whether that affects your decision on finishing your own work. The proliferation of high scores on the LB seems unlikely to have much if any correlation with whether/how many people submit REAL hidden cheat code submissions.\n\nI'm new to the competition, but it seems clear that the answer to the first is there's no change to the rules since the organizers update a few days ago(?). And I would assume no real chance there will be any change. Regardless of people's opinions about the second question, and gray area ways to take advantage, it's very clear that you can't use test set in any way, and that final solutions will need to submit reproducible notebooks that do not use test set or will be disqualified. \n\nJust follow the rules and submit your best model is my suggestion. ", "votes": 1, "replies": [ { "id": 3248027, "postDate": "2025-07-13T20:41:23.763Z", "content": "I agree with your conclusion but it's extremely hard to motivate yourself to continue to spend time on tabular approaches when there's a non-zero chance we'll be met with a \"we're allowing time series approaches\" tomorrow
", "rawMarkdown": "I agree with your conclusion but it's extremely hard to motivate yourself to continue to spend time on tabular approaches when there's a non-zero chance we'll be met with a \"we're allowing time series approaches\" tomorrow", "replies": [ { "id": 3248205, "postDate": "2025-07-14T06:43:15.883Z", "content": "Hmm, got it. I think the chance of them allowing that is basically zero.
", "rawMarkdown": "Hmm, got it. I think the chance of them allowing that is basically zero." }, { "id": 3248949, "postDate": "2025-07-15T12:50:43.020Z", "content": "It seems like DRW went radio silent.
", "rawMarkdown": "It seems like DRW went radio silent.", "votes": -1 } ] } ] }, { "id": 3247976, "postDate": "2025-07-13T18:25:55.157Z", "content": "They could let us use the ordered test set or add a live portion like the Jane Street challenge included. These may be the easiest approaches.
", "rawMarkdown": "They could let us use the ordered test set or add a live portion like the Jane Street challenge included. These may be the easiest approaches.", "votes": 1, "replies": [ { "id": 3247982, "postDate": "2025-07-13T18:38:25.330Z", "content": "Honestly I hope they don't. I think it would be unfair to reward cheaters by allowing them to use models they made whilst the fair-players were wasting time fine tuning models that would then be unusable. I also suspect there are limitations as to why they can't do that or they'd have done it to start with.
", "rawMarkdown": "Honestly I hope they don't. I think it would be unfair to reward cheaters by allowing them to use models they made whilst the fair-players were wasting time fine tuning models that would then be unusable. I also suspect there are limitations as to why they can't do that or they'd have done it to start with.", "votes": 1, "replies": [ { "id": 3247989, "postDate": "2025-07-13T18:54:36.560Z", "content": "I do not see another way out for them, else people can just train their models towards performing like models trained on the ordered test set, at least with hyperparameter tuning and feature selection…
", "rawMarkdown": "I do not see another way out for them, else people can just train their models towards performing like models trained on the ordered test set, at least with hyperparameter tuning and feature selection..." }, { "id": 3248039, "postDate": "2025-07-13T22:10:26.900Z", "content": "Do agree - it's pretty much like that
", "rawMarkdown": "Do agree - it's pretty much like that" } ] } ] }, { "id": 3248992, "postDate": "2025-07-15T13:57:30.800Z", "content": "My opinion about what is going on in this competition:
\nAfter the leaderboard was hacked for the second time, I believe the organizers must take a serious decision — not just regarding the public leaderboard, but also the private one. Both should be disregarded. Instead, the winning solution should be selected based on the best strategy or methodology that can generalize and work well in a real-world environment.
\nThis is a time series regression problem, and I fully understand why the organizers masked the timestamps before shuffling the test set — to prevent cheating by using future information to predict the present. However, the downside is that masking and shuffling add artificial noise that doesn't exist in real scenarios. As a result, only naive models tend to perform well under these conditions, but they are unlikely to be useful in real-life applications.
\nSo I strongly urge the organizers to take meaningful action: the winner of this competition should be selected based on a useful and realistic methodology, not just leaderboard performance. And in future competitions, I suggest using API-based submissions only, without masking or shuffling the time series, to ensure integrity and real-world relevance.
", "rawMarkdown": "My opinion about what is going on in this competition:\n\nAfter the leaderboard was hacked for the second time, I believe the organizers must take a serious decision — not just regarding the public leaderboard, but also the private one. Both should be disregarded. Instead, the winning solution should be selected based on the best strategy or methodology that can generalize and work well in a real-world environment.\n\nThis is a time series regression problem, and I fully understand why the organizers masked the timestamps before shuffling the test set — to prevent cheating by using future information to predict the present. However, the downside is that masking and shuffling add artificial noise that doesn't exist in real scenarios. As a result, only naive models tend to perform well under these conditions, but they are unlikely to be useful in real-life applications.\n\nSo I strongly urge the organizers to take meaningful action: the winner of this competition should be selected based on a useful and realistic methodology, not just leaderboard performance. And in future competitions, I suggest using API-based submissions only, without masking or shuffling the time series, to ensure integrity and real-world relevance.", "votes": 1, "replies": [ { "id": 3249012, "postDate": "2025-07-15T14:16:56.943Z", "content": "@youneseloiarm
\nProbably making this a playground competition and releasing the true dates could be a future recourse.
\nI think this is a lost cause as whatever will be done will always be countered with an easy leak advantage.
I think the best way to handle time series models is an API approach and batched test set release like featured competitions. This will prevent leakage and such consequences.
\nA playground code competition could be a recourse. Perhaps a CPU only code competition???
Noo problem its ok
", "rawMarkdown": "Noo problem its ok" }, { "id": 3249837, "postDate": "2025-07-17T07:56:31.813Z", "content": "I think this is a nonlinear disorder prediction,its time index is useless，becasue test data is disorder,once you use time drift features,the test set attrition rate will be very bad.
", "rawMarkdown": "I think this is a nonlinear disorder prediction,its time index is useless，becasue test data is disorder,once you use time drift features,the test set attrition rate will be very bad." }, { "id": 3249603, "postDate": "2025-07-16T18:31:54.663Z", "content": "i agree about your conclusion
", "rawMarkdown": "i agree about your conclusion" }, { "id": 3248686, "postDate": "2025-07-15T03:34:04.813Z", "content": "I have been dealing with markets in practice, and I agree with the organisers on this.
\nProbably the organisers are looking at, \"if its even possible and if yes how and by how much\" , rather that going down the old road of time series prediction. which is probably the 3rd lesson for data analytics student.
", "rawMarkdown": "I have been dealing with markets in practice, and I agree with the organisers on this.\n\nProbably the organisers are looking at, \"if its even possible and if yes how and by how much\" , rather that going down the old road of time series prediction. which is probably the 3rd lesson for data analytics student.", "isDeleted": true } ], "comments": [ { "id": 3247902, "author_name": "Lucas Morin", "author_url": "", "post_date": "2025-07-13T16:17:30.400000", "content": "There is no real option to salvage this.
", "votes": 7, "replies": [ { "id": 3247923, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2025-07-13T16:49:21.573000", "content": "Agreed
\nThis should have been an API competition @lucasmorin
I think API competitions are only for featured competitions, and I think the only way is for to finish the competition and have DRW check all the notebooks to make sure people aren’t using the leaked targets.
", "votes": 1, "replies": [] }, { "id": 3248005, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2025-07-13T19:38:44.677000", "content": "An LLM based approach can be used here perhaps and potential issues could be highlighted for easier and faster turnaround @paperxd
", "votes": 1, "replies": [] }, { "id": 3248180, "author_name": "Lucas Morin", "author_url": "", "post_date": "2025-07-14T05:44:21.147000", "content": "Any feature engineering will be leaky, it's not just the targets. Removing the dates was not a step in the right direction.
", "votes": -1, "replies": [] } ] }, { "id": 3247983, "author_name": "GreatestCutie", "author_url": "", "post_date": "2025-07-13T18:39:00.220000", "content": "I think I agree but it would be nice to hear that from the host.
", "votes": 0, "replies": [] } ] }, { "id": 3248017, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-07-13T20:06:25.370000", "content": "There's your question, \"what is the status and what are the rules?\" Which in my view has a simple answer, and there's a separate/implicit/other question \"can someone get around the rules without getting caught?\"
\nThe \"there is no salvaging this\" is an answer to the second question. You decide whether you agree and whether that affects your decision on finishing your own work. The proliferation of high scores on the LB seems unlikely to have much if any correlation with whether/how many people submit REAL hidden cheat code submissions.
\nI'm new to the competition, but it seems clear that the answer to the first is there's no change to the rules since the organizers update a few days ago(?). And I would assume no real chance there will be any change. Regardless of people's opinions about the second question, and gray area ways to take advantage, it's very clear that you can't use test set in any way, and that final solutions will need to submit reproducible notebooks that do not use test set or will be disqualified.
\nJust follow the rules and submit your best model is my suggestion.
", "votes": 1, "replies": [ { "id": 3248027, "author_name": "GreatestCutie", "author_url": "", "post_date": "2025-07-13T20:41:23.763000", "content": "I agree with your conclusion but it's extremely hard to motivate yourself to continue to spend time on tabular approaches when there's a non-zero chance we'll be met with a \"we're allowing time series approaches\" tomorrow
", "votes": 0, "replies": [ { "id": 3248205, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-07-14T06:43:15.883000", "content": "Hmm, got it. I think the chance of them allowing that is basically zero.
", "votes": 0, "replies": [] }, { "id": 3248949, "author_name": "Taylor S. Amarel", "author_url": "", "post_date": "2025-07-15T12:50:43.020000", "content": "It seems like DRW went radio silent.
", "votes": -1, "replies": [] } ] } ] }, { "id": 3247976, "author_name": "MalexanderErkel", "author_url": "", "post_date": "2025-07-13T18:25:55.157000", "content": "They could let us use the ordered test set or add a live portion like the Jane Street challenge included. These may be the easiest approaches.
", "votes": 1, "replies": [ { "id": 3247982, "author_name": "GreatestCutie", "author_url": "", "post_date": "2025-07-13T18:38:25.330000", "content": "Honestly I hope they don't. I think it would be unfair to reward cheaters by allowing them to use models they made whilst the fair-players were wasting time fine tuning models that would then be unusable. I also suspect there are limitations as to why they can't do that or they'd have done it to start with.
", "votes": 1, "replies": [ { "id": 3247989, "author_name": "MalexanderErkel", "author_url": "", "post_date": "2025-07-13T18:54:36.560000", "content": "I do not see another way out for them, else people can just train their models towards performing like models trained on the ordered test set, at least with hyperparameter tuning and feature selection…
", "votes": 0, "replies": [] }, { "id": 3248039, "author_name": "Kevin Xu", "author_url": "", "post_date": "2025-07-13T22:10:26.900000", "content": "Do agree - it's pretty much like that
", "votes": 0, "replies": [] } ] } ] }, { "id": 3248992, "author_name": "EL Younes", "author_url": "", "post_date": "2025-07-15T13:57:30.800000", "content": "My opinion about what is going on in this competition:
\nAfter the leaderboard was hacked for the second time, I believe the organizers must take a serious decision — not just regarding the public leaderboard, but also the private one. Both should be disregarded. Instead, the winning solution should be selected based on the best strategy or methodology that can generalize and work well in a real-world environment.
\nThis is a time series regression problem, and I fully understand why the organizers masked the timestamps before shuffling the test set — to prevent cheating by using future information to predict the present. However, the downside is that masking and shuffling add artificial noise that doesn't exist in real scenarios. As a result, only naive models tend to perform well under these conditions, but they are unlikely to be useful in real-life applications.
\nSo I strongly urge the organizers to take meaningful action: the winner of this competition should be selected based on a useful and realistic methodology, not just leaderboard performance. And in future competitions, I suggest using API-based submissions only, without masking or shuffling the time series, to ensure integrity and real-world relevance.
", "votes": 1, "replies": [ { "id": 3249012, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2025-07-15T14:16:56.943000", "content": "@youneseloiarm
\nProbably making this a playground competition and releasing the true dates could be a future recourse.
\nI think this is a lost cause as whatever will be done will always be countered with an easy leak advantage.
I think the best way to handle time series models is an API approach and batched test set release like featured competitions. This will prevent leakage and such consequences.
\nA playground code competition could be a recourse. Perhaps a CPU only code competition???
Noo problem its ok
", "votes": 0, "replies": [] }, { "id": 3249837, "author_name": "Martin Yip", "author_url": "", "post_date": "2025-07-17T07:56:31.813000", "content": "I think this is a nonlinear disorder prediction,its time index is useless，becasue test data is disorder,once you use time drift features,the test set attrition rate will be very bad.
", "votes": 0, "replies": [] }, { "id": 3249603, "author_name": "Kritika Pawar", "author_url": "", "post_date": "2025-07-16T18:31:54.663000", "content": "i agree about your conclusion
", "votes": 0, "replies": [] }, { "id": 3248686, "author_name": "", "author_url": "", "post_date": "2025-07-15T03:34:04.813000", "content": "I have been dealing with markets in practice, and I agree with the organisers on this.
\nProbably the organisers are looking at, \"if its even possible and if yes how and by how much\" , rather that going down the old road of time series prediction. which is probably the 3rd lesson for data analytics student.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3247812": "I appreciate it's a rough position for everyone and I'm sure the hosts are trying to work out the best solution to salvage the competition. I also appreciate this is not a priority for anyone and is fundamentally a fun challenge and that much is remains true. This is also my first real competition so I have no idea of norms, if this is typical or if I am overstepping for asking this, if so I apologise. \n\nBut right now I have no idea what the status of the competition is and it's crickets from hosts and the only activity I see from competitors are time series approach notebooks? Am I missing something or why are people working on time series approaches and why does that have more activity than the real compeition? Have people given up on the \"real\" competition and now just trying to ruin it further or?\n\nCan we get some sort of update? Just a simple \"we are aware the persisting issues and are currently considering options\" would be greatly appreciated and if the competition is not salvageable, say that. As it stands I have no idea if it's worth finalising my submission or if I should just scrap everything I've done and forget about it.\n\nThanks", "3247902": "There is no real option to salvage this. ", "3248017": "There's your question, \"what is the status and what are the rules?\" Which in my view has a simple answer, and there's a separate/implicit/other question \"can someone get around the rules without getting caught?\"\n\nThe \"there is no salvaging this\" is an answer to the second question. You decide whether you agree and whether that affects your decision on finishing your own work. The proliferation of high scores on the LB seems unlikely to have much if any correlation with whether/how many people submit REAL hidden cheat code submissions.\n\nI'm new to the competition, but it seems clear that the answer to the first is there's no change to the rules since the organizers update a few days ago(?). And I would assume no real chance there will be any change. Regardless of people's opinions about the second question, and gray area ways to take advantage, it's very clear that you can't use test set in any way, and that final solutions will need to submit reproducible notebooks that do not use test set or will be disqualified. \n\nJust follow the rules and submit your best model is my suggestion. ", "3247976": "They could let us use the ordered test set or add a live portion like the Jane Street challenge included. These may be the easiest approaches.", "3248992": "My opinion about what is going on in this competition:\n\nAfter the leaderboard was hacked for the second time, I believe the organizers must take a serious decision — not just regarding the public leaderboard, but also the private one. Both should be disregarded. Instead, the winning solution should be selected based on the best strategy or methodology that can generalize and work well in a real-world environment.\n\nThis is a time series regression problem, and I fully understand why the organizers masked the timestamps before shuffling the test set — to prevent cheating by using future information to predict the present. However, the downside is that masking and shuffling add artificial noise that doesn't exist in real scenarios. As a result, only naive models tend to perform well under these conditions, but they are unlikely to be useful in real-life applications.\n\nSo I strongly urge the organizers to take meaningful action: the winner of this competition should be selected based on a useful and realistic methodology, not just leaderboard performance. And in future competitions, I suggest using API-based submissions only, without masking or shuffling the time series, to ensure integrity and real-world relevance.", "3251284": "Noo problem its ok", "3249837": "I think this is a nonlinear disorder prediction,its time index is useless，becasue test data is disorder,once you use time drift features,the test set attrition rate will be very bad.", "3249603": "i agree about your conclusion", "3248686": "I have been dealing with markets in practice, and I agree with the organisers on this.\n\nProbably the organisers are looking at, \"if its even possible and if yes how and by how much\" , rather that going down the old road of time series prediction. which is probably the 3rd lesson for data analytics student." } }