{ "id": 591259, "title": "Final Review Process & Thank You from DRW & Cumberland", "url": "/competitions/drw-crypto-market-prediction/discussion/591259", "author_name": "DRW Trading", "post_date": "2025-07-26T15:37:08.844000", "votes": 15, "comment_count": 24, "views": 0, "content": "

Dear Kagglers,

\n

Thank you to everyone who participated in our first Kaggle competition! We have been so impressed by the creativity and skill you brought to the challenge. Reading through your feedback and comments throughout the competition has been incredibly valuable and gave us great insight into how you approached the problem and what we can improve for future challenges.

\n

We are now in the final stages of reviewing submissions and will announce the winners in the coming weeks. To streamline the process, our research team is conducting a pre-screening phase aimed at identifying the vast majority of submissions with indicators of future peeking, such as:

\n
    \n
  1. Extremely high public leaderboard scores, or strong similarity to submissions with unusually high scores (e.g., linear blending with weaker models still resulting in unusually high correlation with the original submissions in daily score sums).
    2. \n
  2. Abnormal prediction stability metrics, such as unrealistically high Sharpe Ratios or low max drawdowns.
    3. \n
  3. Large discrepancies in performance on the ~30% of private test data that was not included in the earlier dataset. Our research shows that submissions relying on the previous version often perform significantly worse on this unseen portion, leading to much lower private leaderboard scores—whereas valid submissions show only minimal differences.
    4. \n
  4. Unrealistic prediction performance during specific time periods (e.g., >50% correlation scores over several months).
    5. \n
\n

Regarding concerns about feature engineering or hyperparameter tuning using unshuffled test data:

\n
    \n
  1. Our research team tested these approaches on leaked data and found them generally ineffective at significantly improving private leaderboard scores—unless models were directly trained on the leaked labels. This is largely because hacked predictions perform poorly on the unseen portion of the private test set.
    2. \n
  2. We’ve observed that top-performing teams on the private leaderboard typically do not have unusually high public scores—they benefit from consistent performance across both splits.
    3. \n
  3. While we did identify some suspicious activity suggesting tuning on the unshuffled test data, these submissions have not advanced in the final rankings.
    4. \n
  4. That said, we will still require fully reproducible notebooks from top teams and may request clarification on certain tuning strategies for prize consideration.
    5. \n
\n

We expect to share the private leaderboard around July 31, 2025, 12:00 AM CST. Please note this may not be the final ranking—further validation and code review will follow before finalizing prize winners. We will reach out to top-performing teams regarding the details. We deeply appreciate all teams who participated with integrity, especially under circumstances where leaked data was publicly accessible. Your effort has made this competition an invaluable and rewarding experience.

\n

In the meantime, we encourage you to keep sharing your takeaways and interesting discoveries from the challenge. Thank you again for making this such a strong kickoff. We are excited to host more competitions in the future and hope to see many of you join us again. If you are interested in applying your skills to real‑world projects at DRW, check out our current roles here: Work at DRW.

\n

Best,
\nThe DRW & Cumberland Team

", "messages": [ { "id": 3254477, "postDate": "2025-07-26T15:37:08.843Z", "content": "

Dear Kagglers,

\n

Thank you to everyone who participated in our first Kaggle competition! We have been so impressed by the creativity and skill you brought to the challenge. Reading through your feedback and comments throughout the competition has been incredibly valuable and gave us great insight into how you approached the problem and what we can improve for future challenges.

\n

We are now in the final stages of reviewing submissions and will announce the winners in the coming weeks. To streamline the process, our research team is conducting a pre-screening phase aimed at identifying the vast majority of submissions with indicators of future peeking, such as:

\n
    \n
  1. Extremely high public leaderboard scores, or strong similarity to submissions with unusually high scores (e.g., linear blending with weaker models still resulting in unusually high correlation with the original submissions in daily score sums).
    2. \n
  2. Abnormal prediction stability metrics, such as unrealistically high Sharpe Ratios or low max drawdowns.
    3. \n
  3. Large discrepancies in performance on the ~30% of private test data that was not included in the earlier dataset. Our research shows that submissions relying on the previous version often perform significantly worse on this unseen portion, leading to much lower private leaderboard scores—whereas valid submissions show only minimal differences.
    4. \n
  4. Unrealistic prediction performance during specific time periods (e.g., >50% correlation scores over several months).
    5. \n
\n

Regarding concerns about feature engineering or hyperparameter tuning using unshuffled test data:

\n
    \n
  1. Our research team tested these approaches on leaked data and found them generally ineffective at significantly improving private leaderboard scores—unless models were directly trained on the leaked labels. This is largely because hacked predictions perform poorly on the unseen portion of the private test set.
    2. \n
  2. We’ve observed that top-performing teams on the private leaderboard typically do not have unusually high public scores—they benefit from consistent performance across both splits.
    3. \n
  3. While we did identify some suspicious activity suggesting tuning on the unshuffled test data, these submissions have not advanced in the final rankings.
    4. \n
  4. That said, we will still require fully reproducible notebooks from top teams and may request clarification on certain tuning strategies for prize consideration.
    5. \n
\n

We expect to share the private leaderboard around July 31, 2025, 12:00 AM CST. Please note this may not be the final ranking—further validation and code review will follow before finalizing prize winners. We will reach out to top-performing teams regarding the details. We deeply appreciate all teams who participated with integrity, especially under circumstances where leaked data was publicly accessible. Your effort has made this competition an invaluable and rewarding experience.

\n

In the meantime, we encourage you to keep sharing your takeaways and interesting discoveries from the challenge. Thank you again for making this such a strong kickoff. We are excited to host more competitions in the future and hope to see many of you join us again. If you are interested in applying your skills to real‑world projects at DRW, check out our current roles here: Work at DRW.

\n

Best,
\nThe DRW & Cumberland Team

", "rawMarkdown": "Dear Kagglers,\n\nThank you to everyone who participated in our first Kaggle competition! We have been so impressed by the creativity and skill you brought to the challenge. Reading through your feedback and comments throughout the competition has been incredibly valuable and gave us great insight into how you approached the problem and what we can improve for future challenges.\n\nWe are now in the final stages of reviewing submissions and will announce the winners in the coming weeks. To streamline the process, our research team is conducting a pre-screening phase aimed at identifying the vast majority of submissions with indicators of future peeking, such as:\n1. Extremely high public leaderboard scores, or strong similarity to submissions with unusually high scores (e.g., linear blending with weaker models still resulting in unusually high correlation with the original submissions in daily score sums).\n2. Abnormal prediction stability metrics, such as unrealistically high Sharpe Ratios or low max drawdowns.\n3. Large discrepancies in performance on the ~30% of private test data that was not included in the earlier dataset. Our research shows that submissions relying on the previous version often perform significantly worse on this unseen portion, leading to much lower private leaderboard scores—whereas valid submissions show only minimal differences.\n4. Unrealistic prediction performance during specific time periods (e.g., >50% correlation scores over several months).\n\nRegarding concerns about feature engineering or hyperparameter tuning using unshuffled test data:\n1. Our research team tested these approaches on leaked data and found them generally ineffective at significantly improving private leaderboard scores—unless models were directly trained on the leaked labels. This is largely because hacked predictions perform poorly on the unseen portion of the private test set.\n2. We’ve observed that top-performing teams on the private leaderboard typically do not have unusually high public scores—they benefit from consistent performance across both splits.\n3. While we did identify some suspicious activity suggesting tuning on the unshuffled test data, these submissions have not advanced in the final rankings.\n4. That said, we will still require fully reproducible notebooks from top teams and may request clarification on certain tuning strategies for prize consideration.\n\nWe expect to share the private leaderboard around July 31, 2025, 12:00 AM CST. Please note this may not be the final ranking—further validation and code review will follow before finalizing prize winners. We will reach out to top-performing teams regarding the details. We deeply appreciate all teams who participated with integrity, especially under circumstances where leaked data was publicly accessible. Your effort has made this competition an invaluable and rewarding experience.\n\nIn the meantime, we encourage you to keep sharing your takeaways and interesting discoveries from the challenge. Thank you again for making this such a strong kickoff. We are excited to host more competitions in the future and hope to see many of you join us again. **If you are interested in applying your skills to real‑world projects at DRW, check out our current roles here: [Work at DRW](https://www.drw.com/work-at-drw).**\n\nBest,\nThe DRW & Cumberland Team", "votes": 15 }, { "id": 3254531, "postDate": "2025-07-26T17:19:52.227Z", "content": "

I actively participated over the past period. I was aware that there could be data leakage once crypto data was introduced, but I didn’t expect it to be this poorly managed. I had ranked within the top 20, excluding leaderboard scores over 0.3, but after the reboot, the leaderboard became a mess again, and I lost interest. I feel like the time and effort I invested were wasted. I hope proper measures are taken for the next competition.

", "rawMarkdown": " I actively participated over the past period. I was aware that there could be data leakage once crypto data was introduced, but I didn’t expect it to be this poorly managed. I had ranked within the top 20, excluding leaderboard scores over 0.3, but after the reboot, the leaderboard became a mess again, and I lost interest. I feel like the time and effort I invested were wasted. I hope proper measures are taken for the next competition.", "votes": 12, "replies": [ { "id": 3254543, "postDate": "2025-07-26T17:56:44.617Z", "content": "

Thank you for the thoughtful suggestions. We understand how frustrating it can be to see unusual scores on the leaderboard before the competition ends, and we remain fully committed to maintaining the integrity of the competition. We absolutely owe the community a more robust experience next time—with stronger safeguards against future peeking. For future featured competitions, we plan to adopt the time-series API and increase the prize pool a lot to better support fair and competitive participation.

", "rawMarkdown": "Thank you for the thoughtful suggestions. We understand how frustrating it can be to see unusual scores on the leaderboard before the competition ends, and we remain fully committed to maintaining the integrity of the competition. We absolutely owe the community a more robust experience next time—with stronger safeguards against future peeking. For future featured competitions, we plan to adopt the time-series API and increase the prize pool a lot to better support fair and competitive participation.", "votes": 7, "replies": [ { "id": 3254551, "postDate": "2025-07-26T18:13:40.307Z", "content": "

Kudos to that!

", "rawMarkdown": "Kudos to that!" } ] } ] }, { "id": 3254802, "postDate": "2025-07-27T08:58:51.617Z", "content": "

Seconding the other guy, would it be possible to remove the deleted teams? Most of them were likely duplicate accounts anyways (against the competition rules)

", "rawMarkdown": "Seconding the other guy, would it be possible to remove the deleted teams? Most of them were likely duplicate accounts anyways (against the competition rules)", "votes": 5 }, { "id": 3254674, "postDate": "2025-07-27T02:04:49.823Z", "content": "

Could you delete all the already deleted teams (like [Deleted] 625--*) to get a cleaner leaderboard? Thanks for your work.

", "rawMarkdown": "Could you delete all the already deleted teams (like [Deleted] 625*-*-*) to get a cleaner leaderboard? Thanks for your work. \n\n", "votes": 6 }, { "id": 3254525, "postDate": "2025-07-26T17:11:51.097Z", "content": "

To all of the contest organizers, thanks so much for making this competition possible! It was really exciting and rewarding to get to compete. My team and I really learned a lot, and it was really interesting getting to try different things and watch as they helped or didn't help our score.

\n

We greatly appreciate your efforts to ensure fairness in this competition; please note that, throughout this contest, we made sure to act with integrity and carefully follow all contest policies, and, if any questions or concerns arise regarding our submissions, please feel free to reach out and we are happy to share the reproducible code used to generate our results. Again, we really enjoyed the opportunity to participate, and we hope to see more contests like this one!

", "rawMarkdown": "To all of the contest organizers, thanks so much for making this competition possible! It was really exciting and rewarding to get to compete. My team and I really learned a lot, and it was really interesting getting to try different things and watch as they helped or didn't help our score.\n\nWe greatly appreciate your efforts to ensure fairness in this competition; please note that, throughout this contest, we made sure to act with integrity and carefully follow all contest policies, and, if any questions or concerns arise regarding our submissions, please feel free to reach out and we are happy to share the reproducible code used to generate our results. Again, we really enjoyed the opportunity to participate, and we hope to see more contests like this one!", "votes": 3 }, { "id": 3254497, "postDate": "2025-07-26T16:21:03.580Z", "content": "

Thank you for hosting this amazing competition. Although the dataset rebooting phase was a bit frustrating at first, overall it was a truly valuable learning opportunity for me. I gained a lot of insights into building robust pipelines and tackling the challenges of predicting crypto price movements.

\n

I also appreciate the effort your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges.

\n

Thanks again for the opportunity. I look forward to participating in more competitions from you in the future!

", "rawMarkdown": "Thank you for hosting this amazing competition. Although the dataset rebooting phase was a bit frustrating at first, overall it was a truly valuable learning opportunity for me. I gained a lot of insights into building robust pipelines and tackling the challenges of predicting crypto price movements.\n\nI also appreciate the effort your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges.\n\nThanks again for the opportunity. I look forward to participating in more competitions from you in the future!", "replies": [ { "id": 3265997, "postDate": "2025-08-08T08:07:54.170Z", "content": "

joker666666666666

", "rawMarkdown": "joker666666666666" } ] }, { "id": 3254552, "postDate": "2025-07-26T18:17:45.510Z", "content": "

It's disappointing to see my notebook deleted without a clear explanation. My models were trained entirely on the same sample sizes from the training set, and I did not reorder the test set. Other competitors employed similar strategies. In your own words, you emphasized two key directions for this competition:

\n
    \n

  1. Data Exploration and Feature Analysis – Understanding the characteristics of the anonymized proprietary features and extracting meaningful insights from public market data through data mining and statistical analysis.

    2. \n

  2. Advanced Modeling Techniques – Developing machine learning models that effectively select, capture, and integrate as much information as possible from all available features.

    3. \n
\n

I focused on the first aspect, dedicating significant effort to uncover patterns in the data. I shared multiple ideas that could support your work in my supplemental notebooks. I worked diligently, often 12 hours per day for over a month, and yet my submissions were removed without a transparent reason. I want to emphasize that my models were trained solely on the training data.

", "rawMarkdown": "It's disappointing to see my notebook deleted without a clear explanation. My models were trained entirely on the same sample sizes from the training set, and I did not reorder the test set. Other competitors employed similar strategies. In your own words, you emphasized two key directions for this competition:\n\n1. Data Exploration and Feature Analysis – Understanding the characteristics of the anonymized proprietary features and extracting meaningful insights from public market data through data mining and statistical analysis.\n\n2. Advanced Modeling Techniques – Developing machine learning models that effectively select, capture, and integrate as much information as possible from all available features.\n\nI focused on the first aspect, dedicating significant effort to uncover patterns in the data. I shared multiple ideas that could support your work in my supplemental notebooks. I worked diligently, often 12 hours per day for over a month, and yet my submissions were removed without a transparent reason. I want to emphasize that my models were trained solely on the training data.\n", "replies": [ { "id": 3254557, "postDate": "2025-07-26T18:27:53.787Z", "content": "

No worries if your notebook is valid! We noticed borderline-high stability in the prediction, but we can restore the submission if it's confirmed to be valid. Could you please share the notebook with us for review?

", "rawMarkdown": "No worries if your notebook is valid! We noticed borderline-high stability in the prediction, but we can restore the submission if it's confirmed to be valid. Could you please share the notebook with us for review?", "votes": 2, "replies": [ { "id": 3254558, "postDate": "2025-07-26T18:32:51.190Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3254559, "postDate": "2025-07-26T18:34:53.473Z", "content": "

Thank you for your response. Just to clarify — should I share the notebook publicly, or would you prefer I send it directly to you or the competition team for review? Please let me know the preferred method.

", "rawMarkdown": "Thank you for your response. Just to clarify — should I share the notebook publicly, or would you prefer I send it directly to you or the competition team for review? Please let me know the preferred method." }, { "id": 3254562, "postDate": "2025-07-26T18:38:39.297Z", "content": "

We’ll review the message you sent us via email and follow up later.

", "rawMarkdown": "We’ll review the message you sent us via email and follow up later.", "votes": 2 }, { "id": 3254564, "postDate": "2025-07-26T18:41:14.223Z", "content": "

I shared my notebook with you via Kaggle

", "rawMarkdown": "I shared my notebook with you via Kaggle", "votes": 1 } ] } ] }, { "id": 3254544, "postDate": "2025-07-26T17:57:20.907Z", "content": "

It was a great opportunity, the leak and the reboot was extremely frustrating and more so was the uncertainity of the results, it's good to know that an effort has been made to uphold the integrity of the contest. It would've been great had the leak not happened, we could've benefited more as a community from discussions and codes then, but anyways great experience.

", "rawMarkdown": "It was a great opportunity, the leak and the reboot was extremely frustrating and more so was the uncertainity of the results, it's good to know that an effort has been made to uphold the integrity of the contest. It would've been great had the leak not happened, we could've benefited more as a community from discussions and codes then, but anyways great experience.", "votes": 1 }, { "id": 3254506, "postDate": "2025-07-26T16:33:41.120Z", "content": "

Thank you to the DRW & Cumberland team for organizing this fantastic competition and for the thoughtful update on the review process.

\n

While I understand the concerns around overfitting and public data leakage, I just want to sharethat scores around 0.2 on the leaderboard are achievable through legitimate means. Our team was very careful throughout the competition to avoid using any leaked data or engaging in practices that would compromise the integrity of the challenge.

\n

We really appreciated the opportunity to work on such a unique problem, and learned a tremendous amount along the way—especially in areas like modeling financial time series and feature engineering. Regardless of the outcome, this has been one of the most rewarding competitions we've participated in.

\n

Thanks again for running it, and we hope to see more challenges like this in the future!

", "rawMarkdown": "Thank you to the DRW & Cumberland team for organizing this fantastic competition and for the thoughtful update on the review process.\n\nWhile I understand the concerns around overfitting and public data leakage, I just want to sharethat scores around 0.2 on the leaderboard are achievable through legitimate means. Our team was very careful throughout the competition to avoid using any leaked data or engaging in practices that would compromise the integrity of the challenge.\n\nWe really appreciated the opportunity to work on such a unique problem, and learned a tremendous amount along the way—especially in areas like modeling financial time series and feature engineering. Regardless of the outcome, this has been one of the most rewarding competitions we've participated in.\n\nThanks again for running it, and we hope to see more challenges like this in the future!", "votes": -4 }, { "id": 3255192, "postDate": "2025-07-28T07:03:00.393Z", "content": "

Please extend the date.

", "rawMarkdown": "Please extend the date.", "votes": -3 }, { "id": 3258893, "postDate": "2025-07-31T12:20:03.140Z", "content": "

Congratulations to everyone who worked hard in this competition!
\nI'm disappointed that my notebook submission was deleted. Even after requesting a review, there was a misunderstanding about which notebook should have been reviewed.

\n

As you can see, both notebooks used the same methodology, and this is reflected in the final score on the private leaderboard.
\nSo I still don’t understand why one of them was removed.
\n\"my

", "rawMarkdown": "Congratulations to everyone who worked hard in this competition!\nI'm disappointed that my notebook submission was deleted. Even after requesting a review, there was a misunderstanding about which notebook should have been reviewed.\n\nAs you can see, both notebooks used the same methodology, and this is reflected in the final score on the private leaderboard.\nSo I still don’t understand why one of them was removed.\n![my photo](https://i.imgur.com/22EnTD1.png)\n", "replies": [ { "id": 3259053, "postDate": "2025-07-31T19:01:58.360Z", "content": "

Don't worry. You definitely won the competition. I learned a lot from you!

", "rawMarkdown": "Don't worry. You definitely won the competition. I learned a lot from you!", "votes": 2 } ] }, { "id": 3254925, "postDate": "2025-07-27T13:39:36.617Z", "content": "

Thanks for the opportunity.
\nI appreciate the effort of your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges….

", "rawMarkdown": "Thanks for the opportunity.\nI appreciate the effort of your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges....\n" }, { "id": 3254650, "postDate": "2025-07-27T00:20:44.263Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3254607, "postDate": "2025-07-26T21:20:42.867Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3254568, "postDate": "2025-07-26T18:43:55.253Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3254548, "postDate": "2025-07-26T18:08:14.690Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 3254531, "author_name": "seowoohyeon", "author_url": "", "post_date": "2025-07-26T17:19:52.227000", "content": "

I actively participated over the past period. I was aware that there could be data leakage once crypto data was introduced, but I didn’t expect it to be this poorly managed. I had ranked within the top 20, excluding leaderboard scores over 0.3, but after the reboot, the leaderboard became a mess again, and I lost interest. I feel like the time and effort I invested were wasted. I hope proper measures are taken for the next competition.

", "votes": 12, "replies": [ { "id": 3254543, "author_name": "DRW Trading", "author_url": "", "post_date": "2025-07-26T17:56:44.617000", "content": "

Thank you for the thoughtful suggestions. We understand how frustrating it can be to see unusual scores on the leaderboard before the competition ends, and we remain fully committed to maintaining the integrity of the competition. We absolutely owe the community a more robust experience next time—with stronger safeguards against future peeking. For future featured competitions, we plan to adopt the time-series API and increase the prize pool a lot to better support fair and competitive participation.

", "votes": 7, "replies": [ { "id": 3254551, "author_name": "Ary", "author_url": "", "post_date": "2025-07-26T18:13:40.307000", "content": "

Kudos to that!

", "votes": 0, "replies": [] } ] } ] }, { "id": 3254802, "author_name": "Luke Kennedy", "author_url": "", "post_date": "2025-07-27T08:58:51.617000", "content": "

Seconding the other guy, would it be possible to remove the deleted teams? Most of them were likely duplicate accounts anyways (against the competition rules)

", "votes": 5, "replies": [] }, { "id": 3254674, "author_name": "KIT", "author_url": "", "post_date": "2025-07-27T02:04:49.823000", "content": "

Could you delete all the already deleted teams (like [Deleted] 625--*) to get a cleaner leaderboard? Thanks for your work.

", "votes": 6, "replies": [] }, { "id": 3254525, "author_name": "Benjamin Hadad", "author_url": "", "post_date": "2025-07-26T17:11:51.097000", "content": "

To all of the contest organizers, thanks so much for making this competition possible! It was really exciting and rewarding to get to compete. My team and I really learned a lot, and it was really interesting getting to try different things and watch as they helped or didn't help our score.

\n

We greatly appreciate your efforts to ensure fairness in this competition; please note that, throughout this contest, we made sure to act with integrity and carefully follow all contest policies, and, if any questions or concerns arise regarding our submissions, please feel free to reach out and we are happy to share the reproducible code used to generate our results. Again, we really enjoyed the opportunity to participate, and we hope to see more contests like this one!

", "votes": 3, "replies": [] }, { "id": 3254497, "author_name": "Ding Yang Wang", "author_url": "", "post_date": "2025-07-26T16:21:03.580000", "content": "

Thank you for hosting this amazing competition. Although the dataset rebooting phase was a bit frustrating at first, overall it was a truly valuable learning opportunity for me. I gained a lot of insights into building robust pipelines and tackling the challenges of predicting crypto price movements.

\n

I also appreciate the effort your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges.

\n

Thanks again for the opportunity. I look forward to participating in more competitions from you in the future!

", "votes": 0, "replies": [ { "id": 3265997, "author_name": "king ui", "author_url": "", "post_date": "2025-08-08T08:07:54.170000", "content": "

joker666666666666

", "votes": 0, "replies": [] } ] }, { "id": 3254552, "author_name": "EL Younes", "author_url": "", "post_date": "2025-07-26T18:17:45.510000", "content": "

It's disappointing to see my notebook deleted without a clear explanation. My models were trained entirely on the same sample sizes from the training set, and I did not reorder the test set. Other competitors employed similar strategies. In your own words, you emphasized two key directions for this competition:

\n
    \n

  1. Data Exploration and Feature Analysis – Understanding the characteristics of the anonymized proprietary features and extracting meaningful insights from public market data through data mining and statistical analysis.

    2. \n

  2. Advanced Modeling Techniques – Developing machine learning models that effectively select, capture, and integrate as much information as possible from all available features.

    3. \n
\n

I focused on the first aspect, dedicating significant effort to uncover patterns in the data. I shared multiple ideas that could support your work in my supplemental notebooks. I worked diligently, often 12 hours per day for over a month, and yet my submissions were removed without a transparent reason. I want to emphasize that my models were trained solely on the training data.

", "votes": 0, "replies": [ { "id": 3254557, "author_name": "DRW Trading", "author_url": "", "post_date": "2025-07-26T18:27:53.787000", "content": "

No worries if your notebook is valid! We noticed borderline-high stability in the prediction, but we can restore the submission if it's confirmed to be valid. Could you please share the notebook with us for review?

", "votes": 2, "replies": [ { "id": 3254558, "author_name": "", "author_url": "", "post_date": "2025-07-26T18:32:51.190000", "content": "", "votes": 0, "replies": [] }, { "id": 3254559, "author_name": "EL Younes", "author_url": "", "post_date": "2025-07-26T18:34:53.473000", "content": "

Thank you for your response. Just to clarify — should I share the notebook publicly, or would you prefer I send it directly to you or the competition team for review? Please let me know the preferred method.

", "votes": 0, "replies": [] }, { "id": 3254562, "author_name": "DRW Trading", "author_url": "", "post_date": "2025-07-26T18:38:39.297000", "content": "

We’ll review the message you sent us via email and follow up later.

", "votes": 2, "replies": [] }, { "id": 3254564, "author_name": "EL Younes", "author_url": "", "post_date": "2025-07-26T18:41:14.223000", "content": "

I shared my notebook with you via Kaggle

", "votes": 1, "replies": [] } ] } ] }, { "id": 3254544, "author_name": "Ary", "author_url": "", "post_date": "2025-07-26T17:57:20.907000", "content": "

It was a great opportunity, the leak and the reboot was extremely frustrating and more so was the uncertainity of the results, it's good to know that an effort has been made to uphold the integrity of the contest. It would've been great had the leak not happened, we could've benefited more as a community from discussions and codes then, but anyways great experience.

", "votes": 1, "replies": [] }, { "id": 3254506, "author_name": "rookcastles", "author_url": "", "post_date": "2025-07-26T16:33:41.120000", "content": "

Thank you to the DRW & Cumberland team for organizing this fantastic competition and for the thoughtful update on the review process.

\n

While I understand the concerns around overfitting and public data leakage, I just want to sharethat scores around 0.2 on the leaderboard are achievable through legitimate means. Our team was very careful throughout the competition to avoid using any leaked data or engaging in practices that would compromise the integrity of the challenge.

\n

We really appreciated the opportunity to work on such a unique problem, and learned a tremendous amount along the way—especially in areas like modeling financial time series and feature engineering. Regardless of the outcome, this has been one of the most rewarding competitions we've participated in.

\n

Thanks again for running it, and we hope to see more challenges like this in the future!

", "votes": -4, "replies": [] }, { "id": 3255192, "author_name": "Navneet", "author_url": "", "post_date": "2025-07-28T07:03:00.393000", "content": "

Please extend the date.

", "votes": -3, "replies": [] }, { "id": 3258893, "author_name": "EL Younes", "author_url": "", "post_date": "2025-07-31T12:20:03.140000", "content": "

Congratulations to everyone who worked hard in this competition!
\nI'm disappointed that my notebook submission was deleted. Even after requesting a review, there was a misunderstanding about which notebook should have been reviewed.

\n

As you can see, both notebooks used the same methodology, and this is reflected in the final score on the private leaderboard.
\nSo I still don’t understand why one of them was removed.
\n\"my

", "votes": 0, "replies": [ { "id": 3259053, "author_name": "byunjins", "author_url": "", "post_date": "2025-07-31T19:01:58.360000", "content": "

Don't worry. You definitely won the competition. I learned a lot from you!

", "votes": 2, "replies": [] } ] }, { "id": 3254925, "author_name": "Sarah Arshad", "author_url": "", "post_date": "2025-07-27T13:39:36.617000", "content": "

Thanks for the opportunity.
\nI appreciate the effort of your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges….

", "votes": 0, "replies": [] }, { "id": 3254650, "author_name": "", "author_url": "", "post_date": "2025-07-27T00:20:44.263000", "content": "", "votes": 0, "replies": [] }, { "id": 3254607, "author_name": "", "author_url": "", "post_date": "2025-07-26T21:20:42.867000", "content": "", "votes": 0, "replies": [] }, { "id": 3254568, "author_name": "", "author_url": "", "post_date": "2025-07-26T18:43:55.253000", "content": "", "votes": 0, "replies": [] }, { "id": 3254548, "author_name": "", "author_url": "", "post_date": "2025-07-26T18:08:14.690000", "content": "", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "3254477": "Dear Kagglers,\n\nThank you to everyone who participated in our first Kaggle competition! We have been so impressed by the creativity and skill you brought to the challenge. Reading through your feedback and comments throughout the competition has been incredibly valuable and gave us great insight into how you approached the problem and what we can improve for future challenges.\n\nWe are now in the final stages of reviewing submissions and will announce the winners in the coming weeks. To streamline the process, our research team is conducting a pre-screening phase aimed at identifying the vast majority of submissions with indicators of future peeking, such as:\n1. Extremely high public leaderboard scores, or strong similarity to submissions with unusually high scores (e.g., linear blending with weaker models still resulting in unusually high correlation with the original submissions in daily score sums).\n2. Abnormal prediction stability metrics, such as unrealistically high Sharpe Ratios or low max drawdowns.\n3. Large discrepancies in performance on the ~30% of private test data that was not included in the earlier dataset. Our research shows that submissions relying on the previous version often perform significantly worse on this unseen portion, leading to much lower private leaderboard scores—whereas valid submissions show only minimal differences.\n4. Unrealistic prediction performance during specific time periods (e.g., >50% correlation scores over several months).\n\nRegarding concerns about feature engineering or hyperparameter tuning using unshuffled test data:\n1. Our research team tested these approaches on leaked data and found them generally ineffective at significantly improving private leaderboard scores—unless models were directly trained on the leaked labels. This is largely because hacked predictions perform poorly on the unseen portion of the private test set.\n2. We’ve observed that top-performing teams on the private leaderboard typically do not have unusually high public scores—they benefit from consistent performance across both splits.\n3. While we did identify some suspicious activity suggesting tuning on the unshuffled test data, these submissions have not advanced in the final rankings.\n4. That said, we will still require fully reproducible notebooks from top teams and may request clarification on certain tuning strategies for prize consideration.\n\nWe expect to share the private leaderboard around July 31, 2025, 12:00 AM CST. Please note this may not be the final ranking—further validation and code review will follow before finalizing prize winners. We will reach out to top-performing teams regarding the details. We deeply appreciate all teams who participated with integrity, especially under circumstances where leaked data was publicly accessible. Your effort has made this competition an invaluable and rewarding experience.\n\nIn the meantime, we encourage you to keep sharing your takeaways and interesting discoveries from the challenge. Thank you again for making this such a strong kickoff. We are excited to host more competitions in the future and hope to see many of you join us again. **If you are interested in applying your skills to real‑world projects at DRW, check out our current roles here: [Work at DRW](https://www.drw.com/work-at-drw).**\n\nBest,\nThe DRW & Cumberland Team", "3254531": " I actively participated over the past period. I was aware that there could be data leakage once crypto data was introduced, but I didn’t expect it to be this poorly managed. I had ranked within the top 20, excluding leaderboard scores over 0.3, but after the reboot, the leaderboard became a mess again, and I lost interest. I feel like the time and effort I invested were wasted. I hope proper measures are taken for the next competition.", "3254802": "Seconding the other guy, would it be possible to remove the deleted teams? Most of them were likely duplicate accounts anyways (against the competition rules)", "3254674": "Could you delete all the already deleted teams (like [Deleted] 625*-*-*) to get a cleaner leaderboard? Thanks for your work. \n\n", "3254525": "To all of the contest organizers, thanks so much for making this competition possible! It was really exciting and rewarding to get to compete. My team and I really learned a lot, and it was really interesting getting to try different things and watch as they helped or didn't help our score.\n\nWe greatly appreciate your efforts to ensure fairness in this competition; please note that, throughout this contest, we made sure to act with integrity and carefully follow all contest policies, and, if any questions or concerns arise regarding our submissions, please feel free to reach out and we are happy to share the reproducible code used to generate our results. Again, we really enjoyed the opportunity to participate, and we hope to see more contests like this one!", "3254497": "Thank you for hosting this amazing competition. Although the dataset rebooting phase was a bit frustrating at first, overall it was a truly valuable learning opportunity for me. I gained a lot of insights into building robust pipelines and tackling the challenges of predicting crypto price movements.\n\nI also appreciate the effort your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges.\n\nThanks again for the opportunity. I look forward to participating in more competitions from you in the future!", "3254552": "It's disappointing to see my notebook deleted without a clear explanation. My models were trained entirely on the same sample sizes from the training set, and I did not reorder the test set. Other competitors employed similar strategies. In your own words, you emphasized two key directions for this competition:\n\n1. Data Exploration and Feature Analysis – Understanding the characteristics of the anonymized proprietary features and extracting meaningful insights from public market data through data mining and statistical analysis.\n\n2. Advanced Modeling Techniques – Developing machine learning models that effectively select, capture, and integrate as much information as possible from all available features.\n\nI focused on the first aspect, dedicating significant effort to uncover patterns in the data. I shared multiple ideas that could support your work in my supplemental notebooks. I worked diligently, often 12 hours per day for over a month, and yet my submissions were removed without a transparent reason. I want to emphasize that my models were trained solely on the training data.\n", "3254544": "It was a great opportunity, the leak and the reboot was extremely frustrating and more so was the uncertainity of the results, it's good to know that an effort has been made to uphold the integrity of the contest. It would've been great had the leak not happened, we could've benefited more as a community from discussions and codes then, but anyways great experience.", "3254506": "Thank you to the DRW & Cumberland team for organizing this fantastic competition and for the thoughtful update on the review process.\n\nWhile I understand the concerns around overfitting and public data leakage, I just want to sharethat scores around 0.2 on the leaderboard are achievable through legitimate means. Our team was very careful throughout the competition to avoid using any leaked data or engaging in practices that would compromise the integrity of the challenge.\n\nWe really appreciated the opportunity to work on such a unique problem, and learned a tremendous amount along the way—especially in areas like modeling financial time series and feature engineering. Regardless of the outcome, this has been one of the most rewarding competitions we've participated in.\n\nThanks again for running it, and we hope to see more challenges like this in the future!", "3255192": "Please extend the date.", "3258893": "Congratulations to everyone who worked hard in this competition!\nI'm disappointed that my notebook submission was deleted. Even after requesting a review, there was a misunderstanding about which notebook should have been reviewed.\n\nAs you can see, both notebooks used the same methodology, and this is reflected in the final score on the private leaderboard.\nSo I still don’t understand why one of them was removed.\n![my photo](https://i.imgur.com/22EnTD1.png)\n", "3254925": "Thanks for the opportunity.\nI appreciate the effort of your team put into validating the submissions and maintaining the integrity of the competition. The attention to detail in ensuring fairness and reproducibility is impressive and sets a great standard for future challenges....\n", "3254650": "", "3254607": "", "3254568": "", "3254548": "" } }