Hello, everyone.
\nFirst, I want to thank the organizers for hosting this competition. Working with real-world data was challenging but provided an excellent opportunity to enhance my technical skills for practical applications.
\nBelow, I share my solution:
\nThreshold optimization was performed on each fold's training data and applied to the validation/test sets. Using percentiles further improved both accuracy and robustness. Initial optimization values were derived from the discussion here.
\n ():\n tmp_df = pd.DataFrame({: y, : y_pred})\n oof_initial_thresholds = (\n tmp_df.groupby()[].mean().iloc[:].values.tolist()\n )\n\n oof_threshold_percentiles = [\n (tmp_df[] <= threshold).mean()\n threshold oof_initial_thresholds\n ]\n\n oof_threshold_percentiles\n\noof_initial_thresholds = calc_initial_th(y, y_pred)\n\n.optimizer = minimize(\n eval_preds_percentile,\n x0=oof_initial_thresholds,\n args=(y, y_pred),\n method=,\n bounds=[(, ), (, ), (, )],\n)\nI used 5-fold StratifiedKFold. However, as you may know, scores can vary significantly depending on the seed. In the aforementioned notebook, the results were averaged across 10 seeds for submission. Similarly, in validation, I evaluated the voting results from 10 seeds as the score for each experiment. I also evaluated the 10-seed results in validation and optimized parameters for each seed using Optuna.
\nFor example, one submission produced the following seed-dependent score variations:
\n\nI monitored both the voting score and the average of the individual scores. For instance, in the example above, the voting score is 0.49218340134866767, and the average score is 0.4839108063626429. For submissions, I performed a voting ensemble using 5 folds × 10 seeds per model.
\nWhile I aimed to make the evaluation and learning process as robust as possible, I think my ranking still depended heavily on luck.
\nThank you for reading!
Hello, everyone.
\nFirst, I want to thank the organizers for hosting this competition. Working with real-world data was challenging but provided an excellent opportunity to enhance my technical skills for practical applications.
\nBelow, I share my solution:
\nThreshold optimization was performed on each fold's training data and applied to the validation/test sets. Using percentiles further improved both accuracy and robustness. Initial optimization values were derived from the discussion here.
\n ():\n tmp_df = pd.DataFrame({: y, : y_pred})\n oof_initial_thresholds = (\n tmp_df.groupby()[].mean().iloc[:].values.tolist()\n )\n\n oof_threshold_percentiles = [\n (tmp_df[] <= threshold).mean()\n threshold oof_initial_thresholds\n ]\n\n oof_threshold_percentiles\n\noof_initial_thresholds = calc_initial_th(y, y_pred)\n\n.optimizer = minimize(\n eval_preds_percentile,\n x0=oof_initial_thresholds,\n args=(y, y_pred),\n method=,\n bounds=[(, ), (, ), (, )],\n)\nI used 5-fold StratifiedKFold. However, as you may know, scores can vary significantly depending on the seed. In the aforementioned notebook, the results were averaged across 10 seeds for submission. Similarly, in validation, I evaluated the voting results from 10 seeds as the score for each experiment. I also evaluated the 10-seed results in validation and optimized parameters for each seed using Optuna.
\nFor example, one submission produced the following seed-dependent score variations:
\n\nI monitored both the voting score and the average of the individual scores. For instance, in the example above, the voting score is 0.49218340134866767, and the average score is 0.4839108063626429. For submissions, I performed a voting ensemble using 5 folds × 10 seeds per model.
\nWhile I aimed to make the evaluation and learning process as robust as possible, I think my ranking still depended heavily on luck.
\nThank you for reading!