{ "id": 348273, "title": "Leakage from Out-Of-Fold features", "url": "/competitions/amex-default-prediction/discussion/348273", "author_name": "Pavel Vodolazov", "post_date": "2022-08-27T18:02:11.503000", "votes": 7, "comment_count": 0, "views": 0, "content": "

I see people mention leakage when creating a feature in out-of-fold manner and then using it in downstrem models.
\nI mean so-called \"meta-features\" (here here or here ) or \"sequential features\" ( here here or here ).
\nis k-fold CV is leaking? If yes, Why CV does leaking but nested-CV doesn't? Or does?

\n

What if I train it with k-fold CV with different CV-seeds and then use the average?

\n

I think it's an interesting question

\n

Can someone ELI5 me the reason or/and the math behind?

", "messages": [ { "id": 1916239, "postDate": "2022-08-27T18:02:11.503Z", "content": "

I see people mention leakage when creating a feature in out-of-fold manner and then using it in downstrem models.
\nI mean so-called \"meta-features\" (here here or here ) or \"sequential features\" ( here here or here ).
\nis k-fold CV is leaking? If yes, Why CV does leaking but nested-CV doesn't? Or does?

\n

What if I train it with k-fold CV with different CV-seeds and then use the average?

\n

I think it's an interesting question

\n

Can someone ELI5 me the reason or/and the math behind?

", "rawMarkdown": "I see people mention leakage when creating a feature in out-of-fold manner and then using it in downstrem models.\nI mean so-called \"meta-features\" ([here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347786) [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/348097) or [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347908) ) or \"sequential features\" ( [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347651) [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347688) or [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347880) ).\nis k-fold CV is leaking? If yes, Why CV does leaking but nested-CV doesn't? Or does?\n\nWhat if I train it with k-fold CV with different CV-seeds and then use the average?\n\nI think it's an interesting question\n\nCan someone [ELI5](https://www.howtogeek.com/694298/what-does-eli5-mean-and-how-do-you-use-it/) me the reason or/and the math behind?", "votes": 7 } ], "comments": [], "raw_markdown_by_id": { "1916239": "I see people mention leakage when creating a feature in out-of-fold manner and then using it in downstrem models.\nI mean so-called \"meta-features\" ([here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347786) [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/348097) or [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347908) ) or \"sequential features\" ( [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347651) [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347688) or [here](https://www.kaggle.com/competitions/amex-default-prediction/discussion/347880) ).\nis k-fold CV is leaking? If yes, Why CV does leaking but nested-CV doesn't? Or does?\n\nWhat if I train it with k-fold CV with different CV-seeds and then use the average?\n\nI think it's an interesting question\n\nCan someone [ELI5](https://www.howtogeek.com/694298/what-does-eli5-mean-and-how-do-you-use-it/) me the reason or/and the math behind?" } }