{ "id": 341797, "title": "the problem of \"NN\" and \"GBDT\"", "url": "/competitions/amex-default-prediction/discussion/341797", "author_name": "", "post_date": "2022-08-04T10:24:37.283878700Z", "votes": 2, "comment_count": 4, "views": 0, "content": "

Friends, since i am new here, i have a silly question🤐.
\nI see many teams looking forward teamates who LB 0.79x with \"NN\". what dose \"NN\" mean ? Is it \"Neural Network\"? It seems that scores with \"NN\" is relatively lower than GBDT. So why do they seek for \"NN\"? Dose \"NN\" make an extral sense?

", "messages": [ { "id": "1884154", "postDate": "08/04/2022 10:24:37", "content": "

Friends, since i am new here, i have a silly question🤐.
\nI see many teams looking forward teamates who LB 0.79x with \"NN\". what dose \"NN\" mean ? Is it \"Neural Network\"? It seems that scores with \"NN\" is relatively lower than GBDT. So why do they seek for \"NN\"? Dose \"NN\" make an extral sense?

", "rawMarkdown": "Friends, since i am new here, i have a silly question🤐. \n I see many teams looking forward teamates who LB 0.79x with \"NN\". what dose \"NN\" mean ? Is it \"Neural Network\"? It seems that scores with \"NN\" is relatively lower than GBDT. So why do they seek for \"NN\"? Dose \"NN\" make an extral sense?", "votes": null }, { "id": "1884183", "postDate": "08/04/2022 10:40:34", "content": "

It refers to Neural Networks. GBDT is an acronym for Gradient boosted decision tree

", "rawMarkdown": "It refers to Neural Networks. GBDT is an acronym for Gradient boosted decision tree", "votes": null }, { "id": "1884245", "postDate": "08/04/2022 11:03:44", "content": "

Normally the best score comes from 2nd level model:

\n

2nd level model may be:

\n\n

Blending works better with diverse (less correlated models). At the same time (normally) there is no boost if models performances differ a lot and you blend high performance model with low one.

\n\n

NN models are very different in terms of predictions from any gbt model and everyone want to add good NN to ensemble.

\n

To have maximum boost, NN model should score 0.795+ on public LB

\n

Good ensemble with lgbm catboost XGB NN may give you 0.802

\n
\n

LGBM / GBT models are giving the best score right now and most of the people spent time tuning and training lgbm models leaving NN behind.

\n

and now many teams are looking for someone who spent time only on NN and achieved good results with it.

", "rawMarkdown": "Normally the best score comes from 2nd level model:\n\n2nd level model may be:\n- Weighted ensemble (LR) -> let's call it blending\n- Stacking (many times overfits)\n\nBlending works better with diverse (less correlated models). At the same time (normally) there is no boost if models performances differ a lot and you blend high performance model with low one.\n\n- Blend 0.799 lgbm with 0.799 catboost may give you 0.80\n- Blend 0.799 lgbm with 0.795 catboost may not give you any improvement\n\nNN models are very different in terms of predictions from any gbt model and everyone want to add good NN to ensemble.\n\nTo have maximum boost, NN model should score 0.795+ on public LB\n\nGood ensemble with lgbm catboost XGB NN may give you 0.802 \n\n---\n\nLGBM / GBT models are giving the best score right now and most of the people spent time tuning and training lgbm models leaving NN behind.\n\nand now many teams are looking for someone who spent time only on NN and achieved good results with it.", "votes": null }, { "id": "1884295", "postDate": "08/04/2022 11:22:30", "content": "

@kyakovlev Learned a lot. Sincerely appreciate for you detailed reply

", "rawMarkdown": "kyakovlev Learned a lot. Sincerely appreciate for you detailed reply", "votes": null }, { "id": "1884296", "postDate": "08/04/2022 11:23:13", "content": "

@ravi20076 Thanks, i knew it👍

", "rawMarkdown": "ravi20076 Thanks, i knew it👍", "votes": null } ], "comments": [ { "id": 1884183, "author_name": "ravi20076", "author_url": "", "post_date": "08/04/2022 10:40:34", "content": "

It refers to Neural Networks. GBDT is an acronym for Gradient boosted decision tree

", "votes": null, "replies": [] }, { "id": 1884245, "author_name": "kyakovlev", "author_url": "", "post_date": "08/04/2022 11:03:44", "content": "

Normally the best score comes from 2nd level model:

\n

2nd level model may be:

\n\n

Blending works better with diverse (less correlated models). At the same time (normally) there is no boost if models performances differ a lot and you blend high performance model with low one.

\n\n

NN models are very different in terms of predictions from any gbt model and everyone want to add good NN to ensemble.

\n

To have maximum boost, NN model should score 0.795+ on public LB

\n

Good ensemble with lgbm catboost XGB NN may give you 0.802

\n
\n

LGBM / GBT models are giving the best score right now and most of the people spent time tuning and training lgbm models leaving NN behind.

\n

and now many teams are looking for someone who spent time only on NN and achieved good results with it.

", "votes": null, "replies": [] }, { "id": 1884295, "author_name": "logos0", "author_url": "", "post_date": "08/04/2022 11:22:30", "content": "

@kyakovlev Learned a lot. Sincerely appreciate for you detailed reply

", "votes": null, "replies": [] }, { "id": 1884296, "author_name": "logos0", "author_url": "", "post_date": "08/04/2022 11:23:13", "content": "

@ravi20076 Thanks, i knew it👍

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1884154": "Friends, since i am new here, i have a silly question🤐. \n I see many teams looking forward teamates who LB 0.79x with \"NN\". what dose \"NN\" mean ? Is it \"Neural Network\"? It seems that scores with \"NN\" is relatively lower than GBDT. So why do they seek for \"NN\"? Dose \"NN\" make an extral sense?", "1884183": "It refers to Neural Networks. GBDT is an acronym for Gradient boosted decision tree", "1884245": "Normally the best score comes from 2nd level model:\n\n2nd level model may be:\n- Weighted ensemble (LR) -> let's call it blending\n- Stacking (many times overfits)\n\nBlending works better with diverse (less correlated models). At the same time (normally) there is no boost if models performances differ a lot and you blend high performance model with low one.\n\n- Blend 0.799 lgbm with 0.799 catboost may give you 0.80\n- Blend 0.799 lgbm with 0.795 catboost may not give you any improvement\n\nNN models are very different in terms of predictions from any gbt model and everyone want to add good NN to ensemble.\n\nTo have maximum boost, NN model should score 0.795+ on public LB\n\nGood ensemble with lgbm catboost XGB NN may give you 0.802 \n\n---\n\nLGBM / GBT models are giving the best score right now and most of the people spent time tuning and training lgbm models leaving NN behind.\n\nand now many teams are looking for someone who spent time only on NN and achieved good results with it.", "1884295": "kyakovlev Learned a lot. Sincerely appreciate for you detailed reply", "1884296": "ravi20076 Thanks, i knew it👍" }, "source": "meta" }