{ "id": 337891, "title": "Difference between DART algorithm and Neural Network dropout?", "url": "/competitions/amex-default-prediction/discussion/337891", "author_name": "", "post_date": "2022-07-18T07:06:51.770531Z", "votes": 1, "comment_count": 4, "views": 0, "content": "

Can someone answer the following questions?

\n
    \n
  1. what is Architectural Difference between this two terms?
    2. \n
  2. If it is to regularize the training module then due to high training time of DART boosting Technique why
    \ndon't we get same accuracy through supervised NN?
    3. \n
", "messages": [ { "id": "1860169", "postDate": "07/18/2022 07:06:51", "content": "

Can someone answer the following questions?

\n
    \n
  1. what is Architectural Difference between this two terms?
    2. \n
  2. If it is to regularize the training module then due to high training time of DART boosting Technique why
    \ndon't we get same accuracy through supervised NN?
    3. \n
", "rawMarkdown": "Can someone answer the following questions?\n\n1. what is Architectural Difference between this two terms?\n2. If it is to regularize the training module then due to high training time of DART boosting Technique why \ndon't we get same accuracy through supervised NN?", "votes": null }, { "id": "1860262", "postDate": "07/18/2022 08:25:47", "content": "

@dhruv8 Let me answer the first question:

\n\n

In both architectures, dropout occurs only during training; for inference all neurons or trees are active.

", "rawMarkdown": "dhruv8 Let me answer the first question:\n- Dropout in neural networks switches off single neurons at random, which forces the remaining neurons to learn more.\n- Dropout in DART switches off whole decision trees at random, which forces the remaining trees to learn more.\n\nIn both architectures, dropout occurs only during training; for inference all neurons or trees are active.", "votes": null }, { "id": "1860271", "postDate": "07/18/2022 08:27:24", "content": "
\n

why don't we get same accuracy through supervised NN?

\n
\n

Because DART works with decision trees, which seem to be generally superior in this competition. In many tabular data competitions, tree-based models perform the best.

", "rawMarkdown": "> why don't we get same accuracy through supervised NN?\n\nBecause DART works with decision trees, which seem to be generally superior in this competition. In many tabular data competitions, tree-based models perform the best.", "votes": null }, { "id": "1860357", "postDate": "07/18/2022 09:23:10", "content": "

These two references cover the dropout in NNs:
\nImproving neural networks by preventing\nco-adaptation of feature detectors
\nA Gentle Introduction to Dropout for Regularizing Deep Neural Networks

", "rawMarkdown": "These two references cover the dropout in NNs:\n[Improving neural networks by preventing\nco-adaptation of feature detectors](https://arxiv.org/pdf/1207.0580.pdf)\n[A Gentle Introduction to Dropout for Regularizing Deep Neural Networks](https://machinelearningmastery.com/dropout-for-regularizing-deep-neural-networks/)", "votes": null }, { "id": "1860865", "postDate": "07/18/2022 16:12:36", "content": "

For anyone who is interested, the DART paper:

\n

https://arxiv.org/pdf/1505.01866.pdf

", "rawMarkdown": "For anyone who is interested, the DART paper:\n\nhttps://arxiv.org/pdf/1505.01866.pdf", "votes": null } ], "comments": [ { "id": 1860262, "author_name": "ambrosm", "author_url": "", "post_date": "07/18/2022 08:25:47", "content": "

@dhruv8 Let me answer the first question:

\n\n

In both architectures, dropout occurs only during training; for inference all neurons or trees are active.

", "votes": null, "replies": [] }, { "id": 1860271, "author_name": "fritzcremer", "author_url": "", "post_date": "07/18/2022 08:27:24", "content": "
\n

why don't we get same accuracy through supervised NN?

\n
\n

Because DART works with decision trees, which seem to be generally superior in this competition. In many tabular data competitions, tree-based models perform the best.

", "votes": null, "replies": [] }, { "id": 1860357, "author_name": "mohammadrahmati", "author_url": "", "post_date": "07/18/2022 09:23:10", "content": "

These two references cover the dropout in NNs:
\nImproving neural networks by preventing\nco-adaptation of feature detectors
\nA Gentle Introduction to Dropout for Regularizing Deep Neural Networks

", "votes": null, "replies": [] }, { "id": 1860865, "author_name": "aliabdin1", "author_url": "", "post_date": "07/18/2022 16:12:36", "content": "

For anyone who is interested, the DART paper:

\n

https://arxiv.org/pdf/1505.01866.pdf

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1860169": "Can someone answer the following questions?\n\n1. what is Architectural Difference between this two terms?\n2. If it is to regularize the training module then due to high training time of DART boosting Technique why \ndon't we get same accuracy through supervised NN?", "1860262": "dhruv8 Let me answer the first question:\n- Dropout in neural networks switches off single neurons at random, which forces the remaining neurons to learn more.\n- Dropout in DART switches off whole decision trees at random, which forces the remaining trees to learn more.\n\nIn both architectures, dropout occurs only during training; for inference all neurons or trees are active.", "1860271": "> why don't we get same accuracy through supervised NN?\n\nBecause DART works with decision trees, which seem to be generally superior in this competition. In many tabular data competitions, tree-based models perform the best.", "1860357": "These two references cover the dropout in NNs:\n[Improving neural networks by preventing\nco-adaptation of feature detectors](https://arxiv.org/pdf/1207.0580.pdf)\n[A Gentle Introduction to Dropout for Regularizing Deep Neural Networks](https://machinelearningmastery.com/dropout-for-regularizing-deep-neural-networks/)", "1860865": "For anyone who is interested, the DART paper:\n\nhttps://arxiv.org/pdf/1505.01866.pdf" }, "source": "meta" }