Has someone used RNN in this task? I'm a beginner, it seemed to me that RNNs are suitable in this case. However, the maximum that we managed to get on GRU is 0.782. Please share your result on RNN. I wanted to understand, either I built an inefficient network, or RNN is not for this task.
", "messages": [ { "id": "1906555", "postDate": "08/20/2022 01:45:59", "content": "Has someone used RNN in this task? I'm a beginner, it seemed to me that RNNs are suitable in this case. However, the maximum that we managed to get on GRU is 0.782. Please share your result on RNN. I wanted to understand, either I built an inefficient network, or RNN is not for this task.
", "rawMarkdown": "Has someone used RNN in this task? I'm a beginner, it seemed to me that RNNs are suitable in this case. However, the maximum that we managed to get on GRU is 0.782. Please share your result on RNN. I wanted to understand, either I built an inefficient network, or RNN is not for this task.", "votes": null }, { "id": "1907019", "postDate": "08/20/2022 11:58:39", "content": "I've also tried a lot with GRU and transformers, but none of them work well. Maybe I haven't found the right way to use it in this competition.
", "rawMarkdown": "I've also tried a lot with GRU and transformers, but none of them work well. Maybe I haven't found the right way to use it in this competition.", "votes": null }, { "id": "1907054", "postDate": "08/20/2022 12:20:32", "content": "For me, NN on aggregated features turned out to be more effective. About the same as LightGBM.
", "rawMarkdown": "For me, NN on aggregated features turned out to be more effective. About the same as LightGBM.", "votes": null }, { "id": "1907508", "postDate": "08/20/2022 21:05:10", "content": "My maximum on LSTM with Attention is right around 0.780-0.782 as well on a validation set. Ensembling my model was able to boost it a tiny bit to 0.783.
\nThe tricky part seems to be in effectively regularizing the recurrent connections, since it can easily overfit to 0.79-0.8+.
", "rawMarkdown": "My maximum on LSTM with Attention is right around 0.780-0.782 as well on a validation set. [Ensembling](https://www.kaggle.com/competitions/amex-default-prediction/discussion/346143) my model was able to boost it a tiny bit to 0.783.\n\nThe tricky part seems to be in effectively regularizing the recurrent connections, since it can easily overfit to 0.79-0.8+.", "votes": null }, { "id": "1907703", "postDate": "08/21/2022 02:04:08", "content": "Thank you, apparently RNN is not so effective in this task.
", "rawMarkdown": "Thank you, apparently RNN is not so effective in this task.", "votes": null }, { "id": "1907967", "postDate": "08/21/2022 08:40:57", "content": "I tried with 1layer Gru, and was able to achive lb score: .79 after trying lot of changes. My observation is that since most of them are easy examples, we need to make to model focus on the hard ones , i.e to push the gradient.
", "rawMarkdown": "I tried with 1layer Gru, and was able to achive lb score: .79 after trying lot of changes. My observation is that since most of them are easy examples, we need to make to model focus on the hard ones , i.e to push the gradient.", "votes": null }, { "id": "1907993", "postDate": "08/21/2022 09:12:16", "content": "Thank you for advice.
", "rawMarkdown": "Thank you for advice.", "votes": null }, { "id": "1909689", "postDate": "08/22/2022 20:53:32", "content": "I get 795 cv and 796 lb with sequential NN, but it doesn't help)))
", "rawMarkdown": "I get 795 cv and 796 lb with sequential NN, but it doesn't help)))", "votes": null }, { "id": "1910552", "postDate": "08/23/2022 14:02:53", "content": "My maximum on sequential NN was 0.788. For more, apparently, there is not enough experience and knowledge yet).
", "rawMarkdown": "My maximum on sequential NN was 0.788. For more, apparently, there is not enough experience and knowledge yet).", "votes": null } ], "comments": [ { "id": 1907019, "author_name": "zhudong1949", "author_url": "", "post_date": "08/20/2022 11:58:39", "content": "I've also tried a lot with GRU and transformers, but none of them work well. Maybe I haven't found the right way to use it in this competition.
", "votes": null, "replies": [ { "id": 1907054, "author_name": "gribanovsv419", "author_url": "", "post_date": "08/20/2022 12:20:32", "content": "For me, NN on aggregated features turned out to be more effective. About the same as LightGBM.
", "votes": null, "replies": [] } ] }, { "id": 1907508, "author_name": "adamw01", "author_url": "", "post_date": "08/20/2022 21:05:10", "content": "My maximum on LSTM with Attention is right around 0.780-0.782 as well on a validation set. Ensembling my model was able to boost it a tiny bit to 0.783.
\nThe tricky part seems to be in effectively regularizing the recurrent connections, since it can easily overfit to 0.79-0.8+.
", "votes": null, "replies": [ { "id": 1907703, "author_name": "gribanovsv419", "author_url": "", "post_date": "08/21/2022 02:04:08", "content": "Thank you, apparently RNN is not so effective in this task.
", "votes": null, "replies": [] }, { "id": 1909689, "author_name": "simakov", "author_url": "", "post_date": "08/22/2022 20:53:32", "content": "I get 795 cv and 796 lb with sequential NN, but it doesn't help)))
", "votes": null, "replies": [] }, { "id": 1910552, "author_name": "gribanovsv419", "author_url": "", "post_date": "08/23/2022 14:02:53", "content": "My maximum on sequential NN was 0.788. For more, apparently, there is not enough experience and knowledge yet).
", "votes": null, "replies": [] } ] }, { "id": 1907967, "author_name": "narendra", "author_url": "", "post_date": "08/21/2022 08:40:57", "content": "I tried with 1layer Gru, and was able to achive lb score: .79 after trying lot of changes. My observation is that since most of them are easy examples, we need to make to model focus on the hard ones , i.e to push the gradient.
", "votes": null, "replies": [ { "id": 1907993, "author_name": "gribanovsv419", "author_url": "", "post_date": "08/21/2022 09:12:16", "content": "Thank you for advice.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1906555": "Has someone used RNN in this task? I'm a beginner, it seemed to me that RNNs are suitable in this case. However, the maximum that we managed to get on GRU is 0.782. Please share your result on RNN. I wanted to understand, either I built an inefficient network, or RNN is not for this task.", "1907019": "I've also tried a lot with GRU and transformers, but none of them work well. Maybe I haven't found the right way to use it in this competition.", "1907054": "For me, NN on aggregated features turned out to be more effective. About the same as LightGBM.", "1907508": "My maximum on LSTM with Attention is right around 0.780-0.782 as well on a validation set. [Ensembling](https://www.kaggle.com/competitions/amex-default-prediction/discussion/346143) my model was able to boost it a tiny bit to 0.783.\n\nThe tricky part seems to be in effectively regularizing the recurrent connections, since it can easily overfit to 0.79-0.8+.", "1907703": "Thank you, apparently RNN is not so effective in this task.", "1907967": "I tried with 1layer Gru, and was able to achive lb score: .79 after trying lot of changes. My observation is that since most of them are easy examples, we need to make to model focus on the hard ones , i.e to push the gradient.", "1907993": "Thank you for advice.", "1909689": "I get 795 cv and 796 lb with sequential NN, but it doesn't help)))", "1910552": "My maximum on sequential NN was 0.788. For more, apparently, there is not enough experience and knowledge yet)." }, "source": "meta" }