Hi Kagglers,
\nI understand that choice of scalers should be dependent on specific tasks, but I read from @mjahrer's 1st place solution in Porto Seguro’s Safe Driver Prediction Competition that RankGauss works usually much better than standard mean/std scaler or min/max, so I am very tempted to make it work.
The question that I have is that some numerical columns have very few unique numbers, which seems to make RankGauss not so useful. In this case, is it a good idea to use another scaler for these low cardinality features? Or more generally, is it ok to use multiple scalers in data preprocessing?
\nThank you!
", "messages": [ { "id": "1867239", "postDate": "07/23/2022 05:19:11", "content": "Hi Kagglers,
\nI understand that choice of scalers should be dependent on specific tasks, but I read from @mjahrer's 1st place solution in Porto Seguro’s Safe Driver Prediction Competition that RankGauss works usually much better than standard mean/std scaler or min/max, so I am very tempted to make it work.
The question that I have is that some numerical columns have very few unique numbers, which seems to make RankGauss not so useful. In this case, is it a good idea to use another scaler for these low cardinality features? Or more generally, is it ok to use multiple scalers in data preprocessing?
\nThank you!
", "rawMarkdown": "Hi Kagglers,\n\nI understand that choice of scalers should be dependent on specific tasks, but I read from @mjahrer's 1st place solution in [Porto Seguro’s Safe Driver Prediction Competition](https://www.kaggle.com/competitions/porto-seguro-safe-driver-prediction) that RankGauss `works usually much better than standard mean/std scaler or min/max`, so I am very tempted to make it work. \n\nThe question that I have is that some numerical columns have very few unique numbers, which seems to make RankGauss not so useful. In this case, is it a good idea to use another scaler for these low cardinality features? Or more generally, is it ok to use multiple scalers in data preprocessing? \n\nThank you!", "votes": null }, { "id": "1867276", "postDate": "07/23/2022 05:46:06", "content": "I think one could look at multiple scalers in your case
", "rawMarkdown": "I think one could look at multiple scalers in your case", "votes": null }, { "id": "1868358", "postDate": "07/23/2022 23:44:21", "content": "That was a tour de force what Michael did in Porto Seguro. As I recall, he got the lead early and nobody came even close, nor did anyone have a clue what was in his solution.
\nStill, the main takeaway for me was his approach, not the way he scaled the data. I think in most cases when I run the same neural network twice, the results will be different. So it is easy to think that one set of parameters is better than the other. Neural networks like small numbers, and small ranges between min and max. Beyond that, I don't think it matters much whether the data is scaled to [-1, 1] (my preferred range for historical reasons) or [-3, 3]. I don't think that a scaling range would play much of a role here, especially for decidedly non-normal data.
Although it was only a semi-serious approach with a cnn, I varied scalers I used and it made a big difference in the \"images\" although did not impact the score. I didn't try RankGauss though.
", "rawMarkdown": "Although it was only a semi-serious approach with a [cnn](https://www.kaggle.com/code/jakelj/amex-cnn-starter-data-image-prediction), I varied scalers I used and it made a big difference in the \"images\" although did not impact the score. I didn't try RankGauss though.", "votes": null }, { "id": "1872854", "postDate": "07/27/2022 09:19:57", "content": "One option would be to use both.
", "rawMarkdown": "One option would be to use both.", "votes": null }, { "id": "1873780", "postDate": "07/27/2022 22:41:03", "content": "Thanks for your notebook Jake. It's very interesting
", "rawMarkdown": "Thanks for your notebook Jake. It's very interesting", "votes": null } ], "comments": [ { "id": 1867276, "author_name": "ravi20076", "author_url": "", "post_date": "07/23/2022 05:46:06", "content": "I think one could look at multiple scalers in your case
", "votes": null, "replies": [] }, { "id": 1868358, "author_name": "tilii7", "author_url": "", "post_date": "07/23/2022 23:44:21", "content": "That was a tour de force what Michael did in Porto Seguro. As I recall, he got the lead early and nobody came even close, nor did anyone have a clue what was in his solution.
\nStill, the main takeaway for me was his approach, not the way he scaled the data. I think in most cases when I run the same neural network twice, the results will be different. So it is easy to think that one set of parameters is better than the other. Neural networks like small numbers, and small ranges between min and max. Beyond that, I don't think it matters much whether the data is scaled to [-1, 1] (my preferred range for historical reasons) or [-3, 3]. I don't think that a scaling range would play much of a role here, especially for decidedly non-normal data.
Although it was only a semi-serious approach with a cnn, I varied scalers I used and it made a big difference in the \"images\" although did not impact the score. I didn't try RankGauss though.
", "votes": null, "replies": [ { "id": 1873780, "author_name": "cdeotte", "author_url": "", "post_date": "07/27/2022 22:41:03", "content": "Thanks for your notebook Jake. It's very interesting
", "votes": null, "replies": [] } ] }, { "id": 1872854, "author_name": "lucasmorin", "author_url": "", "post_date": "07/27/2022 09:19:57", "content": "One option would be to use both.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1867239": "Hi Kagglers,\n\nI understand that choice of scalers should be dependent on specific tasks, but I read from @mjahrer's 1st place solution in [Porto Seguro’s Safe Driver Prediction Competition](https://www.kaggle.com/competitions/porto-seguro-safe-driver-prediction) that RankGauss `works usually much better than standard mean/std scaler or min/max`, so I am very tempted to make it work. \n\nThe question that I have is that some numerical columns have very few unique numbers, which seems to make RankGauss not so useful. In this case, is it a good idea to use another scaler for these low cardinality features? Or more generally, is it ok to use multiple scalers in data preprocessing? \n\nThank you!", "1867276": "I think one could look at multiple scalers in your case", "1868358": "That was a *tour de force* what Michael did in Porto Seguro. As I recall, he got the lead early and nobody came even close, nor did anyone have a clue what was in his solution.\n\nStill, the main takeaway for me was his approach, not the way he scaled the data. I think in most cases when I run the same neural network twice, the results will be different. So it is easy to think that one set of parameters is better than the other. Neural networks like small numbers, and small ranges between `min` and `max`. Beyond that, I don't think it matters much whether the data is scaled to `[-1, 1]` (my preferred range for historical reasons) or `[-3, 3]`. I don't think that a scaling range would play much of a role here, especially for decidedly non-normal data.", "1869265": "Although it was only a semi-serious approach with a [cnn](https://www.kaggle.com/code/jakelj/amex-cnn-starter-data-image-prediction), I varied scalers I used and it made a big difference in the \"images\" although did not impact the score. I didn't try RankGauss though.", "1872854": "One option would be to use both.", "1873780": "Thanks for your notebook Jake. It's very interesting" }, "source": "meta" }