![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F83267%2Fba23dee0ecfe485358d7287bae5403fd%2FScreenshot%202022-11-10%20140810.png?generation=1668053389916987&alt=media\")
Hey!
\nI was a little bit intrigued by the co-visitation approach. In particular, I was wondering too what extent people were able to find good hyperparams without having local validation?
\nI ran an optuna hyperparameter study and the results were quite interesting 🙂
\nHere are the results for the top (top 20 or top 40) and the tail (10, 20, 30, or 40) from the notebooks ran on local validation.
\n
Quite interestingly, the hyperparams of the public kernels were the best 😄
\nThis tells me two things. There is a lot of \"implicit\" hyperparam optimization going on between various versions of notebooks to get to the top of public LB. In this competition public LB tracks nicely local validation, essentially the results on public LB have a lot of validity, which helps.
\nAnd the other thing is that kagglers generally have a bit of a spidey sense for picking reasonable hyperaparams, or modeling techniques, generally beyond what non-kagglers are able to pull off. This is a very valuable skillset not that common outside of Kaggle 🙂
\nAnyhow -- this also suggests next steps. Probably tweaking the hyperparams of the co-visitation matrix will not get you very far, best to come up with additional heuristics or a completely novel approach!
\nHappy Kaggling 🙂
\nRadek
PS. I ran this on my own hardware, just a slightly beefier 12-core Ryzen, but none of this would be possible without the great work @dpalbrecht in the Fast Co-Visitation Matrix public kernel. Yeah, I am still using pandas, still running in single thread, but now experiments on the co-visitation matrix are tractable due to the speedups! 🙌
\nHey!
\nI was a little bit intrigued by the co-visitation approach. In particular, I was wondering too what extent people were able to find good hyperparams without having local validation?
\nI ran an optuna hyperparameter study and the results were quite interesting 🙂
\nHere are the results for the top (top 20 or top 40) and the tail (10, 20, 30, or 40) from the notebooks ran on local validation.
\n
Quite interestingly, the hyperparams of the public kernels were the best 😄
\nThis tells me two things. There is a lot of \"implicit\" hyperparam optimization going on between various versions of notebooks to get to the top of public LB. In this competition public LB tracks nicely local validation, essentially the results on public LB have a lot of validity, which helps.
\nAnd the other thing is that kagglers generally have a bit of a spidey sense for picking reasonable hyperaparams, or modeling techniques, generally beyond what non-kagglers are able to pull off. This is a very valuable skillset not that common outside of Kaggle 🙂
\nAnyhow -- this also suggests next steps. Probably tweaking the hyperparams of the co-visitation matrix will not get you very far, best to come up with additional heuristics or a completely novel approach!
\nHappy Kaggling 🙂
\nRadek
PS. I ran this on my own hardware, just a slightly beefier 12-core Ryzen, but none of this would be possible without the great work @dpalbrecht in the Fast Co-Visitation Matrix public kernel. Yeah, I am still using pandas, still running in single thread, but now experiments on the co-visitation matrix are tractable due to the speedups! 🙌
\nHi,
\ncan you explain what is meaning of \"co-visitation matrix\" is it common technique or well known Algorithm?
\nit's my first kickstart competition on kaggle and want to know what is the technique used to search for?
\nthanks in advance.
I tried to give an answer here (plus there is great input from others, please see the awesome comments by @cdeotte).
\nHope this can be of help! 🙂
", "rawMarkdown": "I tried [to give an answer here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/365358) (plus there is great input from others, please see the awesome comments by @cdeotte).\n\nHope this can be of help! 🙂", "votes": null } ], "comments": [ { "id": 2044612, "author_name": "ahmedabolfadl", "author_url": "", "post_date": "11/26/2022 16:54:56", "content": "Hi,
\ncan you explain what is meaning of \"co-visitation matrix\" is it common technique or well known Algorithm?
\nit's my first kickstart competition on kaggle and want to know what is the technique used to search for?
\nthanks in advance.
I tried to give an answer here (plus there is great input from others, please see the awesome comments by @cdeotte).
\nHope this can be of help! 🙂
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2023824": "Hey!\n\nI was a little bit intrigued by the co-visitation approach. In particular, I was wondering too what extent people were able to find good hyperparams without having local validation?\n\nI ran an optuna hyperparameter study and the results were quite interesting 🙂\n\nHere are the results for the top (top 20 or top 40) and the tail (10, 20, 30, or 40) from the notebooks ran on local validation.\n\n\n\nQuite interestingly, the hyperparams of the public kernels were the best 😄\n\nThis tells me two things. There is a lot of \"implicit\" hyperparam optimization going on between various versions of notebooks to get to the top of public LB. In this competition public LB tracks nicely local validation, essentially the results on public LB have a lot of validity, which helps.\n\nAnd the other thing is that kagglers generally have a bit of a spidey sense for picking reasonable hyperaparams, or modeling techniques, generally beyond what non-kagglers are able to pull off. This is a very valuable skillset not that common outside of Kaggle 🙂\n\nAnyhow -- this also suggests next steps. Probably tweaking the hyperparams of the co-visitation matrix will not get you very far, best to come up with additional heuristics or a completely novel approach!\n\nHappy Kaggling 🙂\nRadek\n\nPS. I ran this on my own hardware, just a slightly beefier 12-core Ryzen, but none of this would be possible without the great work @dpalbrecht in the [Fast Co-Visitation Matrix](https://www.kaggle.com/code/dpalbrecht/fast-co-visitation-matrix) public kernel. Yeah, I am still using pandas, still running in single thread, but now experiments on the co-visitation matrix are tractable due to the speedups! 🙌\n\n### Other resources you might find useful:\n\n* [💡 [2 methods] How-to ensemble predictions 🏅🏅🏅](https://www.kaggle.com/code/radek1/2-methods-how-to-ensemble-predictions)\n* [local validation tracks public LB perfecty -- here is the setup](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)\n* [💡 For my friends from Twitter and LinkedIn -- here is how to dive into this competition 🐳](https://www.kaggle.com/competitions/otto-recommender-system/discussion/368560)\n* [Full dataset processed to CSV/parquet files with optimized memory footprint](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363843)\n* [co-visitation matrix - simplified, imprvd logic 🔥](https://www.kaggle.com/code/radek1/co-visitation-matrix-simplified-imprvd-logic)\n* [💡 Word2Vec How-to [training and submission]🚀🚀🚀](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission)", "2044612": "Hi,\ncan you explain what is meaning of \"co-visitation matrix\" is it common technique or well known Algorithm?\nit's my first kickstart competition on kaggle and want to know what is the technique used to search for?\nthanks in advance.", "2044967": "I tried [to give an answer here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/365358) (plus there is great input from others, please see the awesome comments by @cdeotte).\n\nHope this can be of help! 🙂" }, "source": "meta" }