One of the ways we can improve candidate generation is via training a matrix factorization model!
\nIn fact, for best results, one should probably generate candidates in multiple ways (including using the co-visitation matrices I discuss here: π‘ What is the co-visitation matrix, really? and pulling it all together using a ranking model like shown here: π‘ [polars] Proof of concept: LGBM Rankerπ§ͺπ§ͺπ§ͺ.
\nEssentially, there are so many things we can do with a matrix factorization model! We can generate candidates, we can score aids based on similarity, cluster them, and so on!
\nSo it is high time in the competition we had a matrix factorization model at our disposal!
\nI share code for how to train one here: π‘Matrix Factorization [PyTorch+Merlin Dataloader]
\nWe preprocess data using polars, obtaining 200+ million aid pairs! To streamline the code I am using data preprocessed to parquet files (no need to deal with jsonl anymore!) that you can find here: π‘ [Howto] Full dataset as parquet/csv files.
We train using Merlin Dataloader and PyTorch!
\nWe then proceed to create a submission using approximate nearest neighbor search.
\nHoping this can open up additional avenues to improve performance!
\nOne of the ways we can improve candidate generation is via training a matrix factorization model!
\nIn fact, for best results, one should probably generate candidates in multiple ways (including using the co-visitation matrices I discuss here: π‘ What is the co-visitation matrix, really? and pulling it all together using a ranking model like shown here: π‘ [polars] Proof of concept: LGBM Rankerπ§ͺπ§ͺπ§ͺ.
\nEssentially, there are so many things we can do with a matrix factorization model! We can generate candidates, we can score aids based on similarity, cluster them, and so on!
\nSo it is high time in the competition we had a matrix factorization model at our disposal!
\nI share code for how to train one here: π‘Matrix Factorization [PyTorch+Merlin Dataloader]
\nWe preprocess data using polars, obtaining 200+ million aid pairs! To streamline the code I am using data preprocessed to parquet files (no need to deal with jsonl anymore!) that you can find here: π‘ [Howto] Full dataset as parquet/csv files.
We train using Merlin Dataloader and PyTorch!
\nWe then proceed to create a submission using approximate nearest neighbor search.
\nHoping this can open up additional avenues to improve performance!
\ngreatβ¦greatβ¦β¦..
", "rawMarkdown": "great...great........", "votes": null }, { "id": "2034739", "postDate": "11/18/2022 11:47:22", "content": "tnx for all the effort!
", "rawMarkdown": "tnx for all the effort!", "votes": null }, { "id": "2036335", "postDate": "11/19/2022 16:43:12", "content": "Nice work π
\nThe Devastator.
", "rawMarkdown": "Nice work π \n\n\nThe Devastator.", "votes": null }, { "id": "2036666", "postDate": "11/20/2022 03:52:19", "content": "Thanks for shareing!
", "rawMarkdown": "Thanks for shareing!", "votes": null }, { "id": "2043588", "postDate": "11/25/2022 19:22:00", "content": "recently i was doing readings about Matrix Factorization and i found many of the ideas relate to recommendation systems solved by this technic rather than going for Graph NN because it has more complexity for modeling the problem situation, i would like to explore this technic using same method you worked on but for instead i will try to use Random search or Greedy algorithm to find best Hyperamaters that performed good fitting ,
\nthis is the resource : Matrix Factorization
Sounds like a plan! π Best of luck in the hyperparameter search! π
", "rawMarkdown": "Sounds like a plan! π Best of luck in the hyperparameter search! π", "votes": null } ], "comments": [ { "id": 2034628, "author_name": "mridulsyed", "author_url": "", "post_date": "11/18/2022 09:44:18", "content": "greatβ¦greatβ¦β¦..
", "votes": null, "replies": [] }, { "id": 2034739, "author_name": "simonveitner", "author_url": "", "post_date": "11/18/2022 11:47:22", "content": "tnx for all the effort!
", "votes": null, "replies": [] }, { "id": 2036335, "author_name": "thedevastator", "author_url": "", "post_date": "11/19/2022 16:43:12", "content": "Nice work π
\nThe Devastator.
", "votes": null, "replies": [] }, { "id": 2036666, "author_name": "yutokaggle", "author_url": "", "post_date": "11/20/2022 03:52:19", "content": "Thanks for shareing!
", "votes": null, "replies": [] }, { "id": 2043588, "author_name": "younesselbrag", "author_url": "", "post_date": "11/25/2022 19:22:00", "content": "recently i was doing readings about Matrix Factorization and i found many of the ideas relate to recommendation systems solved by this technic rather than going for Graph NN because it has more complexity for modeling the problem situation, i would like to explore this technic using same method you worked on but for instead i will try to use Random search or Greedy algorithm to find best Hyperamaters that performed good fitting ,
\nthis is the resource : Matrix Factorization
Sounds like a plan! π Best of luck in the hyperparameter search! π
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "2034460": "One of the ways we can improve candidate generation is via training a matrix factorization model!\n\nIn fact, for best results, one should probably generate candidates in multiple ways (including using the co-visitation matrices I discuss here: [π‘ What is the co-visitation matrix, really?](https://www.kaggle.com/competitions/otto-recommender-system/discussion/365358) and pulling it all together using a ranking model like shown here: [π‘ [polars] Proof of concept: LGBM Rankerπ§ͺπ§ͺπ§ͺ](https://www.kaggle.com/code/radek1/polars-proof-of-concept-lgbm-ranker).\n\nEssentially, there are so many things we can do with a matrix factorization model! We can generate candidates, we can score aids based on similarity, cluster them, and so on!\n\nSo it is high time in the competition we had a matrix factorization model at our disposal!\n\nI share code for how to train one here: [π‘Matrix Factorization [PyTorch+Merlin Dataloader]](https://www.kaggle.com/code/radek1/matrix-factorization-pytorch-merlin-dataloader)\n\nWe preprocess data using polars, obtaining 200+ million aid pairs! To streamline the code I am using data preprocessed to parquet files (no need to deal with `jsonl` anymore!) that you can find here: [π‘ [Howto] Full dataset as parquet/csv files](https://www.kaggle.com/code/radek1/howto-full-dataset-as-parquet-csv-files).\n\nWe train using [Merlin Dataloader](https://twitter.com/radekosmulski/status/1593125626361180160?s=20&t=QEYdqN58jKtN8kXrffUMXQ) and PyTorch!\n\nWe then proceed to create a submission using approximate nearest neighbor search.\n\nHoping this can open up additional avenues to improve performance!\n\n### Other resources you might find useful:\n\n* [π‘ [2 methods] How-to ensemble predictions π π π ](https://www.kaggle.com/code/radek1/2-methods-how-to-ensemble-predictions)\n* [local validation tracks public LB perfecty -- here is the setup](https://www.kaggle.com/competitions/otto-recommender-system/discussion/364991)\n* [π‘ For my friends from Twitter and LinkedIn -- here is how to dive into this competition π³](https://www.kaggle.com/competitions/otto-recommender-system/discussion/368560)\n* [Full dataset processed to CSV/parquet files with optimized memory footprint](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363843)\n* [co-visitation matrix - simplified, imprvd logic π₯](https://www.kaggle.com/code/radek1/co-visitation-matrix-simplified-imprvd-logic)\n* [π‘ Word2Vec How-to [training and submission]πππ](https://www.kaggle.com/code/radek1/word2vec-how-to-training-and-submission)", "2034628": "great...great........", "2034739": "tnx for all the effort!", "2036335": "Nice work π \n\n\nThe Devastator.", "2036666": "Thanks for shareing!", "2043588": "recently i was doing readings about Matrix Factorization and i found many of the ideas relate to recommendation systems solved by this technic rather than going for Graph NN because it has more complexity for modeling the problem situation, i would like to explore this technic using same method you worked on but for instead i will try to use Random search or Greedy algorithm to find best Hyperamaters that performed good fitting , \nthis is the resource : [Matrix Factorization](https://everdark.github.io/k9/notebooks/ml/matrix_factorization/matrix_factorization.nb.html#3_neural_netork_representation)", "2043740": "Sounds like a plan! π Best of luck in the hyperparameter search! π" }, "source": "meta" }