{
  "id": 319267,
  "title": "No deep learning approach here?",
  "url": "/competitions/h-and-m-personalized-fashion-recommendations/discussion/319267",
  "author_name": "",
  "post_date": "2022-04-16T09:43:29.995234100Z",
  "votes": 14,
  "comment_count": 6,
  "views": 0,
  "content": "<p>Hi 'all,<br>\nThrere's so many codes &amp; discussions about lgbmranker or even submissions without modeling, but it seems deep learning approaches don't work because their scores are lower than the others.<br>\nDo you know why deep learning dominated by trees and others?</p>",
  "messages": [
    {
      "id": "1757133",
      "postDate": "04/16/2022 09:43:29",
      "content": "<p>Hi 'all,<br>\nThrere's so many codes &amp; discussions about lgbmranker or even submissions without modeling, but it seems deep learning approaches don't work because their scores are lower than the others.<br>\nDo you know why deep learning dominated by trees and others?</p>",
      "rawMarkdown": "Hi 'all,\nThrere's so many codes & discussions about lgbmranker or even submissions without modeling, but it seems deep learning approaches don't work because their scores are lower than the others.\nDo you know why deep learning dominated by trees and others?",
      "votes": null
    },
    {
      "id": "1762227",
      "postDate": "04/20/2022 14:25:30",
      "content": "<p>I think just because you don't see successful deep models doesn't mean they can't succeed. But there are a few basic reasons why they might be harder to do well with (I've tried a few deep models with very little success proportional to time invested):</p>\n<ul>\n<li>There is a lot of training examples, but if your deep model is trying to learn an embedding for each user it will be challenging because the number of examples <em>per user</em> is very small on average. (this may also hold for tree-based ranking models). Someone pointed this out in another discussion.  On the other hand, classic collaborative filtering models do seem to be able to learn somewhat useful user embeddings (that outperform simple baselines anyways). Classic models have far fewer parameters, so you don't need as many examples per user to fit a useful model.</li>\n<li>The content data about items and users is tabular. It's generally considered to be easier to get tree-based models working well on tabular data, e.g., because converting categorical variables and bow to one-hot encodings for a neural network results in very sparse data matrices and neural networks are generally trickier to train when the data matrices are sparse.</li>\n<li>Generally, implementing and training a good deep model is complicated and takes a long time. There are so many choices in design and making the right ones really requires some expertise. It's for sure much easier to get a decent pre-packed tree-based model up and running and then work to improve that. Optimize for developer time and fast iteration!</li>\n<li>Finally there is the question of what sort of model you are building with your deep NN. Is it a filtering model or a ranking model? The problems of re-ranking and filtering are different. Filtering with a classic collaborative filtering model or with some heuristics seems to be a decent strategy, but it seems that one needs to stack this with a re-ranking algorithm in order to be competetive because the evaluation metric is very sensitive to the order of recommendations. I am not too familiar with the details bbut ranking algos are more complicated than collaborative filtering algos (see the microsoft paper on learning to rank linked from the LGMBRanker docs to get a taste). LGBMRanker is a pre-packaged ranking algo that works pretty well and is tree based. I imagine that trying to roll your own deep ranking algo from scratch here is going to be massively expensive on developer time.</li>\n</ul>",
      "rawMarkdown": "I think just because you don't see successful deep models doesn't mean they can't succeed. But there are a few basic reasons why they might be harder to do well with (I've tried a few deep models with very little success proportional to time invested):\n\n- There is a lot of training examples, but if your deep model is trying to learn an embedding for each user it will be challenging because the number of examples *per user* is very small on average. (this may also hold for tree-based ranking models). Someone pointed this out in another discussion.  On the other hand, classic collaborative filtering models do seem to be able to learn somewhat useful user embeddings (that outperform simple baselines anyways). Classic models have far fewer parameters, so you don't need as many examples per user to fit a useful model.\n- The content data about items and users is tabular. It's generally considered to be easier to get tree-based models working well on tabular data, e.g., because converting categorical variables and bow to one-hot encodings for a neural network results in very sparse data matrices and neural networks are generally trickier to train when the data matrices are sparse.\n- Generally, implementing and training a good deep model is complicated and takes a long time. There are so many choices in design and making the right ones really requires some expertise. It's for sure much easier to get a decent pre-packed tree-based model up and running and then work to improve that. Optimize for developer time and fast iteration!\n- Finally there is the question of what sort of model you are building with your deep NN. Is it a filtering model or a ranking model? The problems of re-ranking and filtering are different. Filtering with a classic collaborative filtering model or with some heuristics seems to be a decent strategy, but it seems that one needs to stack this with a re-ranking algorithm in order to be competetive because the evaluation metric is very sensitive to the order of recommendations. I am not too familiar with the details bbut ranking algos are more complicated than collaborative filtering algos (see the microsoft paper on learning to rank linked from the LGMBRanker docs to get a taste). LGBMRanker is a pre-packaged ranking algo that works pretty well and is tree based. I imagine that trying to roll your own deep ranking algo from scratch here is going to be massively expensive on developer time.",
      "votes": null
    },
    {
      "id": "1763076",
      "postDate": "04/21/2022 08:56:26",
      "content": "<p>So reasonable QA about   deep learning approach absence</p>",
      "rawMarkdown": "So reasonable QA about   deep learning approach absence",
      "votes": null
    },
    {
      "id": "1781582",
      "postDate": "05/08/2022 17:17:17",
      "content": "<p>There are a lot of other better ways of doing this competition</p>",
      "rawMarkdown": "There are a lot of other better ways of doing this competition",
      "votes": null
    },
    {
      "id": "1781657",
      "postDate": "05/08/2022 19:18:40",
      "content": "<p>are you sure ?</p>",
      "rawMarkdown": "are you sure ?",
      "votes": null
    },
    {
      "id": "1782168",
      "postDate": "05/09/2022 10:24:00",
      "content": "<p>We will get to know the answer for this question tomorrow 😄</p>",
      "rawMarkdown": "We will get to know the answer for this question tomorrow 😄",
      "votes": null
    },
    {
      "id": "1782170",
      "postDate": "05/09/2022 10:29:07",
      "content": "<p>I don't use deep learning approach directly for modeling. Very eager to learn the deep learning solutions from the other top ranking teams tomorrow!</p>",
      "rawMarkdown": "I don't use deep learning approach directly for modeling. Very eager to learn the deep learning solutions from the other top ranking teams tomorrow!",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1762227,
      "author_name": "lane203j",
      "author_url": "",
      "post_date": "04/20/2022 14:25:30",
      "content": "<p>I think just because you don't see successful deep models doesn't mean they can't succeed. But there are a few basic reasons why they might be harder to do well with (I've tried a few deep models with very little success proportional to time invested):</p>\n<ul>\n<li>There is a lot of training examples, but if your deep model is trying to learn an embedding for each user it will be challenging because the number of examples <em>per user</em> is very small on average. (this may also hold for tree-based ranking models). Someone pointed this out in another discussion.  On the other hand, classic collaborative filtering models do seem to be able to learn somewhat useful user embeddings (that outperform simple baselines anyways). Classic models have far fewer parameters, so you don't need as many examples per user to fit a useful model.</li>\n<li>The content data about items and users is tabular. It's generally considered to be easier to get tree-based models working well on tabular data, e.g., because converting categorical variables and bow to one-hot encodings for a neural network results in very sparse data matrices and neural networks are generally trickier to train when the data matrices are sparse.</li>\n<li>Generally, implementing and training a good deep model is complicated and takes a long time. There are so many choices in design and making the right ones really requires some expertise. It's for sure much easier to get a decent pre-packed tree-based model up and running and then work to improve that. Optimize for developer time and fast iteration!</li>\n<li>Finally there is the question of what sort of model you are building with your deep NN. Is it a filtering model or a ranking model? The problems of re-ranking and filtering are different. Filtering with a classic collaborative filtering model or with some heuristics seems to be a decent strategy, but it seems that one needs to stack this with a re-ranking algorithm in order to be competetive because the evaluation metric is very sensitive to the order of recommendations. I am not too familiar with the details bbut ranking algos are more complicated than collaborative filtering algos (see the microsoft paper on learning to rank linked from the LGMBRanker docs to get a taste). LGBMRanker is a pre-packaged ranking algo that works pretty well and is tree based. I imagine that trying to roll your own deep ranking algo from scratch here is going to be massively expensive on developer time.</li>\n</ul>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1763076,
      "author_name": "atwelve",
      "author_url": "",
      "post_date": "04/21/2022 08:56:26",
      "content": "<p>So reasonable QA about   deep learning approach absence</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1781582,
      "author_name": "heartquake",
      "author_url": "",
      "post_date": "05/08/2022 17:17:17",
      "content": "<p>There are a lot of other better ways of doing this competition</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1781657,
      "author_name": "tarique7",
      "author_url": "",
      "post_date": "05/08/2022 19:18:40",
      "content": "<p>are you sure ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1782168,
      "author_name": "sudalairajkumar",
      "author_url": "",
      "post_date": "05/09/2022 10:24:00",
      "content": "<p>We will get to know the answer for this question tomorrow 😄</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 1782170,
      "author_name": "lihaorocky",
      "author_url": "",
      "post_date": "05/09/2022 10:29:07",
      "content": "<p>I don't use deep learning approach directly for modeling. Very eager to learn the deep learning solutions from the other top ranking teams tomorrow!</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1757133": "Hi 'all,\nThrere's so many codes & discussions about lgbmranker or even submissions without modeling, but it seems deep learning approaches don't work because their scores are lower than the others.\nDo you know why deep learning dominated by trees and others?",
    "1762227": "I think just because you don't see successful deep models doesn't mean they can't succeed. But there are a few basic reasons why they might be harder to do well with (I've tried a few deep models with very little success proportional to time invested):\n\n- There is a lot of training examples, but if your deep model is trying to learn an embedding for each user it will be challenging because the number of examples *per user* is very small on average. (this may also hold for tree-based ranking models). Someone pointed this out in another discussion.  On the other hand, classic collaborative filtering models do seem to be able to learn somewhat useful user embeddings (that outperform simple baselines anyways). Classic models have far fewer parameters, so you don't need as many examples per user to fit a useful model.\n- The content data about items and users is tabular. It's generally considered to be easier to get tree-based models working well on tabular data, e.g., because converting categorical variables and bow to one-hot encodings for a neural network results in very sparse data matrices and neural networks are generally trickier to train when the data matrices are sparse.\n- Generally, implementing and training a good deep model is complicated and takes a long time. There are so many choices in design and making the right ones really requires some expertise. It's for sure much easier to get a decent pre-packed tree-based model up and running and then work to improve that. Optimize for developer time and fast iteration!\n- Finally there is the question of what sort of model you are building with your deep NN. Is it a filtering model or a ranking model? The problems of re-ranking and filtering are different. Filtering with a classic collaborative filtering model or with some heuristics seems to be a decent strategy, but it seems that one needs to stack this with a re-ranking algorithm in order to be competetive because the evaluation metric is very sensitive to the order of recommendations. I am not too familiar with the details bbut ranking algos are more complicated than collaborative filtering algos (see the microsoft paper on learning to rank linked from the LGMBRanker docs to get a taste). LGBMRanker is a pre-packaged ranking algo that works pretty well and is tree based. I imagine that trying to roll your own deep ranking algo from scratch here is going to be massively expensive on developer time.",
    "1763076": "So reasonable QA about   deep learning approach absence",
    "1781582": "There are a lot of other better ways of doing this competition",
    "1781657": "are you sure ?",
    "1782168": "We will get to know the answer for this question tomorrow 😄",
    "1782170": "I don't use deep learning approach directly for modeling. Very eager to learn the deep learning solutions from the other top ranking teams tomorrow!"
  },
  "source": "meta"
}