{
  "id": 376789,
  "title": "How much of your max-recall do you get?",
  "url": "/competitions/otto-recommender-system/discussion/376789",
  "author_name": "",
  "post_date": "2023-01-08T14:36:42.257577800Z",
  "votes": 21,
  "comment_count": 16,
  "views": 0,
  "content": "<p>We are currently generating <strong><em>100</em></strong> candidates and our <strong><em>max-recalls at-100</em></strong> as follows:</p>\n<pre><code>clicks recall = 0.6515316707053238\ncarts recall = 0.5054104031001835\norders recall = 0.7048352553279094\n\noverall recall = 0.6396774411973329\n</code></pre>\n<p>And after using our re-rank model we get these scores <strong><em>at-20</em></strong>:</p>\n<pre><code>clicks recall = 0.5340063432688679\ncarts recall = 0.4237135902195957\norders recall = 0.6600268258685532\n\noverall recall = 0.5765308069138975\n</code></pre>\n<p>These are both calculated for validation splits of <a href=\"https://www.kaggle.com/radek1\" target=\"_blank\">@radek1</a>. I just wonder what is your recall difference and the number of candidates you use?</p>",
  "messages": [
    {
      "id": "2091582",
      "postDate": "01/08/2023 14:36:42",
      "content": "<p>We are currently generating <strong><em>100</em></strong> candidates and our <strong><em>max-recalls at-100</em></strong> as follows:</p>\n<pre><code>clicks recall = 0.6515316707053238\ncarts recall = 0.5054104031001835\norders recall = 0.7048352553279094\n\noverall recall = 0.6396774411973329\n</code></pre>\n<p>And after using our re-rank model we get these scores <strong><em>at-20</em></strong>:</p>\n<pre><code>clicks recall = 0.5340063432688679\ncarts recall = 0.4237135902195957\norders recall = 0.6600268258685532\n\noverall recall = 0.5765308069138975\n</code></pre>\n<p>These are both calculated for validation splits of <a href=\"https://www.kaggle.com/radek1\" target=\"_blank\">@radek1</a>. I just wonder what is your recall difference and the number of candidates you use?</p>",
      "rawMarkdown": "We are currently generating ***100*** candidates and our ***max-recalls at-100*** as follows:\n\n```\nclicks recall = 0.6515316707053238\ncarts recall = 0.5054104031001835\norders recall = 0.7048352553279094\n\noverall recall = 0.6396774411973329\n```\n\nAnd after using our re-rank model we get these scores ***at-20***:\n```\nclicks recall = 0.5340063432688679\ncarts recall = 0.4237135902195957\norders recall = 0.6600268258685532\n\noverall recall = 0.5765308069138975\n```\n\nThese are both calculated for validation splits of @radek1. I just wonder what is your recall difference and the number of candidates you use?",
      "votes": null
    },
    {
      "id": "2091683",
      "postDate": "01/08/2023 16:47:25",
      "content": "<pre><code>hitrate@20\n{'clicks': 0.5210141483103858,\n'carts': 0.40768774721044854,\n'orders': 0.6506683121382453,\n'total': 0.5648087262771203}\n\nhitrate@100\n\n{'clicks': 0.6499337643160575,\n'carts': 0.5045632726017821,\n'orders': 0.7074081572967987,\n'total': 0.6408072525902196}\n</code></pre>\n<p>May I know what's your public LB using only recall? I got 0.574 by only using recall.</p>",
      "rawMarkdown": "```\nhitrate@20\n{'clicks': 0.5210141483103858,\n'carts': 0.40768774721044854,\n'orders': 0.6506683121382453,\n'total': 0.5648087262771203}\n\nhitrate@100\n\n{'clicks': 0.6499337643160575,\n'carts': 0.5045632726017821,\n'orders': 0.7074081572967987,\n'total': 0.6408072525902196}\n```\n\nMay I know what's your public LB using only recall? I got 0.574 by only using recall.",
      "votes": null
    },
    {
      "id": "2091692",
      "postDate": "01/08/2023 16:53:57",
      "content": "<p>Only candidate sampling (for Radek's validation):</p>\n<pre><code>Local Val: 0.567\nLB: 0.577\n</code></pre>",
      "rawMarkdown": "Only candidate sampling (for Radek's validation):\n```\nLocal Val: 0.567\nLB: 0.577\n```",
      "votes": null
    },
    {
      "id": "2091695",
      "postDate": "01/08/2023 16:54:54",
      "content": "<p>Using only recall, we got <code>0.577</code> which is same with public notebook <a href=\"https://www.kaggle.com/code/utm529fg/otto-tuning-candidate-rerank-model-lb-0-577\" target=\"_blank\">here</a>. With re-rank model, our public LB score is <code>0.587</code>.</p>",
      "rawMarkdown": "Using only recall, we got `0.577` which is same with public notebook [here](https://www.kaggle.com/code/utm529fg/otto-tuning-candidate-rerank-model-lb-0-577). With re-rank model, our public LB score is `0.587`.",
      "votes": null
    },
    {
      "id": "2091698",
      "postDate": "01/08/2023 16:55:59",
      "content": "<p>Good to know. Thanks.</p>",
      "rawMarkdown": "Good to know. Thanks.",
      "votes": null
    },
    {
      "id": "2091699",
      "postDate": "01/08/2023 16:56:45",
      "content": "<p>Got it. Thanks.</p>",
      "rawMarkdown": "Got it. Thanks.",
      "votes": null
    },
    {
      "id": "2091991",
      "postDate": "01/08/2023 23:31:08",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/nlztrk\" target=\"_blank\">@nlztrk</a>! Thanks you for the shoutout 🤗 Very glad to see my work is of help 🙏</p>",
      "rawMarkdown": "Hey @nlztrk! Thanks you for the shoutout 🤗 Very glad to see my work is of help 🙏",
      "votes": null
    },
    {
      "id": "2091992",
      "postDate": "01/08/2023 23:31:34",
      "content": "<p>very valuable piece of info, <a href=\"https://www.kaggle.com/karakasatarik\" target=\"_blank\">@karakasatarik</a>! Thx for sharing this!!!!  </p>",
      "rawMarkdown": "very valuable piece of info, @karakasatarik! Thx for sharing this!!!!",
      "votes": null
    },
    {
      "id": "2092754",
      "postDate": "01/09/2023 15:12:52",
      "content": "<p>Thank you for sharing the info! Can you also share if you are downsampling the negatives during training? If yes, by how much?</p>",
      "rawMarkdown": "Thank you for sharing the info! Can you also share if you are downsampling the negatives during training? If yes, by how much?",
      "votes": null
    },
    {
      "id": "2092844",
      "postDate": "01/09/2023 16:44:37",
      "content": "<p>Currently <strong>15%</strong>.</p>",
      "rawMarkdown": "Currently **15%**.",
      "votes": null
    },
    {
      "id": "2094629",
      "postDate": "01/10/2023 22:21:03",
      "content": "<p>Great ! did you use the same variables and parameters to train your ranker regarding the type ? ( clicks orders carts … )</p>",
      "rawMarkdown": "Great ! did you use the same variables and parameters to train your ranker regarding the type ? ( clicks orders carts ... )",
      "votes": null
    },
    {
      "id": "2095397",
      "postDate": "01/11/2023 10:26:12",
      "content": "<p>Yep, our feature set is fixed for all of our models for now.</p>",
      "rawMarkdown": "Yep, our feature set is fixed for all of our models for now.",
      "votes": null
    },
    {
      "id": "2098396",
      "postDate": "01/13/2023 14:04:35",
      "content": "<p>my max-recalls at-100</p>\n<pre><code>clicks recall = \ncarts recall = \norders recall = \n\noverall recall =\n</code></pre>\n<p>however the score comes out to be quite less, i am figuring out the issue.</p>\n<p>One question, Do your generated candidates have duplicate aids for 'session'? </p>",
      "rawMarkdown": "my max-recalls at-100\n```python\nclicks recall = 0.6375558661922811\ncarts recall = 0.5078653293550293\norders recall = 0.7076405718088917\n\noverall recall =0.64069952851\n```\nhowever the score comes out to be quite less, i am figuring out the issue.\n\nOne question, Do your generated candidates have duplicate aids for 'session'?",
      "votes": null
    },
    {
      "id": "2098757",
      "postDate": "01/13/2023 20:00:47",
      "content": "<p>same here , the more the  local recall get better to me , the more it  be worse in the LB</p>",
      "rawMarkdown": "same here , the more the  local recall get better to me , the more it  be worse in the LB",
      "votes": null
    },
    {
      "id": "2112767",
      "postDate": "01/23/2023 20:15:03",
      "content": "<p>our max-recall at -200 : </p>\n<blockquote>\n  <ul>\n  <li>0.726529 for orders</li>\n  <li>0.5448 for carts</li>\n  <li>0.689319 for clicks</li>\n  </ul>\n</blockquote>\n<hr>\n<p>Using some public features ~13 feat , our First try with GBT Ranker model was a failure with 0.3572 for carts .<br>\nWe are currently creating more significant features in an effort to obtain a good CV-LB. </p>\n<hr>\n<p>PS : <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/363497#2112761\" target=\"_blank\">Looking for another team mate with good Ranker Model here</a></p>",
      "rawMarkdown": "our max-recall at -200 : \n\n> * 0.726529 for orders\n> * 0.5448 for carts\n> * 0.689319 for clicks\n\n--- \nUsing some public features ~13 feat , our First try with GBT Ranker model was a failure with 0.3572 for carts .\nWe are currently creating more significant features in an effort to obtain a good CV-LB. \n\n\n---\nPS : [Looking for another team mate with good Ranker Model here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363497#2112761)",
      "votes": null
    },
    {
      "id": "2113097",
      "postDate": "01/24/2023 04:42:42",
      "content": "<p>Did you figure out what was the problem? <a href=\"https://www.kaggle.com/chaudharypriyanshu\" target=\"_blank\">@chaudharypriyanshu</a> </p>",
      "rawMarkdown": "Did you figure out what was the problem? @chaudharypriyanshu",
      "votes": null
    },
    {
      "id": "2113299",
      "postDate": "01/24/2023 07:55:18",
      "content": "<p>Nope. I am still trying to figure out what could be the reason. <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a> </p>",
      "rawMarkdown": "Nope. I am still trying to figure out what could be the reason. @gunesevitan",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2091683,
      "author_name": "hookman",
      "author_url": "",
      "post_date": "01/08/2023 16:47:25",
      "content": "<pre><code>hitrate@20\n{'clicks': 0.5210141483103858,\n'carts': 0.40768774721044854,\n'orders': 0.6506683121382453,\n'total': 0.5648087262771203}\n\nhitrate@100\n\n{'clicks': 0.6499337643160575,\n'carts': 0.5045632726017821,\n'orders': 0.7074081572967987,\n'total': 0.6408072525902196}\n</code></pre>\n<p>May I know what's your public LB using only recall? I got 0.574 by only using recall.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2091692,
          "author_name": "nlztrk",
          "author_url": "",
          "post_date": "01/08/2023 16:53:57",
          "content": "<p>Only candidate sampling (for Radek's validation):</p>\n<pre><code>Local Val: 0.567\nLB: 0.577\n</code></pre>",
          "votes": null,
          "replies": [
            {
              "id": 2091698,
              "author_name": "hookman",
              "author_url": "",
              "post_date": "01/08/2023 16:55:59",
              "content": "<p>Good to know. Thanks.</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 2091695,
          "author_name": "karakasatarik",
          "author_url": "",
          "post_date": "01/08/2023 16:54:54",
          "content": "<p>Using only recall, we got <code>0.577</code> which is same with public notebook <a href=\"https://www.kaggle.com/code/utm529fg/otto-tuning-candidate-rerank-model-lb-0-577\" target=\"_blank\">here</a>. With re-rank model, our public LB score is <code>0.587</code>.</p>",
          "votes": null,
          "replies": [
            {
              "id": 2091699,
              "author_name": "hookman",
              "author_url": "",
              "post_date": "01/08/2023 16:56:45",
              "content": "<p>Got it. Thanks.</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 2091992,
              "author_name": "radek1",
              "author_url": "",
              "post_date": "01/08/2023 23:31:34",
              "content": "<p>very valuable piece of info, <a href=\"https://www.kaggle.com/karakasatarik\" target=\"_blank\">@karakasatarik</a>! Thx for sharing this!!!!  </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2091991,
      "author_name": "radek1",
      "author_url": "",
      "post_date": "01/08/2023 23:31:08",
      "content": "<p>Hey <a href=\"https://www.kaggle.com/nlztrk\" target=\"_blank\">@nlztrk</a>! Thanks you for the shoutout 🤗 Very glad to see my work is of help 🙏</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2092754,
      "author_name": "kazama28",
      "author_url": "",
      "post_date": "01/09/2023 15:12:52",
      "content": "<p>Thank you for sharing the info! Can you also share if you are downsampling the negatives during training? If yes, by how much?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2092844,
          "author_name": "nlztrk",
          "author_url": "",
          "post_date": "01/09/2023 16:44:37",
          "content": "<p>Currently <strong>15%</strong>.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2094629,
      "author_name": "rayanaay",
      "author_url": "",
      "post_date": "01/10/2023 22:21:03",
      "content": "<p>Great ! did you use the same variables and parameters to train your ranker regarding the type ? ( clicks orders carts … )</p>",
      "votes": null,
      "replies": [
        {
          "id": 2095397,
          "author_name": "nlztrk",
          "author_url": "",
          "post_date": "01/11/2023 10:26:12",
          "content": "<p>Yep, our feature set is fixed for all of our models for now.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2098396,
      "author_name": "chaudharypriyanshu",
      "author_url": "",
      "post_date": "01/13/2023 14:04:35",
      "content": "<p>my max-recalls at-100</p>\n<pre><code>clicks recall = \ncarts recall = \norders recall = \n\noverall recall =\n</code></pre>\n<p>however the score comes out to be quite less, i am figuring out the issue.</p>\n<p>One question, Do your generated candidates have duplicate aids for 'session'? </p>",
      "votes": null,
      "replies": [
        {
          "id": 2098757,
          "author_name": "iraqai",
          "author_url": "",
          "post_date": "01/13/2023 20:00:47",
          "content": "<p>same here , the more the  local recall get better to me , the more it  be worse in the LB</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2113097,
          "author_name": "gunesevitan",
          "author_url": "",
          "post_date": "01/24/2023 04:42:42",
          "content": "<p>Did you figure out what was the problem? <a href=\"https://www.kaggle.com/chaudharypriyanshu\" target=\"_blank\">@chaudharypriyanshu</a> </p>",
          "votes": null,
          "replies": [
            {
              "id": 2113299,
              "author_name": "chaudharypriyanshu",
              "author_url": "",
              "post_date": "01/24/2023 07:55:18",
              "content": "<p>Nope. I am still trying to figure out what could be the reason. <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a> </p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2112767,
      "author_name": "ksouriazer",
      "author_url": "",
      "post_date": "01/23/2023 20:15:03",
      "content": "<p>our max-recall at -200 : </p>\n<blockquote>\n  <ul>\n  <li>0.726529 for orders</li>\n  <li>0.5448 for carts</li>\n  <li>0.689319 for clicks</li>\n  </ul>\n</blockquote>\n<hr>\n<p>Using some public features ~13 feat , our First try with GBT Ranker model was a failure with 0.3572 for carts .<br>\nWe are currently creating more significant features in an effort to obtain a good CV-LB. </p>\n<hr>\n<p>PS : <a href=\"https://www.kaggle.com/competitions/otto-recommender-system/discussion/363497#2112761\" target=\"_blank\">Looking for another team mate with good Ranker Model here</a></p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2091582": "We are currently generating ***100*** candidates and our ***max-recalls at-100*** as follows:\n\n```\nclicks recall = 0.6515316707053238\ncarts recall = 0.5054104031001835\norders recall = 0.7048352553279094\n\noverall recall = 0.6396774411973329\n```\n\nAnd after using our re-rank model we get these scores ***at-20***:\n```\nclicks recall = 0.5340063432688679\ncarts recall = 0.4237135902195957\norders recall = 0.6600268258685532\n\noverall recall = 0.5765308069138975\n```\n\nThese are both calculated for validation splits of @radek1. I just wonder what is your recall difference and the number of candidates you use?",
    "2091683": "```\nhitrate@20\n{'clicks': 0.5210141483103858,\n'carts': 0.40768774721044854,\n'orders': 0.6506683121382453,\n'total': 0.5648087262771203}\n\nhitrate@100\n\n{'clicks': 0.6499337643160575,\n'carts': 0.5045632726017821,\n'orders': 0.7074081572967987,\n'total': 0.6408072525902196}\n```\n\nMay I know what's your public LB using only recall? I got 0.574 by only using recall.",
    "2091692": "Only candidate sampling (for Radek's validation):\n```\nLocal Val: 0.567\nLB: 0.577\n```",
    "2091695": "Using only recall, we got `0.577` which is same with public notebook [here](https://www.kaggle.com/code/utm529fg/otto-tuning-candidate-rerank-model-lb-0-577). With re-rank model, our public LB score is `0.587`.",
    "2091698": "Good to know. Thanks.",
    "2091699": "Got it. Thanks.",
    "2091991": "Hey @nlztrk! Thanks you for the shoutout 🤗 Very glad to see my work is of help 🙏",
    "2091992": "very valuable piece of info, @karakasatarik! Thx for sharing this!!!!",
    "2092754": "Thank you for sharing the info! Can you also share if you are downsampling the negatives during training? If yes, by how much?",
    "2092844": "Currently **15%**.",
    "2094629": "Great ! did you use the same variables and parameters to train your ranker regarding the type ? ( clicks orders carts ... )",
    "2095397": "Yep, our feature set is fixed for all of our models for now.",
    "2098396": "my max-recalls at-100\n```python\nclicks recall = 0.6375558661922811\ncarts recall = 0.5078653293550293\norders recall = 0.7076405718088917\n\noverall recall =0.64069952851\n```\nhowever the score comes out to be quite less, i am figuring out the issue.\n\nOne question, Do your generated candidates have duplicate aids for 'session'?",
    "2098757": "same here , the more the  local recall get better to me , the more it  be worse in the LB",
    "2112767": "our max-recall at -200 : \n\n> * 0.726529 for orders\n> * 0.5448 for carts\n> * 0.689319 for clicks\n\n--- \nUsing some public features ~13 feat , our First try with GBT Ranker model was a failure with 0.3572 for carts .\nWe are currently creating more significant features in an effort to obtain a good CV-LB. \n\n\n---\nPS : [Looking for another team mate with good Ranker Model here](https://www.kaggle.com/competitions/otto-recommender-system/discussion/363497#2112761)",
    "2113097": "Did you figure out what was the problem? @chaudharypriyanshu",
    "2113299": "Nope. I am still trying to figure out what could be the reason. @gunesevitan"
  },
  "source": "meta"
}