{
  "id": 375581,
  "title": "How many rounds/epochs do you train ?",
  "url": "/competitions/otto-recommender-system/discussion/375581",
  "author_name": "",
  "post_date": "2023-01-02T11:48:08.052208300Z",
  "votes": 15,
  "comment_count": 23,
  "views": 0,
  "content": "<p>In this competition, we can use huge dataset for training.</p>\n<p>Do you use all dataset for training ?<br>\nIf so, how many rounds/epochs do you train your model ?</p>\n<p>I currently train my model by only one epoch (It takes 1 day).</p>",
  "messages": [
    {
      "id": "2083288",
      "postDate": "01/02/2023 11:48:08",
      "content": "<p>In this competition, we can use huge dataset for training.</p>\n<p>Do you use all dataset for training ?<br>\nIf so, how many rounds/epochs do you train your model ?</p>\n<p>I currently train my model by only one epoch (It takes 1 day).</p>",
      "rawMarkdown": "In this competition, we can use huge dataset for training.\n\nDo you use all dataset for training ?\nIf so, how many rounds/epochs do you train your model ?\n\nI currently train my model by only one epoch (It takes 1 day).",
      "votes": null
    },
    {
      "id": "2083296",
      "postDate": "01/02/2023 11:50:36",
      "content": "<p>my training takes only 30 minutes with a 16 core machine for lightgbm</p>",
      "rawMarkdown": "my training takes only 30 minutes with a 16 core machine for lightgbm",
      "votes": null
    },
    {
      "id": "2083302",
      "postDate": "01/02/2023 11:57:33",
      "content": "<p>Very Fast ! It must be a smart solution !</p>",
      "rawMarkdown": "Very Fast ! It must be a smart solution !",
      "votes": null
    },
    {
      "id": "2083304",
      "postDate": "01/02/2023 12:00:13",
      "content": "<p>hehe you're very brave to wait for 1 day to complete one epoch, my patience is very less</p>",
      "rawMarkdown": "hehe you're very brave to wait for 1 day to complete one epoch, my patience is very less",
      "votes": null
    },
    {
      "id": "2083322",
      "postDate": "01/02/2023 12:17:48",
      "content": "<p>My Ranking model(GBDT) training takes about an hour.</p>",
      "rawMarkdown": "My Ranking model(GBDT) training takes about an hour.",
      "votes": null
    },
    {
      "id": "2083385",
      "postDate": "01/02/2023 13:12:00",
      "content": "<p>i use around 500K sessions for training with around 300 candidates each, data prep + training runs around 1 hour on 16 cores with lightGBM model.</p>",
      "rawMarkdown": "i use around 500K sessions for training with around 300 candidates each, data prep + training runs around 1 hour on 16 cores with lightGBM model.",
      "votes": null
    },
    {
      "id": "2083397",
      "postDate": "01/02/2023 13:26:00",
      "content": "<ul>\n<li>sessions: 2,000K  </li>\n<li>candidates: 100  </li>\n<li>model: Catboost  </li>\n</ul>",
      "rawMarkdown": "sessions: 2,000K  \n- candidates: 100  \n- model: Catboost",
      "votes": null
    },
    {
      "id": "2083435",
      "postDate": "01/02/2023 14:13:18",
      "content": "<p>Thank you for information !<br>\nThe number of candidates is less than I imagined.<br>\nI suppose you can boost your score by increasing the number of candidates.</p>",
      "rawMarkdown": "Thank you for information !\nThe number of candidates is less than I imagined.\nI suppose you can boost your score by increasing the number of candidates.",
      "votes": null
    },
    {
      "id": "2083438",
      "postDate": "01/02/2023 14:15:04",
      "content": "<p>Amazing ! Your date prep and training is really efficient.</p>",
      "rawMarkdown": "Amazing ! Your date prep and training is really efficient.",
      "votes": null
    },
    {
      "id": "2083444",
      "postDate": "01/02/2023 14:16:16",
      "content": "<p>Thank you for your compliment, but it turns out I'm working on a different game 😂</p>",
      "rawMarkdown": "Thank you for your compliment, but it turns out I'm working on a different game 😂",
      "votes": null
    },
    {
      "id": "2083466",
      "postDate": "01/02/2023 14:39:54",
      "content": "<p>can i ask you how much does your ranker model boost your cv score from the candidates generation?</p>",
      "rawMarkdown": "can i ask you how much does your ranker model boost your cv score from the candidates generation?",
      "votes": null
    },
    {
      "id": "2083783",
      "postDate": "01/02/2023 20:24:04",
      "content": "<p>It is about 0.025.</p>",
      "rawMarkdown": "It is about 0.025.",
      "votes": null
    },
    {
      "id": "2083860",
      "postDate": "01/02/2023 23:48:32",
      "content": "<p>super interesting question, thanks for asking it <a href=\"https://www.kaggle.com/toshik\" target=\"_blank\">@toshik</a>! some answers are really fascinating! 🙂</p>",
      "rawMarkdown": "super interesting question, thanks for asking it @toshik! some answers are really fascinating! 🙂",
      "votes": null
    },
    {
      "id": "2083864",
      "postDate": "01/02/2023 23:49:57",
      "content": "<p>thank you for the answer, <a href=\"https://www.kaggle.com/ymatioun\" target=\"_blank\">@ymatioun</a>! Given that you are 21st the fact that you are training on 500k sessions is really cool 🙂</p>\n<p>Kind of reinforces the notion that especially early on jumping to training on the full dataset is a not a good idea (makes for slower iteration end experimentation)</p>\n<p>Thanks for your answer! 🙂</p>",
      "rawMarkdown": "thank you for the answer, @ymatioun! Given that you are 21st the fact that you are training on 500k sessions is really cool 🙂\n\nKind of reinforces the notion that especially early on jumping to training on the full dataset is a not a good idea (makes for slower iteration end experimentation)\n\nThanks for your answer! 🙂",
      "votes": null
    },
    {
      "id": "2083867",
      "postDate": "01/02/2023 23:50:35",
      "content": "<p>kudos <a href=\"https://www.kaggle.com/nikhilmishradev\" target=\"_blank\">@nikhilmishradev</a>! You seem to be rocking those tabular data competitions! 😁 Best of luck in this one! </p>",
      "rawMarkdown": "kudos @nikhilmishradev! You seem to be rocking those tabular data competitions! 😁 Best of luck in this one!",
      "votes": null
    },
    {
      "id": "2083871",
      "postDate": "01/02/2023 23:51:38",
      "content": "<p>100 candidates <a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a> omg that is amazing! 😁 this thread delivers 🔥 and plus using catboost, wow!</p>\n<p>out of curiosity, is there a reason you went for catboost?</p>\n<p>congrats for the 11th place right now and all the best in the competition! 🙂</p>",
      "rawMarkdown": "100 candidates @dehokanta omg that is amazing! 😁 this thread delivers 🔥 and plus using catboost, wow!\n\nout of curiosity, is there a reason you went for catboost?\n\ncongrats for the 11th place right now and all the best in the competition! 🙂",
      "votes": null
    },
    {
      "id": "2083873",
      "postDate": "01/02/2023 23:52:42",
      "content": "<p>these are some stellar candidates then! 🙂  but it sort of makes sense, just with the co-visitation matrix approach I think yo can get to 0.577?</p>",
      "rawMarkdown": "these are some stellar candidates then! 🙂  but it sort of makes sense, just with the co-visitation matrix approach I think yo can get to 0.577?",
      "votes": null
    },
    {
      "id": "2083897",
      "postDate": "01/03/2023 00:25:12",
      "content": "<blockquote>\n  <p>is there a reason you went for catboost? </p>\n</blockquote>\n<p>This is because of high prediction speed.</p>",
      "rawMarkdown": ">is there a reason you went for catboost? \n\nThis is because of high prediction speed.",
      "votes": null
    },
    {
      "id": "2083935",
      "postDate": "01/03/2023 01:32:23",
      "content": "<p>Got it, thank you <a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a>! Appreciate the reply 🙏 What a wonderful environment for learning this is thanks to threads like this one and replies like yours! 🙂</p>",
      "rawMarkdown": "Got it, thank you @dehokanta! Appreciate the reply 🙏 What a wonderful environment for learning this is thanks to threads like this one and replies like yours! 🙂",
      "votes": null
    },
    {
      "id": "2084032",
      "postDate": "01/03/2023 04:45:25",
      "content": "<p>Thank you, Radek.<br>\nIt seems that my intuitions shed light on something important.</p>",
      "rawMarkdown": "Thank you, Radek.\nIt seems that my intuitions shed light on something important.",
      "votes": null
    },
    {
      "id": "2088658",
      "postDate": "01/06/2023 14:25:23",
      "content": "<p><a href=\"https://www.kaggle.com/nikhilmishradev\" target=\"_blank\">@nikhilmishradev</a> you compete and win prizes on all platforms, please take some rest/vacation to give others a chance to win prizes 😢</p>",
      "rawMarkdown": "nikhilmishradev you compete and win prizes on all platforms, please take some rest/vacation to give others a chance to win prizes 😢",
      "votes": null
    },
    {
      "id": "2088672",
      "postDate": "01/06/2023 14:39:14",
      "content": "<p>I am almost dead in this comp, breathing my last</p>",
      "rawMarkdown": "I am almost dead in this comp, breathing my last",
      "votes": null
    },
    {
      "id": "2088674",
      "postDate": "01/06/2023 14:43:58",
      "content": "<p>Good Luck for solo gold 🙏</p>",
      "rawMarkdown": "Good Luck for solo gold 🙏",
      "votes": null
    },
    {
      "id": "2088676",
      "postDate": "01/06/2023 14:46:31",
      "content": "<p>All the best !</p>",
      "rawMarkdown": "All the best !",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2083296,
      "author_name": "nikhilmishradev",
      "author_url": "",
      "post_date": "01/02/2023 11:50:36",
      "content": "<p>my training takes only 30 minutes with a 16 core machine for lightgbm</p>",
      "votes": null,
      "replies": [
        {
          "id": 2083302,
          "author_name": "toshik",
          "author_url": "",
          "post_date": "01/02/2023 11:57:33",
          "content": "<p>Very Fast ! It must be a smart solution !</p>",
          "votes": null,
          "replies": [
            {
              "id": 2083304,
              "author_name": "nikhilmishradev",
              "author_url": "",
              "post_date": "01/02/2023 12:00:13",
              "content": "<p>hehe you're very brave to wait for 1 day to complete one epoch, my patience is very less</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2083444,
                  "author_name": "toshik",
                  "author_url": "",
                  "post_date": "01/02/2023 14:16:16",
                  "content": "<p>Thank you for your compliment, but it turns out I'm working on a different game 😂</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        },
        {
          "id": 2083867,
          "author_name": "radek1",
          "author_url": "",
          "post_date": "01/02/2023 23:50:35",
          "content": "<p>kudos <a href=\"https://www.kaggle.com/nikhilmishradev\" target=\"_blank\">@nikhilmishradev</a>! You seem to be rocking those tabular data competitions! 😁 Best of luck in this one! </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2088658,
          "author_name": "init27",
          "author_url": "",
          "post_date": "01/06/2023 14:25:23",
          "content": "<p><a href=\"https://www.kaggle.com/nikhilmishradev\" target=\"_blank\">@nikhilmishradev</a> you compete and win prizes on all platforms, please take some rest/vacation to give others a chance to win prizes 😢</p>",
          "votes": null,
          "replies": [
            {
              "id": 2088672,
              "author_name": "nikhilmishradev",
              "author_url": "",
              "post_date": "01/06/2023 14:39:14",
              "content": "<p>I am almost dead in this comp, breathing my last</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2088674,
                  "author_name": "init27",
                  "author_url": "",
                  "post_date": "01/06/2023 14:43:58",
                  "content": "<p>Good Luck for solo gold 🙏</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2088676,
                      "author_name": "nikhilmishradev",
                      "author_url": "",
                      "post_date": "01/06/2023 14:46:31",
                      "content": "<p>All the best !</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2083322,
      "author_name": "dehokanta",
      "author_url": "",
      "post_date": "01/02/2023 12:17:48",
      "content": "<p>My Ranking model(GBDT) training takes about an hour.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2083397,
          "author_name": "dehokanta",
          "author_url": "",
          "post_date": "01/02/2023 13:26:00",
          "content": "<ul>\n<li>sessions: 2,000K  </li>\n<li>candidates: 100  </li>\n<li>model: Catboost  </li>\n</ul>",
          "votes": null,
          "replies": [
            {
              "id": 2083435,
              "author_name": "toshik",
              "author_url": "",
              "post_date": "01/02/2023 14:13:18",
              "content": "<p>Thank you for information !<br>\nThe number of candidates is less than I imagined.<br>\nI suppose you can boost your score by increasing the number of candidates.</p>",
              "votes": null,
              "replies": []
            },
            {
              "id": 2083466,
              "author_name": "enric1296",
              "author_url": "",
              "post_date": "01/02/2023 14:39:54",
              "content": "<p>can i ask you how much does your ranker model boost your cv score from the candidates generation?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2083783,
                  "author_name": "dehokanta",
                  "author_url": "",
                  "post_date": "01/02/2023 20:24:04",
                  "content": "<p>It is about 0.025.</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2083873,
                      "author_name": "radek1",
                      "author_url": "",
                      "post_date": "01/02/2023 23:52:42",
                      "content": "<p>these are some stellar candidates then! 🙂  but it sort of makes sense, just with the co-visitation matrix approach I think yo can get to 0.577?</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            },
            {
              "id": 2083871,
              "author_name": "radek1",
              "author_url": "",
              "post_date": "01/02/2023 23:51:38",
              "content": "<p>100 candidates <a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a> omg that is amazing! 😁 this thread delivers 🔥 and plus using catboost, wow!</p>\n<p>out of curiosity, is there a reason you went for catboost?</p>\n<p>congrats for the 11th place right now and all the best in the competition! 🙂</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2083897,
                  "author_name": "dehokanta",
                  "author_url": "",
                  "post_date": "01/03/2023 00:25:12",
                  "content": "<blockquote>\n  <p>is there a reason you went for catboost? </p>\n</blockquote>\n<p>This is because of high prediction speed.</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2083935,
                      "author_name": "radek1",
                      "author_url": "",
                      "post_date": "01/03/2023 01:32:23",
                      "content": "<p>Got it, thank you <a href=\"https://www.kaggle.com/dehokanta\" target=\"_blank\">@dehokanta</a>! Appreciate the reply 🙏 What a wonderful environment for learning this is thanks to threads like this one and replies like yours! 🙂</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        }
      ]
    },
    {
      "id": 2083385,
      "author_name": "ymatioun",
      "author_url": "",
      "post_date": "01/02/2023 13:12:00",
      "content": "<p>i use around 500K sessions for training with around 300 candidates each, data prep + training runs around 1 hour on 16 cores with lightGBM model.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2083438,
          "author_name": "toshik",
          "author_url": "",
          "post_date": "01/02/2023 14:15:04",
          "content": "<p>Amazing ! Your date prep and training is really efficient.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2083864,
          "author_name": "radek1",
          "author_url": "",
          "post_date": "01/02/2023 23:49:57",
          "content": "<p>thank you for the answer, <a href=\"https://www.kaggle.com/ymatioun\" target=\"_blank\">@ymatioun</a>! Given that you are 21st the fact that you are training on 500k sessions is really cool 🙂</p>\n<p>Kind of reinforces the notion that especially early on jumping to training on the full dataset is a not a good idea (makes for slower iteration end experimentation)</p>\n<p>Thanks for your answer! 🙂</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2083860,
      "author_name": "radek1",
      "author_url": "",
      "post_date": "01/02/2023 23:48:32",
      "content": "<p>super interesting question, thanks for asking it <a href=\"https://www.kaggle.com/toshik\" target=\"_blank\">@toshik</a>! some answers are really fascinating! 🙂</p>",
      "votes": null,
      "replies": [
        {
          "id": 2084032,
          "author_name": "toshik",
          "author_url": "",
          "post_date": "01/03/2023 04:45:25",
          "content": "<p>Thank you, Radek.<br>\nIt seems that my intuitions shed light on something important.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2083288": "In this competition, we can use huge dataset for training.\n\nDo you use all dataset for training ?\nIf so, how many rounds/epochs do you train your model ?\n\nI currently train my model by only one epoch (It takes 1 day).",
    "2083296": "my training takes only 30 minutes with a 16 core machine for lightgbm",
    "2083302": "Very Fast ! It must be a smart solution !",
    "2083304": "hehe you're very brave to wait for 1 day to complete one epoch, my patience is very less",
    "2083322": "My Ranking model(GBDT) training takes about an hour.",
    "2083385": "i use around 500K sessions for training with around 300 candidates each, data prep + training runs around 1 hour on 16 cores with lightGBM model.",
    "2083397": "sessions: 2,000K  \n- candidates: 100  \n- model: Catboost",
    "2083435": "Thank you for information !\nThe number of candidates is less than I imagined.\nI suppose you can boost your score by increasing the number of candidates.",
    "2083438": "Amazing ! Your date prep and training is really efficient.",
    "2083444": "Thank you for your compliment, but it turns out I'm working on a different game 😂",
    "2083466": "can i ask you how much does your ranker model boost your cv score from the candidates generation?",
    "2083783": "It is about 0.025.",
    "2083860": "super interesting question, thanks for asking it @toshik! some answers are really fascinating! 🙂",
    "2083864": "thank you for the answer, @ymatioun! Given that you are 21st the fact that you are training on 500k sessions is really cool 🙂\n\nKind of reinforces the notion that especially early on jumping to training on the full dataset is a not a good idea (makes for slower iteration end experimentation)\n\nThanks for your answer! 🙂",
    "2083867": "kudos @nikhilmishradev! You seem to be rocking those tabular data competitions! 😁 Best of luck in this one!",
    "2083871": "100 candidates @dehokanta omg that is amazing! 😁 this thread delivers 🔥 and plus using catboost, wow!\n\nout of curiosity, is there a reason you went for catboost?\n\ncongrats for the 11th place right now and all the best in the competition! 🙂",
    "2083873": "these are some stellar candidates then! 🙂  but it sort of makes sense, just with the co-visitation matrix approach I think yo can get to 0.577?",
    "2083897": ">is there a reason you went for catboost? \n\nThis is because of high prediction speed.",
    "2083935": "Got it, thank you @dehokanta! Appreciate the reply 🙏 What a wonderful environment for learning this is thanks to threads like this one and replies like yours! 🙂",
    "2084032": "Thank you, Radek.\nIt seems that my intuitions shed light on something important.",
    "2088658": "nikhilmishradev you compete and win prizes on all platforms, please take some rest/vacation to give others a chance to win prizes 😢",
    "2088672": "I am almost dead in this comp, breathing my last",
    "2088674": "Good Luck for solo gold 🙏",
    "2088676": "All the best !"
  },
  "source": "meta"
}