{
  "id": 451670,
  "title": "Incremental learning in XGBoost  {'tree_method': 'exact'}",
  "url": "/competitions/open-problems-single-cell-perturbations/discussion/451670",
  "author_name": "",
  "post_date": "2023-10-30T03:10:47.286000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Incremental learning in XGBoost - work good.</p>\n<p>I use embedding for  SMILES:  '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'<br>\nand embedding for  180 000  columns:      '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'</p>\n<p>11 181 554 rows divide to 50 parts,<br>\nand then incremental learning XGBoost:</p>\n<p>iterations = 25<br>\nmodel = None<br>\nfor j in range(iterations):<br>\n    for i in Range(50):<br>\n        X_train_split, X_valid_split, y_train_split, y_valid_split,    Yr_train, Yr_valid, Yr_valid_real    = get_dataset(i = i)</p>\n<pre><code>    refresh_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n    model = xgb.train(\n        {\n        : , \n        : ,\n        : ,  \n        : },\n        dtrain=xgb.DMatrix(X_train_split, Yr_train), \n        xgb_model=model,\n        evals=[(xgb.DMatrix(X_valid_split, Yr_valid), )],\n        evals_result=refresh_result\n    )\n\n    xg_valid = xgb.DMatrix(X_valid_split, Yr_valid)\n    Yr_pred = model.predict(xg_valid)  \n\n    r2 = r2_score(Yr_valid,  Yr_pred )\n    (, r2)\n\n     log_flag:\n      target_pred=     (np.exp((Yr_pred) )-)/   *Yr_pred/(Yr_pred)\n    : \n      target_pred=  Yr_pred \n\n    s = mean_squared_error(Yr_valid_real, target_pred, squared = )\n    ((s,))\n    file_name = \n    pickle.dump(model, (file_name,))\n    (file_name)\n</code></pre>\n<p>notebook( <a href=\"https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em\" target=\"_blank\">https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em</a>   version 51)</p>",
  "messages": [
    {
      "id": 2504585,
      "postDate": "2023-10-30T03:10:47.287Z",
      "content": "<p>Incremental learning in XGBoost - work good.</p>\n<p>I use embedding for  SMILES:  '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'<br>\nand embedding for  180 000  columns:      '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'</p>\n<p>11 181 554 rows divide to 50 parts,<br>\nand then incremental learning XGBoost:</p>\n<p>iterations = 25<br>\nmodel = None<br>\nfor j in range(iterations):<br>\n    for i in Range(50):<br>\n        X_train_split, X_valid_split, y_train_split, y_valid_split,    Yr_train, Yr_valid, Yr_valid_real    = get_dataset(i = i)</p>\n<pre><code>    refresh_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n    model = xgb.train(\n        {\n        : , \n        : ,\n        : ,  \n        : },\n        dtrain=xgb.DMatrix(X_train_split, Yr_train), \n        xgb_model=model,\n        evals=[(xgb.DMatrix(X_valid_split, Yr_valid), )],\n        evals_result=refresh_result\n    )\n\n    xg_valid = xgb.DMatrix(X_valid_split, Yr_valid)\n    Yr_pred = model.predict(xg_valid)  \n\n    r2 = r2_score(Yr_valid,  Yr_pred )\n    (, r2)\n\n     log_flag:\n      target_pred=     (np.exp((Yr_pred) )-)/   *Yr_pred/(Yr_pred)\n    : \n      target_pred=  Yr_pred \n\n    s = mean_squared_error(Yr_valid_real, target_pred, squared = )\n    ((s,))\n    file_name = \n    pickle.dump(model, (file_name,))\n    (file_name)\n</code></pre>\n<p>notebook( <a href=\"https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em\" target=\"_blank\">https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em</a>   version 51)</p>",
      "rawMarkdown": "Incremental learning in XGBoost - work good.\n\nI use embedding for  SMILES:  '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'\nand embedding for  180 000  columns:      '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'\n \n11 181 554 rows divide to 50 parts,\nand then incremental learning XGBoost:\n\n\niterations = 25\nmodel = None\nfor j in range(iterations):\n    for i in Range(50):\n        X_train_split, X_valid_split, y_train_split, y_valid_split,    Yr_train, Yr_valid, Yr_valid_real    = get_dataset(i = i)\n\n        refresh_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n        model = xgb.train(\n            {\n            'tree_method': 'exact', \n            'refresh_leaf': True,\n            'reg_alpha': 3,  # L1\n            'silent': False},\n            dtrain=xgb.DMatrix(X_train_split, Yr_train), \n            xgb_model=model,\n            evals=[(xgb.DMatrix(X_valid_split, Yr_valid), \"Original\")],\n            evals_result=refresh_result\n        )\n\n        xg_valid = xgb.DMatrix(X_valid_split, Yr_valid)\n        Yr_pred = model.predict(xg_valid)  # \"after\"\n\n        r2 = r2_score(Yr_valid,  Yr_pred )\n        print('r2 valid', r2)\n    \n        if log_flag:\n          target_pred=     (np.exp(abs(Yr_pred) )-1)/192000000   *Yr_pred/abs(Yr_pred)\n        else: \n          target_pred=  Yr_pred \n\n        s = mean_squared_error(Yr_valid_real, target_pred, squared = False)\n        print(round(s,6))\n        file_name = f's{round(s,3)}_r2{round(r2,3)}.cmb'\n        pickle.dump(model, open(file_name,'wb'))\n        print(file_name)\n\nnotebook( https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em   version 51)\n\n\n",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2504585": "Incremental learning in XGBoost - work good.\n\nI use embedding for  SMILES:  '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'\nand embedding for  180 000  columns:      '/kaggle/input/op2-embed-protein-6-union/protein_embeding.csv'\n \n11 181 554 rows divide to 50 parts,\nand then incremental learning XGBoost:\n\n\niterations = 25\nmodel = None\nfor j in range(iterations):\n    for i in Range(50):\n        X_train_split, X_valid_split, y_train_split, y_valid_split,    Yr_train, Yr_valid, Yr_valid_real    = get_dataset(i = i)\n\n        refresh_result: xgb.callback.EvaluationMonitor.EvalsLog = {}\n        model = xgb.train(\n            {\n            'tree_method': 'exact', \n            'refresh_leaf': True,\n            'reg_alpha': 3,  # L1\n            'silent': False},\n            dtrain=xgb.DMatrix(X_train_split, Yr_train), \n            xgb_model=model,\n            evals=[(xgb.DMatrix(X_valid_split, Yr_valid), \"Original\")],\n            evals_result=refresh_result\n        )\n\n        xg_valid = xgb.DMatrix(X_valid_split, Yr_valid)\n        Yr_pred = model.predict(xg_valid)  # \"after\"\n\n        r2 = r2_score(Yr_valid,  Yr_pred )\n        print('r2 valid', r2)\n    \n        if log_flag:\n          target_pred=     (np.exp(abs(Yr_pred) )-1)/192000000   *Yr_pred/abs(Yr_pred)\n        else: \n          target_pred=  Yr_pred \n\n        s = mean_squared_error(Yr_valid_real, target_pred, squared = False)\n        print(round(s,6))\n        file_name = f's{round(s,3)}_r2{round(r2,3)}.cmb'\n        pickle.dump(model, open(file_name,'wb'))\n        print(file_name)\n\nnotebook( https://www.kaggle.com/code/liudacheldieva/2-op2-eda-linearsvr-regressor-nn-em   version 51)\n\n\n"
  }
}