{
  "id": 415723,
  "title": "How to convert submission to new API?",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/415723",
  "author_name": "",
  "post_date": "2023-06-07T18:55:16.353909800Z",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I'm having an issue converting old notebooks to new API (I'm using python 3.7.12).  I ended up going from 0.703 to 0.514 in the latest update, using the old API, so that was awesome.  I tried updating the code, but it doesn't seem to come near the old values, and am asking for some help.  Old version</p>\n<p><code>preds = []</code><br>\n<code>a,b = limits[grp]</code><br>\n<code>for q in range(a, b):</code></p>\n<pre><code>    pred = model(val)\n    preds += \n</code></pre>\n<p><code>sample_submission.loc[sample_submission['session_id'].map(lambda s: int(s.split('_')[0]) == session_id),\"correct\"] = preds</code></p>\n<p>To a new version:</p>\n<p><code>a,b = limits[grp]</code><br>\n<code>for q in range(a, b):</code></p>\n<pre><code>    p = model[q].predict_proba(val)[,]\n    mask = sample_submission.session_id.str.contains(f)\n    sample_submission.loc[mask,] = int( p &gt; best_threshold )\n</code></pre>\n<p>This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).</p>\n<p>Any hints?  <a href=\"https://www.kaggle.com/rskata\" target=\"_blank\">@rskata</a>, <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a>?</p>",
  "messages": [
    {
      "id": "2291708",
      "postDate": "06/07/2023 18:55:16",
      "content": "<p>I'm having an issue converting old notebooks to new API (I'm using python 3.7.12).  I ended up going from 0.703 to 0.514 in the latest update, using the old API, so that was awesome.  I tried updating the code, but it doesn't seem to come near the old values, and am asking for some help.  Old version</p>\n<p><code>preds = []</code><br>\n<code>a,b = limits[grp]</code><br>\n<code>for q in range(a, b):</code></p>\n<pre><code>    pred = model(val)\n    preds += \n</code></pre>\n<p><code>sample_submission.loc[sample_submission['session_id'].map(lambda s: int(s.split('_')[0]) == session_id),\"correct\"] = preds</code></p>\n<p>To a new version:</p>\n<p><code>a,b = limits[grp]</code><br>\n<code>for q in range(a, b):</code></p>\n<pre><code>    p = model[q].predict_proba(val)[,]\n    mask = sample_submission.session_id.str.contains(f)\n    sample_submission.loc[mask,] = int( p &gt; best_threshold )\n</code></pre>\n<p>This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).</p>\n<p>Any hints?  <a href=\"https://www.kaggle.com/rskata\" target=\"_blank\">@rskata</a>, <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a>?</p>",
      "rawMarkdown": "I'm having an issue converting old notebooks to new API (I'm using python 3.7.12).  I ended up going from 0.703 to 0.514 in the latest update, using the old API, so that was awesome.  I tried updating the code, but it doesn't seem to come near the old values, and am asking for some help.  Old version\n\n`preds = []`\n`a,b = limits[grp]`\n`for q in range(a, b):`\n\n        pred = model[q].predict_proba(val)[:,1]\n        preds += [int(x > best_threshold) for x in pred]\n\n  `sample_submission.loc[sample_submission['session_id'].map(lambda s: int(s.split('_')[0]) == session_id),\"correct\"] = preds`\n\n\nTo a new version:\n\n`a,b = limits[grp]`\n`for q in range(a, b):`\n \n        p = model[q].predict_proba(val)[0,1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int( p > best_threshold )\n\nThis goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).\n\nAny hints?  @rskata, @cpmpml?",
      "votes": null
    },
    {
      "id": "2291721",
      "postDate": "06/07/2023 19:19:40",
      "content": "<p>For example, do we know what changed in the APIs?  Does the session data change somehow?</p>",
      "rawMarkdown": "For example, do we know what changed in the APIs?  Does the session data change somehow?",
      "votes": null
    },
    {
      "id": "2291726",
      "postDate": "06/07/2023 19:25:27",
      "content": "<p>You might need to sort by index, see the discussion  <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512\" target=\"_blank\">here</a></p>",
      "rawMarkdown": "You might need to sort by index, see the discussion  [here](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512)",
      "votes": null
    },
    {
      "id": "2291728",
      "postDate": "06/07/2023 19:26:40",
      "content": "<p>For me atleast this is a problem with python 3.7 aswell</p>",
      "rawMarkdown": "For me atleast this is a problem with python 3.7 aswell",
      "votes": null
    },
    {
      "id": "2291736",
      "postDate": "06/07/2023 19:34:18",
      "content": "<p>Thanks, I've added test.sort_index(inplace=True) and tried a submit.</p>",
      "rawMarkdown": "Thanks, I've added test.sort_index(inplace=True) and tried a submit.",
      "votes": null
    },
    {
      "id": "2292021",
      "postDate": "06/08/2023 03:13:16",
      "content": "<blockquote>\n  <p>This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).<br>\n  Sorry.. I don't know why this change made your score go up…the pred values are never changed …</p>\n</blockquote>",
      "rawMarkdown": "> This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).\nSorry.. I don't know why this change made your score go up...the pred values are never changed ...",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2291721,
      "author_name": "danielphalen",
      "author_url": "",
      "post_date": "06/07/2023 19:19:40",
      "content": "<p>For example, do we know what changed in the APIs?  Does the session data change somehow?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2291726,
      "author_name": "erijoel",
      "author_url": "",
      "post_date": "06/07/2023 19:25:27",
      "content": "<p>You might need to sort by index, see the discussion  <a href=\"https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512\" target=\"_blank\">here</a></p>",
      "votes": null,
      "replies": [
        {
          "id": 2291728,
          "author_name": "erijoel",
          "author_url": "",
          "post_date": "06/07/2023 19:26:40",
          "content": "<p>For me atleast this is a problem with python 3.7 aswell</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2291736,
          "author_name": "danielphalen",
          "author_url": "",
          "post_date": "06/07/2023 19:34:18",
          "content": "<p>Thanks, I've added test.sort_index(inplace=True) and tried a submit.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2292021,
      "author_name": "econjt",
      "author_url": "",
      "post_date": "06/08/2023 03:13:16",
      "content": "<blockquote>\n  <p>This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).<br>\n  Sorry.. I don't know why this change made your score go up…the pred values are never changed …</p>\n</blockquote>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2291708": "I'm having an issue converting old notebooks to new API (I'm using python 3.7.12).  I ended up going from 0.703 to 0.514 in the latest update, using the old API, so that was awesome.  I tried updating the code, but it doesn't seem to come near the old values, and am asking for some help.  Old version\n\n`preds = []`\n`a,b = limits[grp]`\n`for q in range(a, b):`\n\n        pred = model[q].predict_proba(val)[:,1]\n        preds += [int(x > best_threshold) for x in pred]\n\n  `sample_submission.loc[sample_submission['session_id'].map(lambda s: int(s.split('_')[0]) == session_id),\"correct\"] = preds`\n\n\nTo a new version:\n\n`a,b = limits[grp]`\n`for q in range(a, b):`\n \n        p = model[q].predict_proba(val)[0,1]\n        mask = sample_submission.session_id.str.contains(f'q{q}')\n        sample_submission.loc[mask,'correct'] = int( p > best_threshold )\n\nThis goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).\n\nAny hints?  @rskata, @cpmpml?",
    "2291721": "For example, do we know what changed in the APIs?  Does the session data change somehow?",
    "2291726": "You might need to sort by index, see the discussion  [here](https://www.kaggle.com/competitions/predict-student-performance-from-game-play/discussion/412512)",
    "2291728": "For me atleast this is a problem with python 3.7 aswell",
    "2291736": "Thanks, I've added test.sort_index(inplace=True) and tried a submit.",
    "2292021": "> This goes from 0.514 to 0.616, but that is obviously not anywhere near the old score value (or the CV value).\nSorry.. I don't know why this change made your score go up...the pred values are never changed ..."
  },
  "source": "meta"
}