{
  "id": 499387,
  "title": "submission scoring error",
  "url": "/competitions/home-credit-credit-risk-model-stability/discussion/499387",
  "author_name": "",
  "post_date": "2024-05-01T15:50:52.039678500Z",
  "votes": 2,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Although my notebook runs successfully, I encounter a 'submission scoring error' and my work isn't graded. Can anyone guess why this might be happening?</p>",
  "messages": [
    {
      "id": "2787141",
      "postDate": "05/01/2024 15:50:52",
      "content": "<p>Although my notebook runs successfully, I encounter a 'submission scoring error' and my work isn't graded. Can anyone guess why this might be happening?</p>",
      "rawMarkdown": "Although my notebook runs successfully, I encounter a 'submission scoring error' and my work isn't graded. Can anyone guess why this might be happening?",
      "votes": null
    },
    {
      "id": "2790370",
      "postDate": "05/03/2024 06:53:04",
      "content": "<p>You don't even make the code public, how can we ask everyone to help you check it?</p>",
      "rawMarkdown": "You don't even make the code public, how can we ask everyone to help you check it?",
      "votes": null
    },
    {
      "id": "2794444",
      "postDate": "05/05/2024 09:43:54",
      "content": "<p>Check the log of the submission what is your error about</p>",
      "rawMarkdown": "Check the log of the submission what is your error about",
      "votes": null
    },
    {
      "id": "2794873",
      "postDate": "05/05/2024 14:54:50",
      "content": "<p>Can you post the error &amp; logs !! if out of memory you need to reduce reduce number of features or numpy data types..</p>\n<p>use below code :</p>\n<h3>from <a href=\"https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts\" target=\"_blank\">https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts</a></h3>\n<p>def reduce_mem_usage(df, float16_as32=True):<br>\n    \"\"\" iterate through all the columns of a dataframe and modify the data type<br>\n        to reduce memory usage.        <br>\n    \"\"\"<br>\n    start_mem = df.memory_usage().sum() / 1024**2<br>\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))</p>\n<pre><code> col  df:\n    col_type = df\n     (col_type)==:\n        continue\n\n\n     col_type != :\n        c_min = df()\n        c_max = df()\n         (col_type) == :\n             c_min &gt; np(np.int8) and c_max &lt; np(np.int8):\n                df = df(np.int8)\n            elif c_min &gt; np(np.int16) and c_max &lt; np(np.int16):\n                df = df(np.int16)\n            elif c_min &gt; np(np.int32) and c_max &lt; np(np.int32):\n                df = df(np.int32)\n            elif c_min &gt; np(np.int64) and c_max &lt; np(np.int64):\n                df = df(np.int64)  \n        :\n             c_min &gt; np(np.float16) and c_max &lt; np(np.float16):\n                 float16_as32:\n                    df = df(np.float32)\n                :\n                    df = df(np.float16)                    \n            elif c_min &gt; np(np.float32) and c_max &lt; np(np.float32):\n                df = df(np.float32)\n            :\n                df = df(np.float64)\n    :\n        df = df()\nend_mem = df()() / **\n)\n / start_mem))\n\nreturn df\n</code></pre>",
      "rawMarkdown": "Can you post the error & logs !! if out of memory you need to reduce reduce number of features or numpy data types..\n\nuse below code :\n\n\n### from https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts\ndef reduce_mem_usage(df, float16_as32=True):\n    \"\"\" iterate through all the columns of a dataframe and modify the data type\n        to reduce memory usage.        \n    \"\"\"\n    start_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n    \n    for col in df.columns:\n        col_type = df[col].dtype\n        if str(col_type)==\"category\":\n            continue\n        \n        \n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    if float16_as32:\n                        df[col] = df[col].astype(np.float32)\n                    else:\n                        df[col] = df[col].astype(np.float16)                    \n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n        else:\n            df[col] = df[col].astype('category')\n    end_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n    \n    return df",
      "votes": null
    },
    {
      "id": "2827485",
      "postDate": "05/21/2024 13:57:18",
      "content": "<p>Guys, thank you for your kind comments!<br>\nI attached the log, but the notebook run successfully. That's why I couldn't troubleshoot this error.</p>",
      "rawMarkdown": "Guys, thank you for your kind comments!\nI attached the log, but the notebook run successfully. That's why I couldn't troubleshoot this error.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2790370,
      "author_name": "yunsuxiaozi",
      "author_url": "",
      "post_date": "05/03/2024 06:53:04",
      "content": "<p>You don't even make the code public, how can we ask everyone to help you check it?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2794444,
      "author_name": "eu1234",
      "author_url": "",
      "post_date": "05/05/2024 09:43:54",
      "content": "<p>Check the log of the submission what is your error about</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2794873,
      "author_name": "kunduruanil",
      "author_url": "",
      "post_date": "05/05/2024 14:54:50",
      "content": "<p>Can you post the error &amp; logs !! if out of memory you need to reduce reduce number of features or numpy data types..</p>\n<p>use below code :</p>\n<h3>from <a href=\"https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts\" target=\"_blank\">https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts</a></h3>\n<p>def reduce_mem_usage(df, float16_as32=True):<br>\n    \"\"\" iterate through all the columns of a dataframe and modify the data type<br>\n        to reduce memory usage.        <br>\n    \"\"\"<br>\n    start_mem = df.memory_usage().sum() / 1024**2<br>\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))</p>\n<pre><code> col  df:\n    col_type = df\n     (col_type)==:\n        continue\n\n\n     col_type != :\n        c_min = df()\n        c_max = df()\n         (col_type) == :\n             c_min &gt; np(np.int8) and c_max &lt; np(np.int8):\n                df = df(np.int8)\n            elif c_min &gt; np(np.int16) and c_max &lt; np(np.int16):\n                df = df(np.int16)\n            elif c_min &gt; np(np.int32) and c_max &lt; np(np.int32):\n                df = df(np.int32)\n            elif c_min &gt; np(np.int64) and c_max &lt; np(np.int64):\n                df = df(np.int64)  \n        :\n             c_min &gt; np(np.float16) and c_max &lt; np(np.float16):\n                 float16_as32:\n                    df = df(np.float32)\n                :\n                    df = df(np.float16)                    \n            elif c_min &gt; np(np.float32) and c_max &lt; np(np.float32):\n                df = df(np.float32)\n            :\n                df = df(np.float64)\n    :\n        df = df()\nend_mem = df()() / **\n)\n / start_mem))\n\nreturn df\n</code></pre>",
      "votes": null,
      "replies": []
    },
    {
      "id": 2827485,
      "author_name": "mountaingoriillas",
      "author_url": "",
      "post_date": "05/21/2024 13:57:18",
      "content": "<p>Guys, thank you for your kind comments!<br>\nI attached the log, but the notebook run successfully. That's why I couldn't troubleshoot this error.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2787141": "Although my notebook runs successfully, I encounter a 'submission scoring error' and my work isn't graded. Can anyone guess why this might be happening?",
    "2790370": "You don't even make the code public, how can we ask everyone to help you check it?",
    "2794444": "Check the log of the submission what is your error about",
    "2794873": "Can you post the error & logs !! if out of memory you need to reduce reduce number of features or numpy data types..\n\nuse below code :\n\n\n### from https://www.kaggle.com/code/batprem/home-credit-risk-mode-utility-scripts\ndef reduce_mem_usage(df, float16_as32=True):\n    \"\"\" iterate through all the columns of a dataframe and modify the data type\n        to reduce memory usage.        \n    \"\"\"\n    start_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage of dataframe is {:.2f} MB'.format(start_mem))\n    \n    for col in df.columns:\n        col_type = df[col].dtype\n        if str(col_type)==\"category\":\n            continue\n        \n        \n        if col_type != object:\n            c_min = df[col].min()\n            c_max = df[col].max()\n            if str(col_type)[:3] == 'int':\n                if c_min > np.iinfo(np.int8).min and c_max < np.iinfo(np.int8).max:\n                    df[col] = df[col].astype(np.int8)\n                elif c_min > np.iinfo(np.int16).min and c_max < np.iinfo(np.int16).max:\n                    df[col] = df[col].astype(np.int16)\n                elif c_min > np.iinfo(np.int32).min and c_max < np.iinfo(np.int32).max:\n                    df[col] = df[col].astype(np.int32)\n                elif c_min > np.iinfo(np.int64).min and c_max < np.iinfo(np.int64).max:\n                    df[col] = df[col].astype(np.int64)  \n            else:\n                if c_min > np.finfo(np.float16).min and c_max < np.finfo(np.float16).max:\n                    if float16_as32:\n                        df[col] = df[col].astype(np.float32)\n                    else:\n                        df[col] = df[col].astype(np.float16)                    \n                elif c_min > np.finfo(np.float32).min and c_max < np.finfo(np.float32).max:\n                    df[col] = df[col].astype(np.float32)\n                else:\n                    df[col] = df[col].astype(np.float64)\n        else:\n            df[col] = df[col].astype('category')\n    end_mem = df.memory_usage().sum() / 1024**2\n    print('Memory usage after optimization is: {:.2f} MB'.format(end_mem))\n    print('Decreased by {:.1f}%'.format(100 * (start_mem - end_mem) / start_mem))\n    \n    return df",
    "2827485": "Guys, thank you for your kind comments!\nI attached the log, but the notebook run successfully. That's why I couldn't troubleshoot this error."
  },
  "source": "meta"
}