{
  "id": 216862,
  "title": "Why do the predictions contain negative values ?",
  "url": "/competitions/rfcx-species-audio-detection/discussion/216862",
  "author_name": "",
  "post_date": "2021-02-04T10:00:55.102075600Z",
  "votes": 1,
  "comment_count": 18,
  "views": 0,
  "content": "<p>Hello</p>\n<p>The submission files of some of the published note books seem to contain negative values or values greater than 1.</p>\n<p>Why is this?</p>\n<p>I thought the task was to predict probabilities, and that the predicted values should be between 0 and 1.</p>\n<p>thank you so much for your time,</p>",
  "messages": [
    {
      "id": "1185707",
      "postDate": "02/04/2021 10:00:55",
      "content": "<p>Hello</p>\n<p>The submission files of some of the published note books seem to contain negative values or values greater than 1.</p>\n<p>Why is this?</p>\n<p>I thought the task was to predict probabilities, and that the predicted values should be between 0 and 1.</p>\n<p>thank you so much for your time,</p>",
      "rawMarkdown": "Hello\n\nThe submission files of some of the published note books seem to contain negative values or values greater than 1.\n\nWhy is this?\n\nI thought the task was to predict probabilities, and that the predicted values should be between 0 and 1.\n\nthank you so much for your time,",
      "votes": null
    },
    {
      "id": "1185765",
      "postDate": "02/04/2021 11:01:10",
      "content": "<p>I haven't looked but maybe they output logits, i.e. values before sigmoid or softmax.  This is fine given that what matters is the ordering of values and not their actual values .</p>",
      "rawMarkdown": "I haven't looked but maybe they output logits, i.e. values before sigmoid or softmax.  This is fine given that what matters is the ordering of values and not their actual values .",
      "votes": null
    },
    {
      "id": "1185771",
      "postDate": "02/04/2021 11:06:49",
      "content": "<p>Hi,</p>\n<p>I tried to do submission where every prediction was set to one and another where everything was predicted above one.<br>\nBoth submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.</p>\n<p>By the way, the 'predict everything to one' model scores 0.861 in my case. It's quite a high score.</p>",
      "rawMarkdown": "Hi,\n\nI tried to do submission where every prediction was set to one and another where everything was predicted above one.\nBoth submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.\n\nBy the way, the 'predict everything to one' model scores 0.861 in my case. It's quite a high score.",
      "votes": null
    },
    {
      "id": "1185849",
      "postDate": "02/04/2021 12:19:24",
      "content": "<p><a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> With your answer and the topic about some Evaluation Metrics, I understood that I need to indicate in order the types that would be included in the audio.</p>\n<p>Thank you very much!</p>",
      "rawMarkdown": "cpmpml With your answer and the topic about some Evaluation Metrics, I understood that I need to indicate in order the types that would be included in the audio.\n\nThank you very much!",
      "votes": null
    },
    {
      "id": "1185857",
      "postDate": "02/04/2021 12:28:35",
      "content": "<p><a href=\"https://www.kaggle.com/nborbit\" target=\"_blank\">@nborbit</a> That's one strong model!<br>\nVery interesting😄</p>\n<p>Thank you very much.</p>",
      "rawMarkdown": "nborbit That's one strong model!\nVery interesting😄\n\nThank you very much.",
      "votes": null
    },
    {
      "id": "1186026",
      "postDate": "02/04/2021 14:48:08",
      "content": "<p>I highly recommend reading the accepted answer of this thread to understand how the LWLRAP metric works.<br>\n<a href=\"https://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score\" target=\"_blank\">https://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score</a></p>",
      "rawMarkdown": "I highly recommend reading the accepted answer of this thread to understand how the LWLRAP metric works.\nhttps://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score",
      "votes": null
    },
    {
      "id": "1186253",
      "postDate": "02/04/2021 17:35:50",
      "content": "<blockquote>\n  <p>Both submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.</p>\n</blockquote>\n<p>I'm afraid you are drawing the wrong conclusion.  </p>\n<p>You get the same score because your values are ordered the same (they are all equal).  The competition metric only depend on how the values are ordered in each row.  Multiplying all values by a constant, as you did, will not change the score.  More generally, applying an increasing function to the values, for instance sigmoid, or its inverse, logit, will not change the score either.</p>",
      "rawMarkdown": "> Both submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.\n\nI'm afraid you are drawing the wrong conclusion.  \n\nYou get the same score because your values are ordered the same (they are all equal).  The competition metric only depend on how the values are ordered in each row.  Multiplying all values by a constant, as you did, will not change the score.  More generally, applying an increasing function to the values, for instance sigmoid, or its inverse, logit, will not change the score either.",
      "votes": null
    },
    {
      "id": "1186588",
      "postDate": "02/04/2021 23:06:10",
      "content": "<p>The scores are not probabilities, only the order of largest to smallest counts.</p>",
      "rawMarkdown": "The scores are not probabilities, only the order of largest to smallest counts.",
      "votes": null
    },
    {
      "id": "1188195",
      "postDate": "02/06/2021 03:53:46",
      "content": "<p><a href=\"https://www.kaggle.com/bigironsphere\" target=\"_blank\">@bigironsphere</a> After asking a question on this topic, I realized that score is not a probability!</p>\n<p>Thank you very much.</p>",
      "rawMarkdown": "bigironsphere After asking a question on this topic, I realized that score is not a probability!\n\nThank you very much.",
      "votes": null
    },
    {
      "id": "1191016",
      "postDate": "02/08/2021 07:28:05",
      "content": "<p>Interesting… <br>\nAlso, I'm getting LB score as <strong>0.185,</strong> if I submit all values as 0.5 and getting <strong>0.861,</strong> if I submit all values as 1. The ranking is the same but the final result is different. </p>",
      "rawMarkdown": "Interesting... \nAlso, I'm getting LB score as **0.185,** if I submit all values as 0.5 and getting **0.861,** if I submit all values as 1. The ranking is the same but the final result is different.",
      "votes": null
    },
    {
      "id": "1191325",
      "postDate": "02/08/2021 11:54:28",
      "content": "<blockquote>\n  <p>every prediction was set to one and another where everything was predicted above one</p>\n</blockquote>\n<p>What exactly do you mean? What do you set to what and what do you do with what you don't set?</p>",
      "rawMarkdown": "> every prediction was set to one and another where everything was predicted above one\n\nWhat exactly do you mean? What do you set to what and what do you do with what you don't set?",
      "votes": null
    },
    {
      "id": "1191429",
      "postDate": "02/08/2021 13:09:44",
      "content": "<blockquote>\n  <p>Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. The ranking is the same but the final result is different. </p>\n</blockquote>\n<p>Wow.  This is really weird.  I started to have doubts about the metric implementation recently.  Actually since the 0.861 score of all 1 was reported.</p>",
      "rawMarkdown": "> Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. The ranking is the same but the final result is different. \n\nWow.  This is really weird.  I started to have doubts about the metric implementation recently.  Actually since the 0.861 score of all 1 was reported.",
      "votes": null
    },
    {
      "id": "1191454",
      "postDate": "02/08/2021 13:15:10",
      "content": "<p>i thought if you submit constants, the LB score should be the same?<br>\n(since rank is the same?)</p>\n<pre><code>def run_check_probe_metric():\n    onehot = np.array([\n        [1, 0, 0],\n        [1, 0, 0],\n        [1, 0, 0],\n        [0, 1, 0],\n        [0, 0, 1],\n        [0, 0, 1],\n    ])\n    # probability = np.array([\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    # ])\n    probability = np.array([\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n    ])\n\n    #https://scikit-learn.org/stable/modules/model_evaluation.html\n    lrap, weight = lwlrap(probability, onehot)\n    print(weight)\n    print(lrap)\n    print((lrap*weight).sum())\n</code></pre>",
      "rawMarkdown": "i thought if you submit constants, the LB score should be the same?\n(since rank is the same?)\n\n```\n\ndef run_check_probe_metric():\n    onehot = np.array([\n        [1, 0, 0],\n        [1, 0, 0],\n        [1, 0, 0],\n        [0, 1, 0],\n        [0, 0, 1],\n        [0, 0, 1],\n    ])\n    # probability = np.array([\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    # ])\n    probability = np.array([\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n    ])\n\n    #https://scikit-learn.org/stable/modules/model_evaluation.html\n    lrap, weight = lwlrap(probability, onehot)\n    print(weight)\n    print(lrap)\n    print((lrap*weight).sum())\n\n\n\n\n```",
      "votes": null
    },
    {
      "id": "1191469",
      "postDate": "02/08/2021 13:25:20",
      "content": "<blockquote>\n  <p>Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. </p>\n</blockquote>\n<p>Are you sure that you are not mistaking something? 0.861 is a score of a popular public notebook. It might be as simple as uploading wrong file for testing, you can download your actual submission from Kaggle and check it.</p>",
      "rawMarkdown": "> Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. \n\nAre you sure that you are not mistaking something? 0.861 is a score of a popular public notebook. It might be as simple as uploading wrong file for testing, you can download your actual submission from Kaggle and check it.",
      "votes": null
    },
    {
      "id": "1191633",
      "postDate": "02/08/2021 15:21:34",
      "content": "<p>Ooops. I was uploading the wrong file for testing. You can ignore what was said above. :)</p>",
      "rawMarkdown": "Ooops. I was uploading the wrong file for testing. You can ignore what was said above. :)",
      "votes": null
    },
    {
      "id": "1191652",
      "postDate": "02/08/2021 15:32:43",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> yes, you should get the same score, i have tried once with logits and one with probabilites. same LB.</p>",
      "rawMarkdown": "hengck23 yes, you should get the same score, i have tried once with logits and one with probabilites. same LB.",
      "votes": null
    },
    {
      "id": "1191760",
      "postDate": "02/08/2021 16:47:54",
      "content": "<p><a href=\"https://www.kaggle.com/nofreewill\" target=\"_blank\">@nofreewill</a> You've helped me to understand it better.</p>\n<p>Thank you very much.</p>",
      "rawMarkdown": "nofreewill You've helped me to understand it better.\n\nThank you very much.",
      "votes": null
    },
    {
      "id": "1191948",
      "postDate": "02/08/2021 19:44:30",
      "content": "<p>Hi all, is anyone including false postives in the model? Which value do you assign to false positives? 0? -1? -100? -10000?</p>",
      "rawMarkdown": "Hi all, is anyone including false postives in the model? Which value do you assign to false positives? 0? -1? -100? -10000?",
      "votes": null
    },
    {
      "id": "1191996",
      "postDate": "02/08/2021 20:53:42",
      "content": "<p>I'm happy that I could help!:)</p>",
      "rawMarkdown": "I'm happy that I could help!:)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 1185765,
      "author_name": "cpmpml",
      "author_url": "",
      "post_date": "02/04/2021 11:01:10",
      "content": "<p>I haven't looked but maybe they output logits, i.e. values before sigmoid or softmax.  This is fine given that what matters is the ordering of values and not their actual values .</p>",
      "votes": null,
      "replies": [
        {
          "id": 1185849,
          "author_name": "toratoratora",
          "author_url": "",
          "post_date": "02/04/2021 12:19:24",
          "content": "<p><a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> With your answer and the topic about some Evaluation Metrics, I understood that I need to indicate in order the types that would be included in the audio.</p>\n<p>Thank you very much!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186026,
          "author_name": "nofreewill",
          "author_url": "",
          "post_date": "02/04/2021 14:48:08",
          "content": "<p>I highly recommend reading the accepted answer of this thread to understand how the LWLRAP metric works.<br>\n<a href=\"https://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score\" target=\"_blank\">https://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score</a></p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191760,
          "author_name": "toratoratora",
          "author_url": "",
          "post_date": "02/08/2021 16:47:54",
          "content": "<p><a href=\"https://www.kaggle.com/nofreewill\" target=\"_blank\">@nofreewill</a> You've helped me to understand it better.</p>\n<p>Thank you very much.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191996,
          "author_name": "nofreewill",
          "author_url": "",
          "post_date": "02/08/2021 20:53:42",
          "content": "<p>I'm happy that I could help!:)</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1185771,
      "author_name": "nborbit",
      "author_url": "",
      "post_date": "02/04/2021 11:06:49",
      "content": "<p>Hi,</p>\n<p>I tried to do submission where every prediction was set to one and another where everything was predicted above one.<br>\nBoth submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.</p>\n<p>By the way, the 'predict everything to one' model scores 0.861 in my case. It's quite a high score.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1185857,
          "author_name": "toratoratora",
          "author_url": "",
          "post_date": "02/04/2021 12:28:35",
          "content": "<p><a href=\"https://www.kaggle.com/nborbit\" target=\"_blank\">@nborbit</a> That's one strong model!<br>\nVery interesting😄</p>\n<p>Thank you very much.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1186253,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "02/04/2021 17:35:50",
          "content": "<blockquote>\n  <p>Both submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.</p>\n</blockquote>\n<p>I'm afraid you are drawing the wrong conclusion.  </p>\n<p>You get the same score because your values are ordered the same (they are all equal).  The competition metric only depend on how the values are ordered in each row.  Multiplying all values by a constant, as you did, will not change the score.  More generally, applying an increasing function to the values, for instance sigmoid, or its inverse, logit, will not change the score either.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191016,
          "author_name": "nborbit",
          "author_url": "",
          "post_date": "02/08/2021 07:28:05",
          "content": "<p>Interesting… <br>\nAlso, I'm getting LB score as <strong>0.185,</strong> if I submit all values as 0.5 and getting <strong>0.861,</strong> if I submit all values as 1. The ranking is the same but the final result is different. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191325,
          "author_name": "nofreewill",
          "author_url": "",
          "post_date": "02/08/2021 11:54:28",
          "content": "<blockquote>\n  <p>every prediction was set to one and another where everything was predicted above one</p>\n</blockquote>\n<p>What exactly do you mean? What do you set to what and what do you do with what you don't set?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191429,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "02/08/2021 13:09:44",
          "content": "<blockquote>\n  <p>Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. The ranking is the same but the final result is different. </p>\n</blockquote>\n<p>Wow.  This is really weird.  I started to have doubts about the metric implementation recently.  Actually since the 0.861 score of all 1 was reported.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191454,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "02/08/2021 13:15:10",
          "content": "<p>i thought if you submit constants, the LB score should be the same?<br>\n(since rank is the same?)</p>\n<pre><code>def run_check_probe_metric():\n    onehot = np.array([\n        [1, 0, 0],\n        [1, 0, 0],\n        [1, 0, 0],\n        [0, 1, 0],\n        [0, 0, 1],\n        [0, 0, 1],\n    ])\n    # probability = np.array([\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    # ])\n    probability = np.array([\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n    ])\n\n    #https://scikit-learn.org/stable/modules/model_evaluation.html\n    lrap, weight = lwlrap(probability, onehot)\n    print(weight)\n    print(lrap)\n    print((lrap*weight).sum())\n</code></pre>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191469,
          "author_name": "fffrrt",
          "author_url": "",
          "post_date": "02/08/2021 13:25:20",
          "content": "<blockquote>\n  <p>Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. </p>\n</blockquote>\n<p>Are you sure that you are not mistaking something? 0.861 is a score of a popular public notebook. It might be as simple as uploading wrong file for testing, you can download your actual submission from Kaggle and check it.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191633,
          "author_name": "nborbit",
          "author_url": "",
          "post_date": "02/08/2021 15:21:34",
          "content": "<p>Ooops. I was uploading the wrong file for testing. You can ignore what was said above. :)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 1191652,
          "author_name": "tugstugi",
          "author_url": "",
          "post_date": "02/08/2021 15:32:43",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> yes, you should get the same score, i have tried once with logits and one with probabilites. same LB.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1186588,
      "author_name": "bigironsphere",
      "author_url": "",
      "post_date": "02/04/2021 23:06:10",
      "content": "<p>The scores are not probabilities, only the order of largest to smallest counts.</p>",
      "votes": null,
      "replies": [
        {
          "id": 1188195,
          "author_name": "toratoratora",
          "author_url": "",
          "post_date": "02/06/2021 03:53:46",
          "content": "<p><a href=\"https://www.kaggle.com/bigironsphere\" target=\"_blank\">@bigironsphere</a> After asking a question on this topic, I realized that score is not a probability!</p>\n<p>Thank you very much.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 1191948,
      "author_name": "hakandogan",
      "author_url": "",
      "post_date": "02/08/2021 19:44:30",
      "content": "<p>Hi all, is anyone including false postives in the model? Which value do you assign to false positives? 0? -1? -100? -10000?</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1185707": "Hello\n\nThe submission files of some of the published note books seem to contain negative values or values greater than 1.\n\nWhy is this?\n\nI thought the task was to predict probabilities, and that the predicted values should be between 0 and 1.\n\nthank you so much for your time,",
    "1185765": "I haven't looked but maybe they output logits, i.e. values before sigmoid or softmax.  This is fine given that what matters is the ordering of values and not their actual values .",
    "1185771": "Hi,\n\nI tried to do submission where every prediction was set to one and another where everything was predicted above one.\nBoth submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.\n\nBy the way, the 'predict everything to one' model scores 0.861 in my case. It's quite a high score.",
    "1185849": "cpmpml With your answer and the topic about some Evaluation Metrics, I understood that I need to indicate in order the types that would be included in the audio.\n\nThank you very much!",
    "1185857": "nborbit That's one strong model!\nVery interesting😄\n\nThank you very much.",
    "1186026": "I highly recommend reading the accepted answer of this thread to understand how the LWLRAP metric works.\nhttps://stackoverflow.com/questions/55881642/how-to-interpret-label-ranking-average-precision-score",
    "1186253": "> Both submissions get exactly the same score, so it looks like everything above 1 get treated as exactly 1.\n\nI'm afraid you are drawing the wrong conclusion.  \n\nYou get the same score because your values are ordered the same (they are all equal).  The competition metric only depend on how the values are ordered in each row.  Multiplying all values by a constant, as you did, will not change the score.  More generally, applying an increasing function to the values, for instance sigmoid, or its inverse, logit, will not change the score either.",
    "1186588": "The scores are not probabilities, only the order of largest to smallest counts.",
    "1188195": "bigironsphere After asking a question on this topic, I realized that score is not a probability!\n\nThank you very much.",
    "1191016": "Interesting... \nAlso, I'm getting LB score as **0.185,** if I submit all values as 0.5 and getting **0.861,** if I submit all values as 1. The ranking is the same but the final result is different.",
    "1191325": "> every prediction was set to one and another where everything was predicted above one\n\nWhat exactly do you mean? What do you set to what and what do you do with what you don't set?",
    "1191429": "> Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. The ranking is the same but the final result is different. \n\nWow.  This is really weird.  I started to have doubts about the metric implementation recently.  Actually since the 0.861 score of all 1 was reported.",
    "1191454": "i thought if you submit constants, the LB score should be the same?\n(since rank is the same?)\n\n```\n\ndef run_check_probe_metric():\n    onehot = np.array([\n        [1, 0, 0],\n        [1, 0, 0],\n        [1, 0, 0],\n        [0, 1, 0],\n        [0, 0, 1],\n        [0, 0, 1],\n    ])\n    # probability = np.array([\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    #     [1, 0, 0],\n    # ])\n    probability = np.array([\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n        [0, 1, 1],\n    ])\n\n    #https://scikit-learn.org/stable/modules/model_evaluation.html\n    lrap, weight = lwlrap(probability, onehot)\n    print(weight)\n    print(lrap)\n    print((lrap*weight).sum())\n\n\n\n\n```",
    "1191469": "> Also, I'm getting LB score as 0.185, if I submit all values as 0.5 and getting 0.861, if I submit all values as 1. \n\nAre you sure that you are not mistaking something? 0.861 is a score of a popular public notebook. It might be as simple as uploading wrong file for testing, you can download your actual submission from Kaggle and check it.",
    "1191633": "Ooops. I was uploading the wrong file for testing. You can ignore what was said above. :)",
    "1191652": "hengck23 yes, you should get the same score, i have tried once with logits and one with probabilites. same LB.",
    "1191760": "nofreewill You've helped me to understand it better.\n\nThank you very much.",
    "1191948": "Hi all, is anyone including false postives in the model? Which value do you assign to false positives? 0? -1? -100? -10000?",
    "1191996": "I'm happy that I could help!:)"
  },
  "source": "meta"
}