{ "id": 135147, "title": "What does 'bounded away' mean during Evaluation?", "url": "/competitions/deepfake-detection-challenge/discussion/135147", "author_name": "", "post_date": "2020-03-12T09:13:07.324289400Z", "votes": null, "comment_count": 3, "views": 0, "content": "

Evaluation page reads this:\n... a prediction that something is true when it is actually false will add infinite to your error score. In order to prevent this, predictions are bounded away from the extremes by a small value.

\n\n
    \n
  1. Bounded away by kaggle? Meaning if prediction is 1, kaggle replace it by 0.999 or 0 replaced by 0.001?
    2. \n
  2. All predictions altered (ie bounded away) or just the extremes, ie zeros and ones?
    3. \n
", "messages": [ { "id": "769789", "postDate": "03/12/2020 09:13:07", "content": "

Evaluation page reads this:\n... a prediction that something is true when it is actually false will add infinite to your error score. In order to prevent this, predictions are bounded away from the extremes by a small value.

\n\n
    \n
  1. Bounded away by kaggle? Meaning if prediction is 1, kaggle replace it by 0.999 or 0 replaced by 0.001?
    2. \n
  2. All predictions altered (ie bounded away) or just the extremes, ie zeros and ones?
    3. \n
", "rawMarkdown": "Evaluation page reads this:\n*... a prediction that something is true when it is actually false will add infinite to your error score. In order to prevent this, predictions are bounded away from the extremes by a small value.*\n\n1. Bounded away by kaggle? Meaning if prediction is 1, kaggle replace it by 0.999 or 0 replaced by 0.001?\n2. All predictions altered (ie bounded away) or just the extremes, ie zeros and ones?", "votes": null }, { "id": "769820", "postDate": "03/12/2020 09:59:30", "content": "

What they mean is that the loss is defined as:

\n\n

\nlogloss = y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)\n

\n\n

But it is actually computed as:

\n\n

\nlogloss = y_true * log(y_pred + small_value) + (1 - y_true) * log(1 - y_pred + small_value)\n

\n\n

where small_value = 1e-15.

", "rawMarkdown": "What they mean is that the loss is defined as:\n\n```\nlogloss = y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)\n```\n\nBut it is actually computed as:\n\n```\nlogloss = y_true * log(y_pred + small_value) + (1 - y_true) * log(1 - y_pred + small_value)\n```\n\nwhere `small_value = 1e-15`.", "votes": null }, { "id": "769838", "postDate": "03/12/2020 10:19:59", "content": "

hi Human Analog,\nthanks for your response. yes, your method would prevent 0 arguments in the log. and if small_value falls in the range of e-15 it should not change logloss very much. \nwell, what I dont quite understand now why my offline logloss calculation is so far from what kaggle calculated for my commit on the LB.

", "rawMarkdown": "hi Human Analog,\nthanks for your response. yes, your method would prevent 0 arguments in the log. and if small_value falls in the range of e-15 it should not change logloss very much. \nwell, what I dont quite understand now why my offline logloss calculation is so far from what kaggle calculated for my commit on the LB.", "votes": null }, { "id": "769856", "postDate": "03/12/2020 10:36:08", "content": "

The LB uses a different dataset, so you can't really compare your offline score with the LB score.

", "rawMarkdown": "The LB uses a different dataset, so you can't really compare your offline score with the LB score.", "votes": null } ], "comments": [ { "id": 769820, "author_name": "humananalog", "author_url": "", "post_date": "03/12/2020 09:59:30", "content": "

What they mean is that the loss is defined as:

\n\n

\nlogloss = y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)\n

\n\n

But it is actually computed as:

\n\n

\nlogloss = y_true * log(y_pred + small_value) + (1 - y_true) * log(1 - y_pred + small_value)\n

\n\n

where small_value = 1e-15.

", "votes": null, "replies": [] }, { "id": 769838, "author_name": "nandor65", "author_url": "", "post_date": "03/12/2020 10:19:59", "content": "

hi Human Analog,\nthanks for your response. yes, your method would prevent 0 arguments in the log. and if small_value falls in the range of e-15 it should not change logloss very much. \nwell, what I dont quite understand now why my offline logloss calculation is so far from what kaggle calculated for my commit on the LB.

", "votes": null, "replies": [ { "id": 769856, "author_name": "humananalog", "author_url": "", "post_date": "03/12/2020 10:36:08", "content": "

The LB uses a different dataset, so you can't really compare your offline score with the LB score.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "769789": "Evaluation page reads this:\n*... a prediction that something is true when it is actually false will add infinite to your error score. In order to prevent this, predictions are bounded away from the extremes by a small value.*\n\n1. Bounded away by kaggle? Meaning if prediction is 1, kaggle replace it by 0.999 or 0 replaced by 0.001?\n2. All predictions altered (ie bounded away) or just the extremes, ie zeros and ones?", "769820": "What they mean is that the loss is defined as:\n\n```\nlogloss = y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)\n```\n\nBut it is actually computed as:\n\n```\nlogloss = y_true * log(y_pred + small_value) + (1 - y_true) * log(1 - y_pred + small_value)\n```\n\nwhere `small_value = 1e-15`.", "769838": "hi Human Analog,\nthanks for your response. yes, your method would prevent 0 arguments in the log. and if small_value falls in the range of e-15 it should not change logloss very much. \nwell, what I dont quite understand now why my offline logloss calculation is so far from what kaggle calculated for my commit on the LB.", "769856": "The LB uses a different dataset, so you can't really compare your offline score with the LB score." }, "source": "meta" }