{
  "id": 177525,
  "title": "Some basic questions about the dataset / challenge",
  "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/177525",
  "author_name": "",
  "post_date": "2020-08-26T08:00:42.346308300Z",
  "votes": 8,
  "comment_count": 6,
  "views": 0,
  "content": "<p>I apologize if this has already been answered, if so please direct me to the appropriate post.</p>\n<p>I understand we are provided the patient's initial <code>FVC</code>.  This is also given with a week value, which is relative to when they had the baseline CT Scan (week 0).  </p>\n<p>Let me use the first patient in the test set as an example.</p>\n<pre><code>Patient                                              Weeks    FVC\nID00419637202311204720264     6             3020\n</code></pre>\n<p>So this means the <code>FVC</code> was taken 6 weeks after the CT scan correct?  And we are to predict the final three <code>FVC</code> measurements, but we don't know when those weeks will be.  They will obviously be after week 6.  How do we know how far out to forecast?  I mean should we compute weeks 7-128? 7-384?  I realize we won't know the exact 3 weeks……and there may be spacing between the weeks correct?</p>\n<p>I see the <code>sample_submission.csv</code> has weeks -12 to 133, perhaps this is the range that the predictions will for sure be in? If so, then we can just submit any value for weeks prior to the initial FVC?  example: prior to week 6 values will not be scored for the above patient is this correct?</p>",
  "messages": [
    {
      "id": "986106",
      "postDate": "08/26/2020 08:00:42",
      "content": "<p>I apologize if this has already been answered, if so please direct me to the appropriate post.</p>\n<p>I understand we are provided the patient's initial <code>FVC</code>.  This is also given with a week value, which is relative to when they had the baseline CT Scan (week 0).  </p>\n<p>Let me use the first patient in the test set as an example.</p>\n<pre><code>Patient                                              Weeks    FVC\nID00419637202311204720264     6             3020\n</code></pre>\n<p>So this means the <code>FVC</code> was taken 6 weeks after the CT scan correct?  And we are to predict the final three <code>FVC</code> measurements, but we don't know when those weeks will be.  They will obviously be after week 6.  How do we know how far out to forecast?  I mean should we compute weeks 7-128? 7-384?  I realize we won't know the exact 3 weeks……and there may be spacing between the weeks correct?</p>\n<p>I see the <code>sample_submission.csv</code> has weeks -12 to 133, perhaps this is the range that the predictions will for sure be in? If so, then we can just submit any value for weeks prior to the initial FVC?  example: prior to week 6 values will not be scored for the above patient is this correct?</p>",
      "rawMarkdown": "I apologize if this has already been answered, if so please direct me to the appropriate post.\n\nI understand we are provided the patient's initial `FVC`.  This is also given with a week value, which is relative to when they had the baseline CT Scan (week 0).  \n\nLet me use the first patient in the test set as an example.\n\n```\nPatient                                              Weeks    FVC\nID00419637202311204720264     6             3020\n```\n\nSo this means the `FVC` was taken 6 weeks after the CT scan correct?  And we are to predict the final three `FVC` measurements, but we don't know when those weeks will be.  They will obviously be after week 6.  How do we know how far out to forecast?  I mean should we compute weeks 7-128? 7-384?  I realize we won't know the exact 3 weeks......and there may be spacing between the weeks correct?\n\nI see the `sample_submission.csv` has weeks -12 to 133, perhaps this is the range that the predictions will for sure be in? If so, then we can just submit any value for weeks prior to the initial FVC?  example: prior to week 6 values will not be scored for the above patient is this correct?",
      "votes": null
    },
    {
      "id": "986150",
      "postDate": "08/26/2020 08:45:22",
      "content": "<blockquote>\n  <p>So this means the FVC was taken 6 weeks after the CT scan correct?</p>\n</blockquote>\n<p>That is correct.</p>\n<blockquote>\n  <p>but we don't know when those weeks will be. They will obviously be after week 6. How do we know how far out to forecast?</p>\n</blockquote>\n<p>You can't know. That's the hard part about this competition. You have to forecast from -12 to 133 weeks.</p>\n<blockquote>\n  <p>we can just submit any value for weeks prior to the initial FVC? example: prior to week 6 values will not be scored for the above patient is this correct?</p>\n</blockquote>\n<p>That is also correct.</p>",
      "rawMarkdown": "> So this means the FVC was taken 6 weeks after the CT scan correct?\n\nThat is correct.\n\n> but we don't know when those weeks will be. They will obviously be after week 6. How do we know how far out to forecast?\n\nYou can't know. That's the hard part about this competition. You have to forecast from -12 to 133 weeks.\n\n> we can just submit any value for weeks prior to the initial FVC? example: prior to week 6 values will not be scored for the above patient is this correct?\n\nThat is also correct.",
      "votes": null
    },
    {
      "id": "986238",
      "postDate": "08/26/2020 10:03:25",
      "content": "<p><a href=\"https://www.kaggle.com/brianfeeny\" target=\"_blank\">@brianfeeny</a>, what Gunes stated is also my understanding. I tried to explain it a little in my <a href=\"https://www.kaggle.com/chrisden/6-82-quantile-reg-w-custom-lr-schedulers\" target=\"_blank\">notebook</a>; every prediction needs to be made from the starting week to week 133. I had a hard time understanding what we are giving and how to pre-process the data. <br>\nSadly we dont know what were the last 3 weeks.<br>\nSo the questions are: 1) is the baseline-week (in your example: 6) a valid feature for prediction? <br>\n2) how can we use CT scans to determine if the lung volume will further decline or increase? Via 3d reconstruction and volume estimation? Via specific shapes of the lung, which indicate a bad prognosis? I am currently looking for the answer ;)</p>",
      "rawMarkdown": "brianfeeny, what Gunes stated is also my understanding. I tried to explain it a little in my [notebook](https://www.kaggle.com/chrisden/6-82-quantile-reg-w-custom-lr-schedulers); every prediction needs to be made from the starting week to week 133. I had a hard time understanding what we are giving and how to pre-process the data. \nSadly we dont know what were the last 3 weeks.\nSo the questions are: 1) is the baseline-week (in your example: 6) a valid feature for prediction? \n2) how can we use CT scans to determine if the lung volume will further decline or increase? Via 3d reconstruction and volume estimation? Via specific shapes of the lung, which indicate a bad prognosis? I am currently looking for the answer ;)",
      "votes": null
    },
    {
      "id": "986253",
      "postDate": "08/26/2020 10:11:32",
      "content": "<p>Yes, I would say we can use all the info in test.csv as features for prediction, thats what I am going with.</p>",
      "rawMarkdown": "Yes, I would say we can use all the info in test.csv as features for prediction, thats what I am going with.",
      "votes": null
    },
    {
      "id": "987470",
      "postDate": "08/27/2020 09:43:25",
      "content": "<p>thanks <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a>. However, if prior weeks won't be scored, why should we predict them? for instance, in this case week 5 and prior won't be used for scoring, is that right?</p>",
      "rawMarkdown": "thanks @gunesevitan. However, if prior weeks won't be scored, why should we predict them? for instance, in this case week 5 and prior won't be used for scoring, is that right?",
      "votes": null
    },
    {
      "id": "987555",
      "postDate": "08/27/2020 10:44:14",
      "content": "<p>Yes, that's right. Although the reason we want you to provide predictions in this range is that you don't know the weeks of the last three visits. This range has been chosen basically to cover the whole range of weeks in the test set. It shouldn't cause any harm to your model or your score. </p>",
      "rawMarkdown": "Yes, that's right. Although the reason we want you to provide predictions in this range is that you don't know the weeks of the last three visits. This range has been chosen basically to cover the whole range of weeks in the test set. It shouldn't cause any harm to your model or your score.",
      "votes": null
    },
    {
      "id": "987604",
      "postDate": "08/27/2020 11:41:36",
      "content": "<p><a href=\"https://www.kaggle.com/ahmedhshahin\" target=\"_blank\">@ahmedhshahin</a> understood. Thanks for clarifying. Realized later that this had been answered on other discussion topics as well. </p>",
      "rawMarkdown": "ahmedhshahin understood. Thanks for clarifying. Realized later that this had been answered on other discussion topics as well.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 986150,
      "author_name": "gunesevitan",
      "author_url": "",
      "post_date": "08/26/2020 08:45:22",
      "content": "<blockquote>\n  <p>So this means the FVC was taken 6 weeks after the CT scan correct?</p>\n</blockquote>\n<p>That is correct.</p>\n<blockquote>\n  <p>but we don't know when those weeks will be. They will obviously be after week 6. How do we know how far out to forecast?</p>\n</blockquote>\n<p>You can't know. That's the hard part about this competition. You have to forecast from -12 to 133 weeks.</p>\n<blockquote>\n  <p>we can just submit any value for weeks prior to the initial FVC? example: prior to week 6 values will not be scored for the above patient is this correct?</p>\n</blockquote>\n<p>That is also correct.</p>",
      "votes": null,
      "replies": [
        {
          "id": 987470,
          "author_name": "oshanm",
          "author_url": "",
          "post_date": "08/27/2020 09:43:25",
          "content": "<p>thanks <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a>. However, if prior weeks won't be scored, why should we predict them? for instance, in this case week 5 and prior won't be used for scoring, is that right?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 987555,
          "author_name": "ahmedhshahin",
          "author_url": "",
          "post_date": "08/27/2020 10:44:14",
          "content": "<p>Yes, that's right. Although the reason we want you to provide predictions in this range is that you don't know the weeks of the last three visits. This range has been chosen basically to cover the whole range of weeks in the test set. It shouldn't cause any harm to your model or your score. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 987604,
          "author_name": "oshanm",
          "author_url": "",
          "post_date": "08/27/2020 11:41:36",
          "content": "<p><a href=\"https://www.kaggle.com/ahmedhshahin\" target=\"_blank\">@ahmedhshahin</a> understood. Thanks for clarifying. Realized later that this had been answered on other discussion topics as well. </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 986238,
      "author_name": "ChristianDenich",
      "author_url": "",
      "post_date": "08/26/2020 10:03:25",
      "content": "<p><a href=\"https://www.kaggle.com/brianfeeny\" target=\"_blank\">@brianfeeny</a>, what Gunes stated is also my understanding. I tried to explain it a little in my <a href=\"https://www.kaggle.com/chrisden/6-82-quantile-reg-w-custom-lr-schedulers\" target=\"_blank\">notebook</a>; every prediction needs to be made from the starting week to week 133. I had a hard time understanding what we are giving and how to pre-process the data. <br>\nSadly we dont know what were the last 3 weeks.<br>\nSo the questions are: 1) is the baseline-week (in your example: 6) a valid feature for prediction? <br>\n2) how can we use CT scans to determine if the lung volume will further decline or increase? Via 3d reconstruction and volume estimation? Via specific shapes of the lung, which indicate a bad prognosis? I am currently looking for the answer ;)</p>",
      "votes": null,
      "replies": [
        {
          "id": 986253,
          "author_name": "brianfeeny",
          "author_url": "",
          "post_date": "08/26/2020 10:11:32",
          "content": "<p>Yes, I would say we can use all the info in test.csv as features for prediction, thats what I am going with.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "986106": "I apologize if this has already been answered, if so please direct me to the appropriate post.\n\nI understand we are provided the patient's initial `FVC`.  This is also given with a week value, which is relative to when they had the baseline CT Scan (week 0).  \n\nLet me use the first patient in the test set as an example.\n\n```\nPatient                                              Weeks    FVC\nID00419637202311204720264     6             3020\n```\n\nSo this means the `FVC` was taken 6 weeks after the CT scan correct?  And we are to predict the final three `FVC` measurements, but we don't know when those weeks will be.  They will obviously be after week 6.  How do we know how far out to forecast?  I mean should we compute weeks 7-128? 7-384?  I realize we won't know the exact 3 weeks......and there may be spacing between the weeks correct?\n\nI see the `sample_submission.csv` has weeks -12 to 133, perhaps this is the range that the predictions will for sure be in? If so, then we can just submit any value for weeks prior to the initial FVC?  example: prior to week 6 values will not be scored for the above patient is this correct?",
    "986150": "> So this means the FVC was taken 6 weeks after the CT scan correct?\n\nThat is correct.\n\n> but we don't know when those weeks will be. They will obviously be after week 6. How do we know how far out to forecast?\n\nYou can't know. That's the hard part about this competition. You have to forecast from -12 to 133 weeks.\n\n> we can just submit any value for weeks prior to the initial FVC? example: prior to week 6 values will not be scored for the above patient is this correct?\n\nThat is also correct.",
    "986238": "brianfeeny, what Gunes stated is also my understanding. I tried to explain it a little in my [notebook](https://www.kaggle.com/chrisden/6-82-quantile-reg-w-custom-lr-schedulers); every prediction needs to be made from the starting week to week 133. I had a hard time understanding what we are giving and how to pre-process the data. \nSadly we dont know what were the last 3 weeks.\nSo the questions are: 1) is the baseline-week (in your example: 6) a valid feature for prediction? \n2) how can we use CT scans to determine if the lung volume will further decline or increase? Via 3d reconstruction and volume estimation? Via specific shapes of the lung, which indicate a bad prognosis? I am currently looking for the answer ;)",
    "986253": "Yes, I would say we can use all the info in test.csv as features for prediction, thats what I am going with.",
    "987470": "thanks @gunesevitan. However, if prior weeks won't be scored, why should we predict them? for instance, in this case week 5 and prior won't be used for scoring, is that right?",
    "987555": "Yes, that's right. Although the reason we want you to provide predictions in this range is that you don't know the weeks of the last three visits. This range has been chosen basically to cover the whole range of weeks in the test set. It shouldn't cause any harm to your model or your score.",
    "987604": "ahmedhshahin understood. Thanks for clarifying. Realized later that this had been answered on other discussion topics as well."
  },
  "source": "meta"
}