{
  "id": 93018,
  "title": "Fold 3 is smaller",
  "url": "/competitions/bigdata2019-flare-prediction/discussion/93018",
  "author_name": "",
  "post_date": "2019-05-22T12:01:38.176753400Z",
  "votes": null,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Based off of the data descriptions, talking about holding out 1/3 of the data, disjointness, etc., I would expect the 3 folds of data to be the same size. However, I noticed based on file size, that fold 3 is 1/3 the size of folds 1 and 2.  I guess this makes the hold out ratios a little trickier trying to keep unoverlapping samples. Any comments or thoughts from anyone regarding this or how you are handling it?</p>",
  "messages": [
    {
      "id": "535159",
      "postDate": "05/22/2019 12:01:38",
      "content": "<p>Based off of the data descriptions, talking about holding out 1/3 of the data, disjointness, etc., I would expect the 3 folds of data to be the same size. However, I noticed based on file size, that fold 3 is 1/3 the size of folds 1 and 2.  I guess this makes the hold out ratios a little trickier trying to keep unoverlapping samples. Any comments or thoughts from anyone regarding this or how you are handling it?</p>",
      "rawMarkdown": "Based off of the data descriptions, talking about holding out 1/3 of the data, disjointness, etc., I would expect the 3 folds of data to be the same size. However, I noticed based on file size, that fold 3 is 1/3 the size of folds 1 and 2.  I guess this makes the hold out ratios a little trickier trying to keep unoverlapping samples. Any comments or thoughts from anyone regarding this or how you are handling it?",
      "votes": null
    },
    {
      "id": "536392",
      "postDate": "05/24/2019 11:47:18",
      "content": "<p>When splitting the folds, we attempted to keep the number of positive examples in each fold the same or close to the same.  The sun goes through phases of higher and lower activity in the number of flares produced,  the number of active regions seen, and the percentage of those active regions seen actually producing flares. This leads to having a bit of variation in the time it takes for there to be N observations of the positive flare example.  Additionally, we don't limit the number of negative examples in a partition, so this is indeed wildly variable.  </p>\n\n<p>The partitions are as follows P1 (05-2010 to 03-2012), P2 (03-2012 to 10-2013), P3 (10-2013 to 03-2014), P4 (03-2014 to 03-2015), and P5 (03-2015 to 08-2018).  </p>",
      "rawMarkdown": "When splitting the folds, we attempted to keep the number of positive examples in each fold the same or close to the same.  The sun goes through phases of higher and lower activity in the number of flares produced,  the number of active regions seen, and the percentage of those active regions seen actually producing flares. This leads to having a bit of variation in the time it takes for there to be N observations of the positive flare example.  Additionally, we don't limit the number of negative examples in a partition, so this is indeed wildly variable.  \n\nThe partitions are as follows P1 (05-2010 to 03-2012), P2 (03-2012 to 10-2013), P3 (10-2013 to 03-2014), P4 (03-2014 to 03-2015), and P5 (03-2015 to 08-2018).",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 536392,
      "author_name": "dkempton",
      "author_url": "",
      "post_date": "05/24/2019 11:47:18",
      "content": "<p>When splitting the folds, we attempted to keep the number of positive examples in each fold the same or close to the same.  The sun goes through phases of higher and lower activity in the number of flares produced,  the number of active regions seen, and the percentage of those active regions seen actually producing flares. This leads to having a bit of variation in the time it takes for there to be N observations of the positive flare example.  Additionally, we don't limit the number of negative examples in a partition, so this is indeed wildly variable.  </p>\n\n<p>The partitions are as follows P1 (05-2010 to 03-2012), P2 (03-2012 to 10-2013), P3 (10-2013 to 03-2014), P4 (03-2014 to 03-2015), and P5 (03-2015 to 08-2018).  </p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "535159": "Based off of the data descriptions, talking about holding out 1/3 of the data, disjointness, etc., I would expect the 3 folds of data to be the same size. However, I noticed based on file size, that fold 3 is 1/3 the size of folds 1 and 2.  I guess this makes the hold out ratios a little trickier trying to keep unoverlapping samples. Any comments or thoughts from anyone regarding this or how you are handling it?",
    "536392": "When splitting the folds, we attempted to keep the number of positive examples in each fold the same or close to the same.  The sun goes through phases of higher and lower activity in the number of flares produced,  the number of active regions seen, and the percentage of those active regions seen actually producing flares. This leads to having a bit of variation in the time it takes for there to be N observations of the positive flare example.  Additionally, we don't limit the number of negative examples in a partition, so this is indeed wildly variable.  \n\nThe partitions are as follows P1 (05-2010 to 03-2012), P2 (03-2012 to 10-2013), P3 (10-2013 to 03-2014), P4 (03-2014 to 03-2015), and P5 (03-2015 to 08-2018)."
  },
  "source": "meta"
}