{
  "id": 39756,
  "title": "`transactions` dataset hard to reason",
  "url": "/competitions/kkbox-churn-prediction-challenge/discussion/39756",
  "author_name": "",
  "post_date": "2017-09-20T15:16:59.404728400Z",
  "votes": 10,
  "comment_count": 25,
  "views": 0,
  "content": "<p>I found the <code>transactions</code> dataset a little bit hard to reason.</p>\n\n<p>For example, take a look at the user <code>IwE/pih8PuqrY/rsnoZ/4TazDliyH9S8VWNc2/d7mJg=</code> in the <code>train</code> set, the related records in <code>transactions</code> look like there are multiple records within a day and multiple associated <code>membership_expire_date</code>, ranging across 3 years (2015-2017).</p>\n\n<p>What's more, in <code>members</code> this user has a <code>expiration_date</code> of <code>2017-12-18</code>, which can not be found in any of the associated transaction record.</p>\n\n<p>It will be appreciated that we can have an official elaboration about this.</p>",
  "messages": [
    {
      "id": "222919",
      "postDate": "09/20/2017 15:16:59",
      "content": "<p>I found the <code>transactions</code> dataset a little bit hard to reason.</p>\n\n<p>For example, take a look at the user <code>IwE/pih8PuqrY/rsnoZ/4TazDliyH9S8VWNc2/d7mJg=</code> in the <code>train</code> set, the related records in <code>transactions</code> look like there are multiple records within a day and multiple associated <code>membership_expire_date</code>, ranging across 3 years (2015-2017).</p>\n\n<p>What's more, in <code>members</code> this user has a <code>expiration_date</code> of <code>2017-12-18</code>, which can not be found in any of the associated transaction record.</p>\n\n<p>It will be appreciated that we can have an official elaboration about this.</p>",
      "rawMarkdown": "I found the `transactions` dataset a little bit hard to reason.\n\nFor example, take a look at the user `IwE/pih8PuqrY/rsnoZ/4TazDliyH9S8VWNc2/d7mJg=` in the `train` set, the related records in `transactions` look like there are multiple records within a day and multiple associated `membership_expire_date`, ranging across 3 years (2015-2017).\n\nWhat's more, in `members` this user has a `expiration_date` of `2017-12-18`, which can not be found in any of the associated transaction record.\n\nIt will be appreciated that we can have an official elaboration about this.",
      "votes": null
    },
    {
      "id": "223077",
      "postDate": "09/21/2017 03:25:48",
      "content": "<p>I also found that the max expiration_date of a user in the training set ranged from 20170201 to 20170331, however, the max expiration_date of a user in the test set ranged from 2015 to 2017. This is so weird.</p>",
      "rawMarkdown": "I also found that the max expiration_date of a user in the training set ranged from 20170201 to 20170331, however, the max expiration_date of a user in the test set ranged from 2015 to 2017. This is so weird.",
      "votes": null
    },
    {
      "id": "223243",
      "postDate": "09/21/2017 15:28:30",
      "content": "<p>I think one of my general question is:</p>\n\n<p>What is the relationship between the <code>membership_expire_date</code> in <code>transactions</code> and <code>expiration_date</code> in <code>members</code>? Because it seems that <code>expiration_date</code> can not be derived from <code>transactions</code>, are they indeed referring to totally different concept of expiration?</p>\n\n<p>One possible reason I can think of is the <code>members</code> data are derived from transaction records more (newer) than those in <code>transactions</code>, so surely the <code>expiration_date</code> will be newer than those recorded in <code>transactions</code>. But still I'll appreciate any official details about how we should reason these dataset before we can deliver meaningful prediction. :)</p>",
      "rawMarkdown": "I think one of my general question is:\n\nWhat is the relationship between the `membership_expire_date` in `transactions` and `expiration_date` in `members`? Because it seems that `expiration_date` can not be derived from `transactions`, are they indeed referring to totally different concept of expiration?\n\nOne possible reason I can think of is the `members` data are derived from transaction records more (newer) than those in `transactions`, so surely the `expiration_date` will be newer than those recorded in `transactions`. But still I'll appreciate any official details about how we should reason these dataset before we can deliver meaningful prediction. :)",
      "votes": null
    },
    {
      "id": "223358",
      "postDate": "09/22/2017 02:25:43",
      "content": "<p>The membership data is indeed derived using transaction data. We deliberately removed the transaction logs after 20170331. Even you see that a member expiration date of 2017-12-18 in membership table. This date does not imply that the subscriber stopped his/her subscription during the month of March-April, 2017.</p>",
      "rawMarkdown": "The membership data is indeed derived using transaction data. We deliberately removed the transaction logs after 20170331. Even you see that a member expiration date of 2017-12-18 in membership table. This date does not imply that the subscriber stopped his/her subscription during the month of March-April, 2017.",
      "votes": null
    },
    {
      "id": "223363",
      "postDate": "09/22/2017 03:02:12",
      "content": "<p>The provided training set is only serves as an example. Our data set is imbalanced, refer to <a href=\"https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda\">https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda</a>). In this dataset, we includ more users behaviors than the ones in train and test datasets, in order to enable participants to explore different user behaviors outside of the train and test sets. For example, a user could actively cancel the subscription, but renew 30 days after the previous cancellation. We encourage participants to discover new patterns/features from the transaction log and generate your own set of training data.</p>",
      "rawMarkdown": "The provided training set is only serves as an example. Our data set is imbalanced, refer to https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda). In this dataset, we includ more users behaviors than the ones in train and test datasets, in order to enable participants to explore different user behaviors outside of the train and test sets. For example, a user could actively cancel the subscription, but renew 30 days after the previous cancellation. We encourage participants to discover new patterns/features from the transaction log and generate your own set of training data.",
      "votes": null
    },
    {
      "id": "223549",
      "postDate": "09/22/2017 14:39:18",
      "content": "<p>This is very very confusing... you're saying that the 'train' dataset is not derived from 'transaction'? So we have '2' training sets? One was given (train.csv), the other one we should generate (from transactions.csv) ?</p>",
      "rawMarkdown": "This is very very confusing... you're saying that the 'train' dataset is not derived from 'transaction'? So we have '2' training sets? One was given (train.csv), the other one we should generate (from transactions.csv) ?",
      "votes": null
    },
    {
      "id": "223662",
      "postDate": "09/23/2017 00:36:00",
      "content": "<p>The provided training data set is derived from transaction log. We picked the users who have their expiration dates fall in Feb, 2017 and check whether those people renew their subscription with 30 days after expiration to generate training label. Our method is not the only way to generate the training data. The training data set can be generate using different logic. Say, you can check each user's transaction log and calculate the interval between two consecutive entries. In this case, you will generate a training data set much bigger than what we provided in the data section.  </p>",
      "rawMarkdown": "The provided training data set is derived from transaction log. We picked the users who have their expiration dates fall in Feb, 2017 and check whether those people renew their subscription with 30 days after expiration to generate training label. Our method is not the only way to generate the training data. The training data set can be generate using different logic. Say, you can check each user's transaction log and calculate the interval between two consecutive entries. In this case, you will generate a training data set much bigger than what we provided in the data section.",
      "votes": null
    },
    {
      "id": "223663",
      "postDate": "09/23/2017 00:40:27",
      "content": "<p>The expiration_date in members table is the membership_expire_date in the last entry for a user in the transaction log.  For the purpose of this competition, we filter out all transactions occurred after March 2017 in the transaction log.</p>",
      "rawMarkdown": "The expiration_date in members table is the membership_expire_date in the last entry for a user in the transaction log.  For the purpose of this competition, we filter out all transactions occurred after March 2017 in the transaction log.",
      "votes": null
    },
    {
      "id": "223693",
      "postDate": "09/23/2017 03:24:30",
      "content": "<p>In data page, it said </p>\n\n<blockquote>\n  <p>transactions of users up until 2/28/2017.</p>\n</blockquote>\n\n<p>Is \"removed the transaction logs after 20170331\" a typo in your reply?</p>",
      "rawMarkdown": "In data page, it said \n\n&gt; transactions of users up until 2/28/2017.\n\nIs \"removed the transaction logs after 20170331\" a typo in your reply?",
      "votes": null
    },
    {
      "id": "223697",
      "postDate": "09/23/2017 03:38:52",
      "content": "<p>As you mentioned </p>\n\n<blockquote>\n  <p>We picked the users who have their expiration dates fall in Feb, 2017</p>\n</blockquote>\n\n<p>As extremin mentioned</p>\n\n<blockquote>\n  <p>max expiration_date of a user in the training set ranged from 20170201 to 20170331</p>\n</blockquote>\n\n<p>Does it mean some users in provided train dataset must have an expiration fall in Feb and and other expiration(in transactions file) fall in Mar?</p>",
      "rawMarkdown": "As you mentioned \n\n&gt; We picked the users who have their expiration dates fall in Feb, 2017\n\nAs extremin mentioned\n\n&gt; max expiration_date of a user in the training set ranged from 20170201 to 20170331\n\nDoes it mean some users in provided train dataset must have an expiration fall in Feb and and other expiration(in transactions file) fall in Mar?",
      "votes": null
    },
    {
      "id": "223953",
      "postDate": "09/24/2017 12:24:42",
      "content": "",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "223954",
      "postDate": "09/24/2017 12:25:29",
      "content": "<p>Can we use the expiration date in the members.csv? Is it future information?</p>",
      "rawMarkdown": "Can we use the expiration date in the members.csv? Is it future information?",
      "votes": null
    },
    {
      "id": "224073",
      "postDate": "09/25/2017 02:56:28",
      "content": "<p>The expiration date in the members.csv is a snapshot of our member table. Hence it is possible to contain future information, but it may not give you much useful information. Say a user made a two-year term subscription on 2017-03-15. We will have the membership expiration in member.csv set to 2019-05-15, however, this does not mean the user will not have other transaction between those two dates (2017-03-15- 2019-03-15).</p>",
      "rawMarkdown": "The expiration date in the members.csv is a snapshot of our member table. Hence it is possible to contain future information, but it may not give you much useful information. Say a user made a two-year term subscription on 2017-03-15. We will have the membership expiration in member.csv set to 2019-05-15, however, this does not mean the user will not have other transaction between those two dates (2017-03-15- 2019-03-15).",
      "votes": null
    },
    {
      "id": "224076",
      "postDate": "09/25/2017 03:17:22",
      "content": "<p>Sorry, my bad, it is a typo. Please refer to the data page for the correct dates.</p>",
      "rawMarkdown": "Sorry, my bad, it is a typo. Please refer to the data page for the correct dates.",
      "votes": null
    },
    {
      "id": "224077",
      "postDate": "09/25/2017 03:18:50",
      "content": "<p>One reminder, we did make a filter on the expiration date associated with each transaction. We removed the entries that have expiration date &gt; 2017-03-31.</p>",
      "rawMarkdown": "One reminder, we did make a filter on the expiration date associated with each transaction. We removed the entries that have expiration date &gt; 2017-03-31.",
      "votes": null
    },
    {
      "id": "224097",
      "postDate": "09/25/2017 05:28:48",
      "content": "<p>So the question is, can we use the expiration date in the members.csv? If we use the information, do you regard it as a trick ?</p>",
      "rawMarkdown": "So the question is, can we use the expiration date in the members.csv? If we use the information, do you regard it as a trick ?",
      "votes": null
    },
    {
      "id": "224098",
      "postDate": "09/25/2017 05:39:52",
      "content": "<p>I found it confusing that the same user id has different \"expiration data\" in members.csv and transaction.csv, such as \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\".</p>",
      "rawMarkdown": "I found it confusing that the same user id has different \"expiration data\" in members.csv and transaction.csv, such as \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\".",
      "votes": null
    },
    {
      "id": "224108",
      "postDate": "09/25/2017 06:23:37",
      "content": "<p>I still have some questions. Like the latest expiration date in February of the user 'QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' in training set is 20170221 and he has a transaction record occured on 20170224. But the label of this user is churn.</p>\n\n<p>And if the latest expiration date in Jan. of the user is 20170104, 30 days means 20170104-20170202 or 20170105-20170203? \n@ardenkkbox</p>",
      "rawMarkdown": "I still have some questions. Like the latest expiration date in February of the user 'QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' in training set is 20170221 and he has a transaction record occured on 20170224. But the label of this user is churn.\n\nAnd if the latest expiration date in Jan. of the user is 20170104, 30 days means 20170104-20170202 or 20170105-20170203? \n@ardenkkbox",
      "votes": null
    },
    {
      "id": "224186",
      "postDate": "09/25/2017 13:32:17",
      "content": "<p>There is a user with msno='QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' and is_churn = 1 in the train.csv whose transaction log is\n(20161231,20170221,false) \n(20170131,20170321,false)\n(20170224,20170321,True)</p>\n\n<p>How can you make the user as a churned user, in the second entry,the user resubscribe and make the membership_expire_date = 20170321,</p>\n\n<p>Base on the offical churned user rules, if the user does not make a subsctribe between 20170321 and 20170420, this user is a churn user.\nBut the membership_expire_date = 20170321 doesn't fall in Feb.,so it should not be a sample in the data train. </p>",
      "rawMarkdown": "There is a user with msno='QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' and is_churn = 1 in the train.csv whose transaction log is\n(20161231,20170221,false) \n(20170131,20170321,false)\n(20170224,20170321,True)\n\nHow can you make the user as a churned user, in the second entry,the user resubscribe and make the membership_expire_date = 20170321,\n\nBase on the offical churned user rules, if the user does not make a subsctribe between 20170321 and 20170420, this user is a churn user.\nBut the membership_expire_date = 20170321 doesn't fall in Feb.,so it should not be a sample in the data train.",
      "votes": null
    },
    {
      "id": "224187",
      "postDate": "09/25/2017 13:39:11",
      "content": "<p>The ransaction record occured on 20170224 is a cancel record</p>",
      "rawMarkdown": "The ransaction record occured on 20170224 is a cancel record",
      "votes": null
    },
    {
      "id": "224314",
      "postDate": "09/25/2017 22:16:07",
      "content": "<p>If that is the case, do you consider this user as churn or not churn? What is your evidence or churn since you do not have more information about the future transaction? I found user \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\" in train set has expiration data 20170907 in members but 20170206 in train set.</p>",
      "rawMarkdown": "If that is the case, do you consider this user as churn or not churn? What is your evidence or churn since you do not have more information about the future transaction? I found user \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\" in train set has expiration data 20170907 in members but 20170206 in train set.",
      "votes": null
    },
    {
      "id": "224746",
      "postDate": "09/27/2017 14:39:25",
      "content": "<p>should we treat him as a churned one?But he renewed his subscription on 20170131 to 20170321 which confuses me.</p>",
      "rawMarkdown": "should we treat him as a churned one?But he renewed his subscription on 20170131 to 20170321 which confuses me.",
      "votes": null
    },
    {
      "id": "226865",
      "postDate": "10/03/2017 08:23:52",
      "content": "<p>I understood the good point. We can generate more training set from \"transactions.csv\".\nHowever, there are some cases that are difficult to understand.</p>\n\n<p>For example msno = 'K+/OWGHAidYXaVfZ+gxABCaQH1xJFWUAPvYyJHNgL9s='</p>\n\n<ul>\n<li>payment_plan_days, transaction_date, membership_expire_date, is_cancel</li>\n<li>30, 20170123, 20170323, 0</li>\n<li>30, 20170122, 20170221, 0</li>\n<li>30, 20161221, 20170120, 0</li>\n</ul>\n\n<p>This user add a month on January 22 and 23, and expire on March 23 after two months.\nIn this case, however, it is defined as an churn user.</p>",
      "rawMarkdown": "I understood the good point. We can generate more training set from \"transactions.csv\".\nHowever, there are some cases that are difficult to understand.\n\nFor example msno = 'K+/OWGHAidYXaVfZ+gxABCaQH1xJFWUAPvYyJHNgL9s='\n\n - payment_plan_days, transaction_date, membership_expire_date, is_cancel\n - 30, 20170123, 20170323, 0\n - 30, 20170122, 20170221, 0\n - 30, 20161221, 20170120, 0\n\nThis user add a month on January 22 and 23, and expire on March 23 after two months.\nIn this case, however, it is defined as an churn user.",
      "votes": null
    },
    {
      "id": "230731",
      "postDate": "10/12/2017 15:56:33",
      "content": "<p>Dear Arden Chiu, Why do you remove the entries that have expiration date &gt; 2017-03-31 ? it is comfusing</p>",
      "rawMarkdown": "Dear Arden Chiu, Why do you remove the entries that have expiration date &gt; 2017-03-31 ? it is comfusing",
      "votes": null
    },
    {
      "id": "230879",
      "postDate": "10/13/2017 02:15:03",
      "content": "<p>Hi</p>",
      "rawMarkdown": "Hi",
      "votes": null
    },
    {
      "id": "230882",
      "postDate": "10/13/2017 02:22:47",
      "content": "<p>you can use the expire date in member.csv. It is not a trick. Bear in mind that the date in member.csv is not as accurate as the ones in transaction.csv.</p>",
      "rawMarkdown": "you can use the expire date in member.csv. It is not a trick. Bear in mind that the date in member.csv is not as accurate as the ones in transaction.csv.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 223077,
      "author_name": "extremin",
      "author_url": "",
      "post_date": "09/21/2017 03:25:48",
      "content": "<p>I also found that the max expiration_date of a user in the training set ranged from 20170201 to 20170331, however, the max expiration_date of a user in the test set ranged from 2015 to 2017. This is so weird.</p>",
      "votes": null,
      "replies": [
        {
          "id": 223363,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/22/2017 03:02:12",
          "content": "<p>The provided training set is only serves as an example. Our data set is imbalanced, refer to <a href=\"https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda\">https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda</a>). In this dataset, we includ more users behaviors than the ones in train and test datasets, in order to enable participants to explore different user behaviors outside of the train and test sets. For example, a user could actively cancel the subscription, but renew 30 days after the previous cancellation. We encourage participants to discover new patterns/features from the transaction log and generate your own set of training data.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 223549,
          "author_name": "snowdog",
          "author_url": "",
          "post_date": "09/22/2017 14:39:18",
          "content": "<p>This is very very confusing... you're saying that the 'train' dataset is not derived from 'transaction'? So we have '2' training sets? One was given (train.csv), the other one we should generate (from transactions.csv) ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 223662,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/23/2017 00:36:00",
          "content": "<p>The provided training data set is derived from transaction log. We picked the users who have their expiration dates fall in Feb, 2017 and check whether those people renew their subscription with 30 days after expiration to generate training label. Our method is not the only way to generate the training data. The training data set can be generate using different logic. Say, you can check each user's transaction log and calculate the interval between two consecutive entries. In this case, you will generate a training data set much bigger than what we provided in the data section.  </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 223697,
          "author_name": "soundwaveli00",
          "author_url": "",
          "post_date": "09/23/2017 03:38:52",
          "content": "<p>As you mentioned </p>\n\n<blockquote>\n  <p>We picked the users who have their expiration dates fall in Feb, 2017</p>\n</blockquote>\n\n<p>As extremin mentioned</p>\n\n<blockquote>\n  <p>max expiration_date of a user in the training set ranged from 20170201 to 20170331</p>\n</blockquote>\n\n<p>Does it mean some users in provided train dataset must have an expiration fall in Feb and and other expiration(in transactions file) fall in Mar?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 226865,
          "author_name": "junyoung",
          "author_url": "",
          "post_date": "10/03/2017 08:23:52",
          "content": "<p>I understood the good point. We can generate more training set from \"transactions.csv\".\nHowever, there are some cases that are difficult to understand.</p>\n\n<p>For example msno = 'K+/OWGHAidYXaVfZ+gxABCaQH1xJFWUAPvYyJHNgL9s='</p>\n\n<ul>\n<li>payment_plan_days, transaction_date, membership_expire_date, is_cancel</li>\n<li>30, 20170123, 20170323, 0</li>\n<li>30, 20170122, 20170221, 0</li>\n<li>30, 20161221, 20170120, 0</li>\n</ul>\n\n<p>This user add a month on January 22 and 23, and expire on March 23 after two months.\nIn this case, however, it is defined as an churn user.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 223243,
      "author_name": "kylechung",
      "author_url": "",
      "post_date": "09/21/2017 15:28:30",
      "content": "<p>I think one of my general question is:</p>\n\n<p>What is the relationship between the <code>membership_expire_date</code> in <code>transactions</code> and <code>expiration_date</code> in <code>members</code>? Because it seems that <code>expiration_date</code> can not be derived from <code>transactions</code>, are they indeed referring to totally different concept of expiration?</p>\n\n<p>One possible reason I can think of is the <code>members</code> data are derived from transaction records more (newer) than those in <code>transactions</code>, so surely the <code>expiration_date</code> will be newer than those recorded in <code>transactions</code>. But still I'll appreciate any official details about how we should reason these dataset before we can deliver meaningful prediction. :)</p>",
      "votes": null,
      "replies": [
        {
          "id": 223663,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/23/2017 00:40:27",
          "content": "<p>The expiration_date in members table is the membership_expire_date in the last entry for a user in the transaction log.  For the purpose of this competition, we filter out all transactions occurred after March 2017 in the transaction log.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 223358,
      "author_name": "ardenkkbox",
      "author_url": "",
      "post_date": "09/22/2017 02:25:43",
      "content": "<p>The membership data is indeed derived using transaction data. We deliberately removed the transaction logs after 20170331. Even you see that a member expiration date of 2017-12-18 in membership table. This date does not imply that the subscriber stopped his/her subscription during the month of March-April, 2017.</p>",
      "votes": null,
      "replies": [
        {
          "id": 223693,
          "author_name": "soundwaveli00",
          "author_url": "",
          "post_date": "09/23/2017 03:24:30",
          "content": "<p>In data page, it said </p>\n\n<blockquote>\n  <p>transactions of users up until 2/28/2017.</p>\n</blockquote>\n\n<p>Is \"removed the transaction logs after 20170331\" a typo in your reply?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 223954,
          "author_name": "huoyangyang",
          "author_url": "",
          "post_date": "09/24/2017 12:25:29",
          "content": "<p>Can we use the expiration date in the members.csv? Is it future information?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224073,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/25/2017 02:56:28",
          "content": "<p>The expiration date in the members.csv is a snapshot of our member table. Hence it is possible to contain future information, but it may not give you much useful information. Say a user made a two-year term subscription on 2017-03-15. We will have the membership expiration in member.csv set to 2019-05-15, however, this does not mean the user will not have other transaction between those two dates (2017-03-15- 2019-03-15).</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224076,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/25/2017 03:17:22",
          "content": "<p>Sorry, my bad, it is a typo. Please refer to the data page for the correct dates.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224077,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "09/25/2017 03:18:50",
          "content": "<p>One reminder, we did make a filter on the expiration date associated with each transaction. We removed the entries that have expiration date &gt; 2017-03-31.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224097,
          "author_name": "huoyangyang",
          "author_url": "",
          "post_date": "09/25/2017 05:28:48",
          "content": "<p>So the question is, can we use the expiration date in the members.csv? If we use the information, do you regard it as a trick ?</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224186,
          "author_name": "huoyangyang",
          "author_url": "",
          "post_date": "09/25/2017 13:32:17",
          "content": "<p>There is a user with msno='QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' and is_churn = 1 in the train.csv whose transaction log is\n(20161231,20170221,false) \n(20170131,20170321,false)\n(20170224,20170321,True)</p>\n\n<p>How can you make the user as a churned user, in the second entry,the user resubscribe and make the membership_expire_date = 20170321,</p>\n\n<p>Base on the offical churned user rules, if the user does not make a subsctribe between 20170321 and 20170420, this user is a churn user.\nBut the membership_expire_date = 20170321 doesn't fall in Feb.,so it should not be a sample in the data train. </p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224314,
          "author_name": "fangmingyuyang",
          "author_url": "",
          "post_date": "09/25/2017 22:16:07",
          "content": "<p>If that is the case, do you consider this user as churn or not churn? What is your evidence or churn since you do not have more information about the future transaction? I found user \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\" in train set has expiration data 20170907 in members but 20170206 in train set.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 230731,
          "author_name": "huichengxiao",
          "author_url": "",
          "post_date": "10/12/2017 15:56:33",
          "content": "<p>Dear Arden Chiu, Why do you remove the entries that have expiration date &gt; 2017-03-31 ? it is comfusing</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 230879,
          "author_name": "huichengxiao",
          "author_url": "",
          "post_date": "10/13/2017 02:15:03",
          "content": "<p>Hi</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 230882,
          "author_name": "ardenkkbox",
          "author_url": "",
          "post_date": "10/13/2017 02:22:47",
          "content": "<p>you can use the expire date in member.csv. It is not a trick. Bear in mind that the date in member.csv is not as accurate as the ones in transaction.csv.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 223953,
      "author_name": "huoyangyang",
      "author_url": "",
      "post_date": "09/24/2017 12:24:42",
      "content": "",
      "votes": null,
      "replies": []
    },
    {
      "id": 224098,
      "author_name": "fangmingyuyang",
      "author_url": "",
      "post_date": "09/25/2017 05:39:52",
      "content": "<p>I found it confusing that the same user id has different \"expiration data\" in members.csv and transaction.csv, such as \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\".</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 224108,
      "author_name": "extremin",
      "author_url": "",
      "post_date": "09/25/2017 06:23:37",
      "content": "<p>I still have some questions. Like the latest expiration date in February of the user 'QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' in training set is 20170221 and he has a transaction record occured on 20170224. But the label of this user is churn.</p>\n\n<p>And if the latest expiration date in Jan. of the user is 20170104, 30 days means 20170104-20170202 or 20170105-20170203? \n@ardenkkbox</p>",
      "votes": null,
      "replies": [
        {
          "id": 224187,
          "author_name": "huoyangyang",
          "author_url": "",
          "post_date": "09/25/2017 13:39:11",
          "content": "<p>The ransaction record occured on 20170224 is a cancel record</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 224746,
          "author_name": "yxfish13",
          "author_url": "",
          "post_date": "09/27/2017 14:39:25",
          "content": "<p>should we treat him as a churned one?But he renewed his subscription on 20170131 to 20170321 which confuses me.</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "222919": "I found the `transactions` dataset a little bit hard to reason.\n\nFor example, take a look at the user `IwE/pih8PuqrY/rsnoZ/4TazDliyH9S8VWNc2/d7mJg=` in the `train` set, the related records in `transactions` look like there are multiple records within a day and multiple associated `membership_expire_date`, ranging across 3 years (2015-2017).\n\nWhat's more, in `members` this user has a `expiration_date` of `2017-12-18`, which can not be found in any of the associated transaction record.\n\nIt will be appreciated that we can have an official elaboration about this.",
    "223077": "I also found that the max expiration_date of a user in the training set ranged from 20170201 to 20170331, however, the max expiration_date of a user in the test set ranged from 2015 to 2017. This is so weird.",
    "223243": "I think one of my general question is:\n\nWhat is the relationship between the `membership_expire_date` in `transactions` and `expiration_date` in `members`? Because it seems that `expiration_date` can not be derived from `transactions`, are they indeed referring to totally different concept of expiration?\n\nOne possible reason I can think of is the `members` data are derived from transaction records more (newer) than those in `transactions`, so surely the `expiration_date` will be newer than those recorded in `transactions`. But still I'll appreciate any official details about how we should reason these dataset before we can deliver meaningful prediction. :)",
    "223358": "The membership data is indeed derived using transaction data. We deliberately removed the transaction logs after 20170331. Even you see that a member expiration date of 2017-12-18 in membership table. This date does not imply that the subscriber stopped his/her subscription during the month of March-April, 2017.",
    "223363": "The provided training set is only serves as an example. Our data set is imbalanced, refer to https://www.kaggle.com/headsortails/should-i-stay-or-should-i-go-kkbox-eda). In this dataset, we includ more users behaviors than the ones in train and test datasets, in order to enable participants to explore different user behaviors outside of the train and test sets. For example, a user could actively cancel the subscription, but renew 30 days after the previous cancellation. We encourage participants to discover new patterns/features from the transaction log and generate your own set of training data.",
    "223549": "This is very very confusing... you're saying that the 'train' dataset is not derived from 'transaction'? So we have '2' training sets? One was given (train.csv), the other one we should generate (from transactions.csv) ?",
    "223662": "The provided training data set is derived from transaction log. We picked the users who have their expiration dates fall in Feb, 2017 and check whether those people renew their subscription with 30 days after expiration to generate training label. Our method is not the only way to generate the training data. The training data set can be generate using different logic. Say, you can check each user's transaction log and calculate the interval between two consecutive entries. In this case, you will generate a training data set much bigger than what we provided in the data section.",
    "223663": "The expiration_date in members table is the membership_expire_date in the last entry for a user in the transaction log.  For the purpose of this competition, we filter out all transactions occurred after March 2017 in the transaction log.",
    "223693": "In data page, it said \n\n&gt; transactions of users up until 2/28/2017.\n\nIs \"removed the transaction logs after 20170331\" a typo in your reply?",
    "223697": "As you mentioned \n\n&gt; We picked the users who have their expiration dates fall in Feb, 2017\n\nAs extremin mentioned\n\n&gt; max expiration_date of a user in the training set ranged from 20170201 to 20170331\n\nDoes it mean some users in provided train dataset must have an expiration fall in Feb and and other expiration(in transactions file) fall in Mar?",
    "223953": "",
    "223954": "Can we use the expiration date in the members.csv? Is it future information?",
    "224073": "The expiration date in the members.csv is a snapshot of our member table. Hence it is possible to contain future information, but it may not give you much useful information. Say a user made a two-year term subscription on 2017-03-15. We will have the membership expiration in member.csv set to 2019-05-15, however, this does not mean the user will not have other transaction between those two dates (2017-03-15- 2019-03-15).",
    "224076": "Sorry, my bad, it is a typo. Please refer to the data page for the correct dates.",
    "224077": "One reminder, we did make a filter on the expiration date associated with each transaction. We removed the entries that have expiration date &gt; 2017-03-31.",
    "224097": "So the question is, can we use the expiration date in the members.csv? If we use the information, do you regard it as a trick ?",
    "224098": "I found it confusing that the same user id has different \"expiration data\" in members.csv and transaction.csv, such as \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\".",
    "224108": "I still have some questions. Like the latest expiration date in February of the user 'QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' in training set is 20170221 and he has a transaction record occured on 20170224. But the label of this user is churn.\n\nAnd if the latest expiration date in Jan. of the user is 20170104, 30 days means 20170104-20170202 or 20170105-20170203? \n@ardenkkbox",
    "224186": "There is a user with msno='QA7uiXy8vIbUSPOkCf9RwQ3FsT8jVq2OxDr8zqa7bRQ=' and is_churn = 1 in the train.csv whose transaction log is\n(20161231,20170221,false) \n(20170131,20170321,false)\n(20170224,20170321,True)\n\nHow can you make the user as a churned user, in the second entry,the user resubscribe and make the membership_expire_date = 20170321,\n\nBase on the offical churned user rules, if the user does not make a subsctribe between 20170321 and 20170420, this user is a churn user.\nBut the membership_expire_date = 20170321 doesn't fall in Feb.,so it should not be a sample in the data train.",
    "224187": "The ransaction record occured on 20170224 is a cancel record",
    "224314": "If that is the case, do you consider this user as churn or not churn? What is your evidence or churn since you do not have more information about the future transaction? I found user \"waLDQMmcOu2jLDaV1ddDkgCrB/jl6sD66Xzs0Vqax1Y=\" in train set has expiration data 20170907 in members but 20170206 in train set.",
    "224746": "should we treat him as a churned one?But he renewed his subscription on 20170131 to 20170321 which confuses me.",
    "226865": "I understood the good point. We can generate more training set from \"transactions.csv\".\nHowever, there are some cases that are difficult to understand.\n\nFor example msno = 'K+/OWGHAidYXaVfZ+gxABCaQH1xJFWUAPvYyJHNgL9s='\n\n - payment_plan_days, transaction_date, membership_expire_date, is_cancel\n - 30, 20170123, 20170323, 0\n - 30, 20170122, 20170221, 0\n - 30, 20161221, 20170120, 0\n\nThis user add a month on January 22 and 23, and expire on March 23 after two months.\nIn this case, however, it is defined as an churn user.",
    "230731": "Dear Arden Chiu, Why do you remove the entries that have expiration date &gt; 2017-03-31 ? it is comfusing",
    "230879": "Hi",
    "230882": "you can use the expire date in member.csv. It is not a trick. Bear in mind that the date in member.csv is not as accurate as the ones in transaction.csv."
  },
  "source": "meta"
}