{
  "id": 71949,
  "title": "A recurrent neural network for classification of unevenly sampled variable stars",
  "url": "/competitions/PLAsTiCC-2018/discussion/71949",
  "author_name": "",
  "post_date": "2018-11-18T17:17:03.568836300Z",
  "votes": 22,
  "comment_count": 16,
  "views": 0,
  "content": "<p>Something I want to try: <a href=\"https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper\">https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper</a></p>\n\n<p>A <a href=\"https://arxiv.org/abs/1711.10609\">link to the paper</a> was shared on the forum I think.</p>\n\n<p>This uses LSTM on light curves to build an auto encoder or a classifier.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/423603/10690/Screenshot_2018-11-18%201711%2010609%20pdf.png\" alt=\"rnn\"></p>",
  "messages": [
    {
      "id": "423603",
      "postDate": "11/18/2018 17:17:03",
      "content": "<p>Something I want to try: <a href=\"https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper\">https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper</a></p>\n\n<p>A <a href=\"https://arxiv.org/abs/1711.10609\">link to the paper</a> was shared on the forum I think.</p>\n\n<p>This uses LSTM on light curves to build an auto encoder or a classifier.</p>\n\n<p><img src=\"https://storage.googleapis.com/kaggle-forum-message-attachments/423603/10690/Screenshot_2018-11-18%201711%2010609%20pdf.png\" alt=\"rnn\"></p>",
      "rawMarkdown": "Something I want to try: https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper\n\nA [link to the paper][1] was shared on the forum I think.\n\nThis uses LSTM on light curves to build an auto encoder or a classifier.\n\n![rnn][2]\n\n\n  [1]: https://arxiv.org/abs/1711.10609\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/423603/10690/Screenshot_2018-11-18%201711%2010609%20pdf.png",
      "votes": null
    },
    {
      "id": "423628",
      "postDate": "11/18/2018 18:42:15",
      "content": "<p>Hi,  could I ask you what you used for your diagram?</p>",
      "rawMarkdown": "Hi,  could I ask you what you used for your diagram?",
      "votes": null
    },
    {
      "id": "423630",
      "postDate": "11/18/2018 18:43:23",
      "content": "<p>A screenshot of their paper ;)</p>",
      "rawMarkdown": "A screenshot of their paper ;)",
      "votes": null
    },
    {
      "id": "423636",
      "postDate": "11/18/2018 18:56:52",
      "content": "<p>Ah OK - they did a nice job didn't they ;) - I think you might be better doing an autoencoder based on just test due to the small amount of training data - good luck! </p>",
      "rawMarkdown": "Ah OK - they did a nice job didn't they ;) - I think you might be better doing an autoencoder based on just test due to the small amount of training data - good luck!",
      "votes": null
    },
    {
      "id": "423666",
      "postDate": "11/18/2018 20:52:06",
      "content": "<p>I had a good look at the paper as well. I think one of the main challenges of implementing this is that as they mentioned, period folding is still required to obtain sensible reconstructions for short period objects, so it might be necessary to deal with small-timescale periodic cases separately. Also multiband input is more tricky than their single-band input, but masking should be sufficient.</p>",
      "rawMarkdown": "I had a good look at the paper as well. I think one of the main challenges of implementing this is that as they mentioned, period folding is still required to obtain sensible reconstructions for short period objects, so it might be necessary to deal with small-timescale periodic cases separately. Also multiband input is more tricky than their single-band input, but masking should be sufficient.",
      "votes": null
    },
    {
      "id": "423718",
      "postDate": "11/18/2018 23:23:03",
      "content": "<blockquote>\n  <p>period folding is still required to obtain sensible reconstructions</p>\n</blockquote>\n\n<p>Right, this is why I did not try it until now.</p>",
      "rawMarkdown": "&gt; period folding is still required to obtain sensible reconstructions\n\nRight, this is why I did not try it until now.",
      "votes": null
    },
    {
      "id": "424565",
      "postDate": "11/20/2018 10:23:51",
      "content": "<p>I tried to implement this model as it was described in the paper and it failed miserably. But that is probably because it was my second model ever that I wrote in Pytorch.</p>",
      "rawMarkdown": "I tried to implement this model as it was described in the paper and it failed miserably. But that is probably because it was my second model ever that I wrote in Pytorch.",
      "votes": null
    },
    {
      "id": "425046",
      "postDate": "11/21/2018 03:55:02",
      "content": "<p>Were you able to get any non-random reconstructions? I had a similar model and I was able to recover some of the curve shape from the latent variables. However it is impossible for it to work with short-term periodic objects because too many local optima.</p>",
      "rawMarkdown": "Were you able to get any non-random reconstructions? I had a similar model and I was able to recover some of the curve shape from the latent variables. However it is impossible for it to work with short-term periodic objects because too many local optima.",
      "votes": null
    },
    {
      "id": "435219",
      "postDate": "12/07/2018 17:49:25",
      "content": "<p>I was able to get this to work reasonably well, but it still only gave a small boost to my best model (like  .01-.015). i think FE can capture most of the information that an autoencoder would pull</p>",
      "rawMarkdown": "I was able to get this to work reasonably well, but it still only gave a small boost to my best model (like  .01-.015). i think FE can capture most of the information that an autoencoder would pull",
      "votes": null
    },
    {
      "id": "435227",
      "postDate": "12/07/2018 17:56:33",
      "content": "<p>i would be interested to see how you made this work, even after the competition ends.</p>",
      "rawMarkdown": "i would be interested to see how you made this work, even after the competition ends.",
      "votes": null
    },
    {
      "id": "435286",
      "postDate": "12/07/2018 19:52:36",
      "content": "<p>I use transformations of flux instead of the actual flux. I can share more later!</p>",
      "rawMarkdown": "I use transformations of flux instead of the actual flux. I can share more later!",
      "votes": null
    },
    {
      "id": "435409",
      "postDate": "12/08/2018 01:59:10",
      "content": "<p>This is similar to what I have found, although I do believe tree models often have issue making use of dense features like embeddings. My autoencode does give me some decent looking tSNE though.\n<img src=\"https://i.imgur.com/36HL16Z.png\" alt=\"tSNE\"></p>",
      "rawMarkdown": "This is similar to what I have found, although I do believe tree models often have issue making use of dense features like embeddings. My autoencode does give me some decent looking tSNE though.\n![tSNE][1]\n\n\n  [1]: https://i.imgur.com/36HL16Z.png",
      "votes": null
    },
    {
      "id": "436733",
      "postDate": "12/10/2018 20:54:07",
      "content": "<p>Thank you for the thread! I didn't follow the paper exactly but it is immensely helpful!</p>",
      "rawMarkdown": "Thank you for the thread! I didn't follow the paper exactly but it is immensely helpful!",
      "votes": null
    },
    {
      "id": "436931",
      "postDate": "12/11/2018 05:37:02",
      "content": "<p>Thanks.  I hope you'll share how you used it after competition end!</p>",
      "rawMarkdown": "Thanks.  I hope you'll share how you used it after competition end!",
      "votes": null
    },
    {
      "id": "436933",
      "postDate": "12/11/2018 05:37:36",
      "content": "<p>sure, I will have a writeup. good luck!</p>",
      "rawMarkdown": "sure, I will have a writeup. good luck!",
      "votes": null
    },
    {
      "id": "439074",
      "postDate": "12/14/2018 17:32:24",
      "content": "<p>I found out that I failed to train my model because I didn't turn off dropout while making predictions. I'm learning PyTorch the hard way, so to speak</p>",
      "rawMarkdown": "I found out that I failed to train my model because I didn't turn off dropout while making predictions. I'm learning PyTorch the hard way, so to speak",
      "votes": null
    },
    {
      "id": "518343",
      "postDate": "04/17/2019 05:29:45",
      "content": "<p>Difference between scaling and transformation for a dataset. \ni want to know the difference between the MinMax transformation and log transformation.</p>\n\n<p>should both be applied and when one should be applied , also what does these transformation do to the dataset? \nim working with time series dataset.</p>",
      "rawMarkdown": "Difference between scaling and transformation for a dataset. \ni want to know the difference between the MinMax transformation and log transformation.\n\nshould both be applied and when one should be applied , also what does these transformation do to the dataset? \nim working with time series dataset.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 423628,
      "author_name": "scirpus",
      "author_url": "",
      "post_date": "11/18/2018 18:42:15",
      "content": "<p>Hi,  could I ask you what you used for your diagram?</p>",
      "votes": null,
      "replies": [
        {
          "id": 423630,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "11/18/2018 18:43:23",
          "content": "<p>A screenshot of their paper ;)</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 423636,
          "author_name": "scirpus",
          "author_url": "",
          "post_date": "11/18/2018 18:56:52",
          "content": "<p>Ah OK - they did a nice job didn't they ;) - I think you might be better doing an autoencoder based on just test due to the small amount of training data - good luck! </p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 423666,
      "author_name": "mithrillion",
      "author_url": "",
      "post_date": "11/18/2018 20:52:06",
      "content": "<p>I had a good look at the paper as well. I think one of the main challenges of implementing this is that as they mentioned, period folding is still required to obtain sensible reconstructions for short period objects, so it might be necessary to deal with small-timescale periodic cases separately. Also multiband input is more tricky than their single-band input, but masking should be sufficient.</p>",
      "votes": null,
      "replies": [
        {
          "id": 423718,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "11/18/2018 23:23:03",
          "content": "<blockquote>\n  <p>period folding is still required to obtain sensible reconstructions</p>\n</blockquote>\n\n<p>Right, this is why I did not try it until now.</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 424565,
      "author_name": "gennadylaptev",
      "author_url": "",
      "post_date": "11/20/2018 10:23:51",
      "content": "<p>I tried to implement this model as it was described in the paper and it failed miserably. But that is probably because it was my second model ever that I wrote in Pytorch.</p>",
      "votes": null,
      "replies": [
        {
          "id": 425046,
          "author_name": "mithrillion",
          "author_url": "",
          "post_date": "11/21/2018 03:55:02",
          "content": "<p>Were you able to get any non-random reconstructions? I had a similar model and I was able to recover some of the curve shape from the latent variables. However it is impossible for it to work with short-term periodic objects because too many local optima.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 439074,
          "author_name": "gennadylaptev",
          "author_url": "",
          "post_date": "12/14/2018 17:32:24",
          "content": "<p>I found out that I failed to train my model because I didn't turn off dropout while making predictions. I'm learning PyTorch the hard way, so to speak</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 435219,
      "author_name": "rhgrossm",
      "author_url": "",
      "post_date": "12/07/2018 17:49:25",
      "content": "<p>I was able to get this to work reasonably well, but it still only gave a small boost to my best model (like  .01-.015). i think FE can capture most of the information that an autoencoder would pull</p>",
      "votes": null,
      "replies": [
        {
          "id": 435227,
          "author_name": "iprapas",
          "author_url": "",
          "post_date": "12/07/2018 17:56:33",
          "content": "<p>i would be interested to see how you made this work, even after the competition ends.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435286,
          "author_name": "rhgrossm",
          "author_url": "",
          "post_date": "12/07/2018 19:52:36",
          "content": "<p>I use transformations of flux instead of the actual flux. I can share more later!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 435409,
          "author_name": "mithrillion",
          "author_url": "",
          "post_date": "12/08/2018 01:59:10",
          "content": "<p>This is similar to what I have found, although I do believe tree models often have issue making use of dense features like embeddings. My autoencode does give me some decent looking tSNE though.\n<img src=\"https://i.imgur.com/36HL16Z.png\" alt=\"tSNE\"></p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 436733,
      "author_name": "jiweiliu",
      "author_url": "",
      "post_date": "12/10/2018 20:54:07",
      "content": "<p>Thank you for the thread! I didn't follow the paper exactly but it is immensely helpful!</p>",
      "votes": null,
      "replies": [
        {
          "id": 436931,
          "author_name": "cpmpml",
          "author_url": "",
          "post_date": "12/11/2018 05:37:02",
          "content": "<p>Thanks.  I hope you'll share how you used it after competition end!</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 436933,
          "author_name": "jiweiliu",
          "author_url": "",
          "post_date": "12/11/2018 05:37:36",
          "content": "<p>sure, I will have a writeup. good luck!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 518343,
      "author_name": "kunalbasnet",
      "author_url": "",
      "post_date": "04/17/2019 05:29:45",
      "content": "<p>Difference between scaling and transformation for a dataset. \ni want to know the difference between the MinMax transformation and log transformation.</p>\n\n<p>should both be applied and when one should be applied , also what does these transformation do to the dataset? \nim working with time series dataset.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "423603": "Something I want to try: https://github.com/bnaul/IrregularTimeSeriesAutoencoderPaper\n\nA [link to the paper][1] was shared on the forum I think.\n\nThis uses LSTM on light curves to build an auto encoder or a classifier.\n\n![rnn][2]\n\n\n  [1]: https://arxiv.org/abs/1711.10609\n  [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/423603/10690/Screenshot_2018-11-18%201711%2010609%20pdf.png",
    "423628": "Hi,  could I ask you what you used for your diagram?",
    "423630": "A screenshot of their paper ;)",
    "423636": "Ah OK - they did a nice job didn't they ;) - I think you might be better doing an autoencoder based on just test due to the small amount of training data - good luck!",
    "423666": "I had a good look at the paper as well. I think one of the main challenges of implementing this is that as they mentioned, period folding is still required to obtain sensible reconstructions for short period objects, so it might be necessary to deal with small-timescale periodic cases separately. Also multiband input is more tricky than their single-band input, but masking should be sufficient.",
    "423718": "&gt; period folding is still required to obtain sensible reconstructions\n\nRight, this is why I did not try it until now.",
    "424565": "I tried to implement this model as it was described in the paper and it failed miserably. But that is probably because it was my second model ever that I wrote in Pytorch.",
    "425046": "Were you able to get any non-random reconstructions? I had a similar model and I was able to recover some of the curve shape from the latent variables. However it is impossible for it to work with short-term periodic objects because too many local optima.",
    "435219": "I was able to get this to work reasonably well, but it still only gave a small boost to my best model (like  .01-.015). i think FE can capture most of the information that an autoencoder would pull",
    "435227": "i would be interested to see how you made this work, even after the competition ends.",
    "435286": "I use transformations of flux instead of the actual flux. I can share more later!",
    "435409": "This is similar to what I have found, although I do believe tree models often have issue making use of dense features like embeddings. My autoencode does give me some decent looking tSNE though.\n![tSNE][1]\n\n\n  [1]: https://i.imgur.com/36HL16Z.png",
    "436733": "Thank you for the thread! I didn't follow the paper exactly but it is immensely helpful!",
    "436931": "Thanks.  I hope you'll share how you used it after competition end!",
    "436933": "sure, I will have a writeup. good luck!",
    "439074": "I found out that I failed to train my model because I didn't turn off dropout while making predictions. I'm learning PyTorch the hard way, so to speak",
    "518343": "Difference between scaling and transformation for a dataset. \ni want to know the difference between the MinMax transformation and log transformation.\n\nshould both be applied and when one should be applied , also what does these transformation do to the dataset? \nim working with time series dataset."
  },
  "source": "meta"
}