{
  "id": 72646,
  "title": "All passbands estimations from autoencoder",
  "url": "/competitions/PLAsTiCC-2018/discussion/72646",
  "author_name": "hklee",
  "post_date": "2018-11-25T18:57:52.093000",
  "votes": 25,
  "comment_count": 25,
  "views": 0,
  "content": "<p>Here are results from my implementation of an autoencoder which consists of three layers of bidirectional LSTMs with preceding passband embedding. I am not sure of accuracies of these curves without saying their meanings. </p>\n\n<h2>target 6</h2>\n\n<p><img src=\"https://i.imgur.com/twgFcWs.png\" alt=\"target 6\">\n<img src=\"https://i.imgur.com/uVrUHKj.png\" alt=\"target 6\"></p>\n\n<h2>target 15</h2>\n\n<p><img src=\"https://i.imgur.com/5w43EjX.png\" alt=\"target 15\">\n<img src=\"https://i.imgur.com/52lOjF6.png\" alt=\"target 15\"></p>\n\n<h2>target 16</h2>\n\n<p><img src=\"https://i.imgur.com/Z3u3glp.png\" alt=\"target 16\">\n<img src=\"https://i.imgur.com/u3BQirj.png\" alt=\"target 16\"></p>\n\n<h2>target 42</h2>\n\n<p><img src=\"https://i.imgur.com/jm3icMe.png\" alt=\"target 42\">\n<img src=\"https://i.imgur.com/ISW9uQz.png\" alt=\"target 42\"></p>\n\n<h2>target 52</h2>\n\n<p><img src=\"https://i.imgur.com/KITkaRH.png\" alt=\"target 52\">\n<img src=\"https://i.imgur.com/xQ6KcFz.png\" alt=\"target 52\"></p>\n\n<h2>target 53</h2>\n\n<p><img src=\"https://i.imgur.com/XdHO20N.png\" alt=\"target 53\">\n<img src=\"https://i.imgur.com/hiUaRJp.png\" alt=\"target 53\"></p>\n\n<h2>target 62</h2>\n\n<p><img src=\"https://i.imgur.com/uBuNpaN.png\" alt=\"target 62\">\n<img src=\"https://i.imgur.com/DW9cJSf.png\" alt=\"target 62\"></p>\n\n<h2>target 64</h2>\n\n<p><img src=\"https://i.imgur.com/yg0BqiD.png\" alt=\"target 64\">\n<img src=\"https://i.imgur.com/t0tbCF0.png\" alt=\"target 64\"></p>\n\n<h2>target 65</h2>\n\n<p><img src=\"https://i.imgur.com/OhAW4zN.png\" alt=\"target 65\">\n<img src=\"https://i.imgur.com/Pv6DZVD.png\" alt=\"target 65\"></p>\n\n<h2>target 67</h2>\n\n<p><img src=\"https://i.imgur.com/4p87BCM.png\" alt=\"target 67\">\n<img src=\"https://i.imgur.com/MaRHrgc.png\" alt=\"target 67\"></p>\n\n<h2>target 88</h2>\n\n<p><img src=\"https://i.imgur.com/wwEQFSU.png\" alt=\"target 88\">\n<img src=\"https://i.imgur.com/ZLekRoy.png\" alt=\"target 88\"></p>\n\n<h2>target 90</h2>\n\n<p><img src=\"https://i.imgur.com/FgELfO9.png\" alt=\"target 90\">\n<img src=\"https://i.imgur.com/t5oVvLd.png\" alt=\"target 90\"></p>\n\n<h2>target 92</h2>\n\n<p><img src=\"https://i.imgur.com/Nq8E4fr.png\" alt=\"target 92\">\n<img src=\"https://i.imgur.com/BynImgo.png\" alt=\"target 92\"></p>\n\n<h2>target 95</h2>\n\n<p><img src=\"https://i.imgur.com/blM1KEQ.png\" alt=\"target 95\">\n<img src=\"https://i.imgur.com/y2tvnnd.png\" alt=\"target 95\"></p>",
  "messages": [
    {
      "id": 427576,
      "postDate": "2018-11-25T18:57:52.093Z",
      "content": "<p>Here are results from my implementation of an autoencoder which consists of three layers of bidirectional LSTMs with preceding passband embedding. I am not sure of accuracies of these curves without saying their meanings. </p>\n\n<h2>target 6</h2>\n\n<p><img src=\"https://i.imgur.com/twgFcWs.png\" alt=\"target 6\">\n<img src=\"https://i.imgur.com/uVrUHKj.png\" alt=\"target 6\"></p>\n\n<h2>target 15</h2>\n\n<p><img src=\"https://i.imgur.com/5w43EjX.png\" alt=\"target 15\">\n<img src=\"https://i.imgur.com/52lOjF6.png\" alt=\"target 15\"></p>\n\n<h2>target 16</h2>\n\n<p><img src=\"https://i.imgur.com/Z3u3glp.png\" alt=\"target 16\">\n<img src=\"https://i.imgur.com/u3BQirj.png\" alt=\"target 16\"></p>\n\n<h2>target 42</h2>\n\n<p><img src=\"https://i.imgur.com/jm3icMe.png\" alt=\"target 42\">\n<img src=\"https://i.imgur.com/ISW9uQz.png\" alt=\"target 42\"></p>\n\n<h2>target 52</h2>\n\n<p><img src=\"https://i.imgur.com/KITkaRH.png\" alt=\"target 52\">\n<img src=\"https://i.imgur.com/xQ6KcFz.png\" alt=\"target 52\"></p>\n\n<h2>target 53</h2>\n\n<p><img src=\"https://i.imgur.com/XdHO20N.png\" alt=\"target 53\">\n<img src=\"https://i.imgur.com/hiUaRJp.png\" alt=\"target 53\"></p>\n\n<h2>target 62</h2>\n\n<p><img src=\"https://i.imgur.com/uBuNpaN.png\" alt=\"target 62\">\n<img src=\"https://i.imgur.com/DW9cJSf.png\" alt=\"target 62\"></p>\n\n<h2>target 64</h2>\n\n<p><img src=\"https://i.imgur.com/yg0BqiD.png\" alt=\"target 64\">\n<img src=\"https://i.imgur.com/t0tbCF0.png\" alt=\"target 64\"></p>\n\n<h2>target 65</h2>\n\n<p><img src=\"https://i.imgur.com/OhAW4zN.png\" alt=\"target 65\">\n<img src=\"https://i.imgur.com/Pv6DZVD.png\" alt=\"target 65\"></p>\n\n<h2>target 67</h2>\n\n<p><img src=\"https://i.imgur.com/4p87BCM.png\" alt=\"target 67\">\n<img src=\"https://i.imgur.com/MaRHrgc.png\" alt=\"target 67\"></p>\n\n<h2>target 88</h2>\n\n<p><img src=\"https://i.imgur.com/wwEQFSU.png\" alt=\"target 88\">\n<img src=\"https://i.imgur.com/ZLekRoy.png\" alt=\"target 88\"></p>\n\n<h2>target 90</h2>\n\n<p><img src=\"https://i.imgur.com/FgELfO9.png\" alt=\"target 90\">\n<img src=\"https://i.imgur.com/t5oVvLd.png\" alt=\"target 90\"></p>\n\n<h2>target 92</h2>\n\n<p><img src=\"https://i.imgur.com/Nq8E4fr.png\" alt=\"target 92\">\n<img src=\"https://i.imgur.com/BynImgo.png\" alt=\"target 92\"></p>\n\n<h2>target 95</h2>\n\n<p><img src=\"https://i.imgur.com/blM1KEQ.png\" alt=\"target 95\">\n<img src=\"https://i.imgur.com/y2tvnnd.png\" alt=\"target 95\"></p>",
      "rawMarkdown": "Here are results from my implementation of an autoencoder which consists of three layers of bidirectional LSTMs with preceding passband embedding. I am not sure of accuracies of these curves without saying their meanings. \n\n## target 6\n![target 6][1]\n![target 6][2]\n## target 15\n![target 15][3]\n![target 15][4]\n## target 16\n![target 16][5]\n![target 16][6]\n## target 42\n![target 42][7]\n![target 42][8]\n## target 52\n![target 52][9]\n![target 52][10]\n## target 53\n![target 53][11]\n![target 53][12]\n## target 62\n![target 62][13]\n![target 62][14]\n## target 64\n![target 64][15]\n![target 64][16]\n## target 65\n![target 65][17]\n![target 65][18]\n## target 67\n![target 67][19]\n![target 67][20]\n## target 88\n![target 88][21]\n![target 88][22]\n## target 90\n![target 90][23]\n![target 90][24]\n## target 92\n![target 92][25]\n![target 92][26]\n## target 95\n![target 95][27]\n![target 95][28]\n\n\n  [1]: https://i.imgur.com/twgFcWs.png\n  [2]: https://i.imgur.com/uVrUHKj.png\n  [3]: https://i.imgur.com/5w43EjX.png\n  [4]: https://i.imgur.com/52lOjF6.png\n  [5]: https://i.imgur.com/Z3u3glp.png\n  [6]: https://i.imgur.com/u3BQirj.png\n  [7]: https://i.imgur.com/jm3icMe.png\n  [8]: https://i.imgur.com/ISW9uQz.png\n  [9]: https://i.imgur.com/KITkaRH.png\n  [10]: https://i.imgur.com/xQ6KcFz.png\n  [11]: https://i.imgur.com/XdHO20N.png\n  [12]: https://i.imgur.com/hiUaRJp.png\n  [13]: https://i.imgur.com/uBuNpaN.png\n  [14]: https://i.imgur.com/DW9cJSf.png\n  [15]: https://i.imgur.com/yg0BqiD.png\n  [16]: https://i.imgur.com/t0tbCF0.png\n  [17]: https://i.imgur.com/OhAW4zN.png\n  [18]: https://i.imgur.com/Pv6DZVD.png\n  [19]: https://i.imgur.com/4p87BCM.png\n  [20]: https://i.imgur.com/MaRHrgc.png\n  [21]: https://i.imgur.com/wwEQFSU.png\n  [22]: https://i.imgur.com/ZLekRoy.png\n  [23]: https://i.imgur.com/FgELfO9.png\n  [24]: https://i.imgur.com/t5oVvLd.png\n  [25]: https://i.imgur.com/Nq8E4fr.png\n  [26]: https://i.imgur.com/BynImgo.png\n  [27]: https://i.imgur.com/blM1KEQ.png\n  [28]: https://i.imgur.com/y2tvnnd.png",
      "votes": 25
    },
    {
      "id": 428878,
      "postDate": "2018-11-28T02:47:52.280Z",
      "content": "<p>Wondering how long it need to train the autoencoder model?</p>",
      "rawMarkdown": "Wondering how long it need to train the autoencoder model?",
      "votes": 1
    },
    {
      "id": 427589,
      "postDate": "2018-11-25T19:40:42.980Z",
      "content": "<p>Are these your reconstructions? They look pretty decent to me, but it always helps to plot against the original points. Is your autoencoder able to generate a full curve (with arbitrary number of points)?\nAlso, you used embeddings to get around multiple bands, but how are you getting around the unevenly-spaced input problem?\nHere are some of the experiments I had with autoencoders.</p>\n\n<p><img src=\"https://i.imgur.com/8DhyPix.png\" alt=\"Reconstruction\"></p>",
      "rawMarkdown": "Are these your reconstructions? They look pretty decent to me, but it always helps to plot against the original points. Is your autoencoder able to generate a full curve (with arbitrary number of points)?\nAlso, you used embeddings to get around multiple bands, but how are you getting around the unevenly-spaced input problem?\nHere are some of the experiments I had with autoencoders.\n\n![Reconstruction][1]\n\n\n  [1]: https://i.imgur.com/8DhyPix.png",
      "votes": 1,
      "replies": [
        {
          "id": 427715,
          "postDate": "2018-11-26T02:29:29.683Z",
          "content": "<p>This looks great.  Does it help your modeling?</p>",
          "rawMarkdown": "This looks great.  Does it help your modeling?"
        },
        {
          "id": 427718,
          "postDate": "2018-11-26T02:39:19.443Z",
          "content": "<p>It adds 4-10 useful features to LGB, but I think I need to scale down the network for better representations as the representation dimension might be too large, because it is overfitting some <strong>validation</strong> examples. </p>",
          "rawMarkdown": "It adds 4-10 useful features to LGB, but I think I need to scale down the network for better representations as the representation dimension might be too large, because it is overfitting some **validation** examples. "
        },
        {
          "id": 427788,
          "postDate": "2018-11-26T06:19:43.043Z",
          "content": "<p>I use time differences between adjacent measurements as inputs to rnn with some proper scale transformations/normalizations. I modeled the output of rnn will be proportional to time difference like linear regression. That is, if time interval is zero, rnn should emit current state unchanged.</p>",
          "rawMarkdown": "I use time differences between adjacent measurements as inputs to rnn with some proper scale transformations/normalizations. I modeled the output of rnn will be proportional to time difference like linear regression. That is, if time interval is zero, rnn should emit current state unchanged.",
          "votes": 1
        },
        {
          "id": 429960,
          "postDate": "2018-11-29T15:46:57.653Z",
          "content": "<p>@Mithrillion I am wondering how do you infer missing value here for your input time series? I saw your kenerl that simply fill all missing value as 0. Do you still using the same method for your autoencoder?</p>",
          "rawMarkdown": "@Mithrillion I am wondering how do you infer missing value here for your input time series? I saw your kenerl that simply fill all missing value as 0. Do you still using the same method for your autoencoder?"
        },
        {
          "id": 429998,
          "postDate": "2018-11-29T16:29:22.897Z",
          "content": "<p>There are many techniques you can use to deal with missing values, zero-padding is the first one I tried but not the best. The problem with padding + masking is the network needs a lot of examples to learn the relationship between value missingness and mask. It is convenient to think of missing values as similar to the 'edge' or 'holes' in a picture, and there exists some techniques to deal with it, e.g. <a href=\"https://github.com/NVIDIA/partialconv\">https://github.com/NVIDIA/partialconv</a> . I'm still experimenting with different techniques to see which works best.</p>",
          "rawMarkdown": "There are many techniques you can use to deal with missing values, zero-padding is the first one I tried but not the best. The problem with padding + masking is the network needs a lot of examples to learn the relationship between value missingness and mask. It is convenient to think of missing values as similar to the 'edge' or 'holes' in a picture, and there exists some techniques to deal with it, e.g. https://github.com/NVIDIA/partialconv . I'm still experimenting with different techniques to see which works best.",
          "votes": 1
        },
        {
          "id": 430533,
          "postDate": "2018-11-30T15:00:34.910Z",
          "content": "<p>May I ask do you use any supernova model to fit these data or you just fit the data by a simple model? Does it take long time to fit each source? Thanks!</p>",
          "rawMarkdown": "May I ask do you use any supernova model to fit these data or you just fit the data by a simple model? Does it take long time to fit each source? Thanks!"
        },
        {
          "id": 430554,
          "postDate": "2018-11-30T15:22:18.183Z",
          "content": "<p>I modifed some of the network layers to deal with the missing values / uneven spacing of observations, but it did not use supernova-specific models. The training speed varies a lot as you change the layers. Typically an RNN will be much slower than straight forward passes or CNNs. I'm still looking for designs that are both fast and fit the data well.</p>",
          "rawMarkdown": "I modifed some of the network layers to deal with the missing values / uneven spacing of observations, but it did not use supernova-specific models. The training speed varies a lot as you change the layers. Typically an RNN will be much slower than straight forward passes or CNNs. I'm still looking for designs that are both fast and fit the data well.",
          "votes": 1
        },
        {
          "id": 431664,
          "postDate": "2018-12-02T17:46:25.443Z",
          "content": "<p>Hi, @Mithrillion wondering if you know some good tutorial for implementation an autoencoder for extracting feature from sequence data. I found my implementation always have problems</p>",
          "rawMarkdown": "Hi, @Mithrillion wondering if you know some good tutorial for implementation an autoencoder for extracting feature from sequence data. I found my implementation always have problems"
        },
        {
          "id": 432215,
          "postDate": "2018-12-03T15:05:47.293Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 432217,
          "postDate": "2018-12-03T15:07:05.017Z",
          "content": "<p>I tried to use Gaussian Process. It will be very smooth. But it will spend too much time if you want to use it on the test set. </p>",
          "rawMarkdown": "I tried to use Gaussian Process. It will be very smooth. But it will spend too much time if you want to use it on the test set. "
        },
        {
          "id": 438110,
          "postDate": "2018-12-13T05:36:22.973Z",
          "content": "<p>@Mithrillion, how did the use the reconstruction as feature?</p>",
          "rawMarkdown": "@Mithrillion, how did the use the reconstruction as feature?"
        },
        {
          "id": 438111,
          "postDate": "2018-12-13T05:44:31.360Z",
          "content": "<p>The latent variables are the obvious choice, but you could also try with reconstruction loss and VAE KL divergence.</p>",
          "rawMarkdown": "The latent variables are the obvious choice, but you could also try with reconstruction loss and VAE KL divergence."
        },
        {
          "id": 438112,
          "postDate": "2018-12-13T05:49:02.807Z",
          "content": "<p>did you use NN or RNN?</p>",
          "rawMarkdown": "did you use NN or RNN?"
        },
        {
          "id": 438114,
          "postDate": "2018-12-13T05:57:52.960Z",
          "content": "<p>CNN and RNN both work. The tricky part is to evaluate the reconstruction at exactly the points you want. RNN is easier but it is very hard to make predictions not depend on number of points to evaluate (vanishing hidden state).</p>",
          "rawMarkdown": "CNN and RNN both work. The tricky part is to evaluate the reconstruction at exactly the points you want. RNN is easier but it is very hard to make predictions not depend on number of points to evaluate (vanishing hidden state)."
        },
        {
          "id": 438116,
          "postDate": "2018-12-13T05:59:36.440Z",
          "content": "<p>i see. thanks! one last question, did you include the mjd in training data for the encoders?</p>",
          "rawMarkdown": "i see. thanks! one last question, did you include the mjd in training data for the encoders?"
        },
        {
          "id": 438121,
          "postDate": "2018-12-13T06:10:45.417Z",
          "content": "<p>It is actually a very tricky problem. You want your network to only care about how an event happens, not when, but you need to know when to properly reconstruct. You kind of need to pass partial time information to the network but also feed more complete time information to the network at a later stage.</p>",
          "rawMarkdown": "It is actually a very tricky problem. You want your network to only care about how an event happens, not when, but you need to know when to properly reconstruct. You kind of need to pass partial time information to the network but also feed more complete time information to the network at a later stage."
        },
        {
          "id": 438122,
          "postDate": "2018-12-13T06:15:14.160Z",
          "content": "<p>yea that is the reason i stopped the track of CNN a while ago. but now that im stuck with my score and no more ideas im trying to do it and see if i can improve even by a little. i dropped 30 ranks already</p>",
          "rawMarkdown": "yea that is the reason i stopped the track of CNN a while ago. but now that im stuck with my score and no more ideas im trying to do it and see if i can improve even by a little. i dropped 30 ranks already"
        }
      ]
    },
    {
      "id": 427771,
      "postDate": "2018-11-26T05:58:14.457Z",
      "content": "<p>Regular time interval (highnoon and midnight, twice a day) estimations from autoencoder model. I see lots of spikes which may indicate I need some regularizer enforcing continuous results.</p>\n\n<p><img src=\"https://i.imgur.com/mopR4ye.png\" alt=\"target 6\">\n<img src=\"https://i.imgur.com/fekYexg.png\" alt=\"target 42\">\n<img src=\"https://i.imgur.com/0GpxZBR.png\" alt=\"target 53\">\n<img src=\"https://i.imgur.com/Nn5Eai8.png\" alt=\"target 88\"></p>",
      "rawMarkdown": "Regular time interval (highnoon and midnight, twice a day) estimations from autoencoder model. I see lots of spikes which may indicate I need some regularizer enforcing continuous results.\n\n![target 6][1]\n![target 42][2]\n![target 53][3]\n![target 88][4]\n\n\n  [1]: https://i.imgur.com/mopR4ye.png \"target 6\"\n  [2]: https://i.imgur.com/fekYexg.png\n  [3]: https://i.imgur.com/0GpxZBR.png\n  [4]: https://i.imgur.com/Nn5Eai8.png",
      "votes": 2
    },
    {
      "id": 430231,
      "postDate": "2018-11-30T03:34:04.983Z",
      "content": "<p>Nice work recreating these light curves.  Any luck with using it as a feature extractor?</p>",
      "rawMarkdown": "Nice work recreating these light curves.  Any luck with using it as a feature extractor?"
    },
    {
      "id": 427714,
      "postDate": "2018-11-26T02:28:36.053Z",
      "content": "<p>This looks great.  Does it help your modeling?</p>",
      "rawMarkdown": "This looks great.  Does it help your modeling?",
      "replies": [
        {
          "id": 427779,
          "postDate": "2018-11-26T06:05:59.790Z",
          "content": "<p>Not yet. I will use it as a feature extractor for the classifier. </p>",
          "rawMarkdown": "Not yet. I will use it as a feature extractor for the classifier. "
        },
        {
          "id": 430616,
          "postDate": "2018-11-30T18:07:26.973Z",
          "content": "<p>THELW NA PETHANEIS</p>",
          "rawMarkdown": "THELW NA PETHANEIS",
          "votes": -2
        },
        {
          "id": 435787,
          "postDate": "2018-12-08T19:17:04.630Z",
          "content": "<p>So did it help your model ? you seem to have a big jump</p>",
          "rawMarkdown": "So did it help your model ? you seem to have a big jump"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 428878,
      "author_name": "Strideradu",
      "author_url": "",
      "post_date": "2018-11-28T02:47:52.280000",
      "content": "<p>Wondering how long it need to train the autoencoder model?</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 427589,
      "author_name": "Mithrillion",
      "author_url": "",
      "post_date": "2018-11-25T19:40:42.980000",
      "content": "<p>Are these your reconstructions? They look pretty decent to me, but it always helps to plot against the original points. Is your autoencoder able to generate a full curve (with arbitrary number of points)?\nAlso, you used embeddings to get around multiple bands, but how are you getting around the unevenly-spaced input problem?\nHere are some of the experiments I had with autoencoders.</p>\n\n<p><img src=\"https://i.imgur.com/8DhyPix.png\" alt=\"Reconstruction\"></p>",
      "votes": 1,
      "replies": [
        {
          "id": 427715,
          "author_name": "CPMP",
          "author_url": "",
          "post_date": "2018-11-26T02:29:29.683000",
          "content": "<p>This looks great.  Does it help your modeling?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 427718,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-11-26T02:39:19.443000",
          "content": "<p>It adds 4-10 useful features to LGB, but I think I need to scale down the network for better representations as the representation dimension might be too large, because it is overfitting some <strong>validation</strong> examples. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 427788,
          "author_name": "hklee",
          "author_url": "",
          "post_date": "2018-11-26T06:19:43.043000",
          "content": "<p>I use time differences between adjacent measurements as inputs to rnn with some proper scale transformations/normalizations. I modeled the output of rnn will be proportional to time difference like linear regression. That is, if time interval is zero, rnn should emit current state unchanged.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 429960,
          "author_name": "Strideradu",
          "author_url": "",
          "post_date": "2018-11-29T15:46:57.653000",
          "content": "<p>@Mithrillion I am wondering how do you infer missing value here for your input time series? I saw your kenerl that simply fill all missing value as 0. Do you still using the same method for your autoencoder?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 429998,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-11-29T16:29:22.897000",
          "content": "<p>There are many techniques you can use to deal with missing values, zero-padding is the first one I tried but not the best. The problem with padding + masking is the network needs a lot of examples to learn the relationship between value missingness and mask. It is convenient to think of missing values as similar to the 'edge' or 'holes' in a picture, and there exists some techniques to deal with it, e.g. <a href=\"https://github.com/NVIDIA/partialconv\">https://github.com/NVIDIA/partialconv</a> . I'm still experimenting with different techniques to see which works best.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 430533,
          "author_name": "Ningxiao Zhang",
          "author_url": "",
          "post_date": "2018-11-30T15:00:34.910000",
          "content": "<p>May I ask do you use any supernova model to fit these data or you just fit the data by a simple model? Does it take long time to fit each source? Thanks!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 430554,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-11-30T15:22:18.183000",
          "content": "<p>I modifed some of the network layers to deal with the missing values / uneven spacing of observations, but it did not use supernova-specific models. The training speed varies a lot as you change the layers. Typically an RNN will be much slower than straight forward passes or CNNs. I'm still looking for designs that are both fast and fit the data well.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 431664,
          "author_name": "Strideradu",
          "author_url": "",
          "post_date": "2018-12-02T17:46:25.443000",
          "content": "<p>Hi, @Mithrillion wondering if you know some good tutorial for implementation an autoencoder for extracting feature from sequence data. I found my implementation always have problems</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 432215,
          "author_name": "",
          "author_url": "",
          "post_date": "2018-12-03T15:05:47.293000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 432217,
          "author_name": "Ningxiao Zhang",
          "author_url": "",
          "post_date": "2018-12-03T15:07:05.017000",
          "content": "<p>I tried to use Gaussian Process. It will be very smooth. But it will spend too much time if you want to use it on the test set. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438110,
          "author_name": "dylonLL",
          "author_url": "",
          "post_date": "2018-12-13T05:36:22.973000",
          "content": "<p>@Mithrillion, how did the use the reconstruction as feature?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438111,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-12-13T05:44:31.360000",
          "content": "<p>The latent variables are the obvious choice, but you could also try with reconstruction loss and VAE KL divergence.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438112,
          "author_name": "dylonLL",
          "author_url": "",
          "post_date": "2018-12-13T05:49:02.807000",
          "content": "<p>did you use NN or RNN?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438114,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-12-13T05:57:52.960000",
          "content": "<p>CNN and RNN both work. The tricky part is to evaluate the reconstruction at exactly the points you want. RNN is easier but it is very hard to make predictions not depend on number of points to evaluate (vanishing hidden state).</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438116,
          "author_name": "dylonLL",
          "author_url": "",
          "post_date": "2018-12-13T05:59:36.440000",
          "content": "<p>i see. thanks! one last question, did you include the mjd in training data for the encoders?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438121,
          "author_name": "Mithrillion",
          "author_url": "",
          "post_date": "2018-12-13T06:10:45.417000",
          "content": "<p>It is actually a very tricky problem. You want your network to only care about how an event happens, not when, but you need to know when to properly reconstruct. You kind of need to pass partial time information to the network but also feed more complete time information to the network at a later stage.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 438122,
          "author_name": "dylonLL",
          "author_url": "",
          "post_date": "2018-12-13T06:15:14.160000",
          "content": "<p>yea that is the reason i stopped the track of CNN a while ago. but now that im stuck with my score and no more ideas im trying to do it and see if i can improve even by a little. i dropped 30 ranks already</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 427771,
      "author_name": "hklee",
      "author_url": "",
      "post_date": "2018-11-26T05:58:14.457000",
      "content": "<p>Regular time interval (highnoon and midnight, twice a day) estimations from autoencoder model. I see lots of spikes which may indicate I need some regularizer enforcing continuous results.</p>\n\n<p><img src=\"https://i.imgur.com/mopR4ye.png\" alt=\"target 6\">\n<img src=\"https://i.imgur.com/fekYexg.png\" alt=\"target 42\">\n<img src=\"https://i.imgur.com/0GpxZBR.png\" alt=\"target 53\">\n<img src=\"https://i.imgur.com/Nn5Eai8.png\" alt=\"target 88\"></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 430231,
      "author_name": "Paul M",
      "author_url": "",
      "post_date": "2018-11-30T03:34:04.983000",
      "content": "<p>Nice work recreating these light curves.  Any luck with using it as a feature extractor?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 427714,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2018-11-26T02:28:36.053000",
      "content": "<p>This looks great.  Does it help your modeling?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 427779,
          "author_name": "hklee",
          "author_url": "",
          "post_date": "2018-11-26T06:05:59.790000",
          "content": "<p>Not yet. I will use it as a feature extractor for the classifier. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 430616,
          "author_name": "Vasilis Stamatopoulos",
          "author_url": "",
          "post_date": "2018-11-30T18:07:26.973000",
          "content": "<p>THELW NA PETHANEIS</p>",
          "votes": -2,
          "replies": []
        },
        {
          "id": 435787,
          "author_name": "Blonde",
          "author_url": "",
          "post_date": "2018-12-08T19:17:04.630000",
          "content": "<p>So did it help your model ? you seem to have a big jump</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "427576": "Here are results from my implementation of an autoencoder which consists of three layers of bidirectional LSTMs with preceding passband embedding. I am not sure of accuracies of these curves without saying their meanings. \n\n## target 6\n![target 6][1]\n![target 6][2]\n## target 15\n![target 15][3]\n![target 15][4]\n## target 16\n![target 16][5]\n![target 16][6]\n## target 42\n![target 42][7]\n![target 42][8]\n## target 52\n![target 52][9]\n![target 52][10]\n## target 53\n![target 53][11]\n![target 53][12]\n## target 62\n![target 62][13]\n![target 62][14]\n## target 64\n![target 64][15]\n![target 64][16]\n## target 65\n![target 65][17]\n![target 65][18]\n## target 67\n![target 67][19]\n![target 67][20]\n## target 88\n![target 88][21]\n![target 88][22]\n## target 90\n![target 90][23]\n![target 90][24]\n## target 92\n![target 92][25]\n![target 92][26]\n## target 95\n![target 95][27]\n![target 95][28]\n\n\n  [1]: https://i.imgur.com/twgFcWs.png\n  [2]: https://i.imgur.com/uVrUHKj.png\n  [3]: https://i.imgur.com/5w43EjX.png\n  [4]: https://i.imgur.com/52lOjF6.png\n  [5]: https://i.imgur.com/Z3u3glp.png\n  [6]: https://i.imgur.com/u3BQirj.png\n  [7]: https://i.imgur.com/jm3icMe.png\n  [8]: https://i.imgur.com/ISW9uQz.png\n  [9]: https://i.imgur.com/KITkaRH.png\n  [10]: https://i.imgur.com/xQ6KcFz.png\n  [11]: https://i.imgur.com/XdHO20N.png\n  [12]: https://i.imgur.com/hiUaRJp.png\n  [13]: https://i.imgur.com/uBuNpaN.png\n  [14]: https://i.imgur.com/DW9cJSf.png\n  [15]: https://i.imgur.com/yg0BqiD.png\n  [16]: https://i.imgur.com/t0tbCF0.png\n  [17]: https://i.imgur.com/OhAW4zN.png\n  [18]: https://i.imgur.com/Pv6DZVD.png\n  [19]: https://i.imgur.com/4p87BCM.png\n  [20]: https://i.imgur.com/MaRHrgc.png\n  [21]: https://i.imgur.com/wwEQFSU.png\n  [22]: https://i.imgur.com/ZLekRoy.png\n  [23]: https://i.imgur.com/FgELfO9.png\n  [24]: https://i.imgur.com/t5oVvLd.png\n  [25]: https://i.imgur.com/Nq8E4fr.png\n  [26]: https://i.imgur.com/BynImgo.png\n  [27]: https://i.imgur.com/blM1KEQ.png\n  [28]: https://i.imgur.com/y2tvnnd.png",
    "428878": "Wondering how long it need to train the autoencoder model?",
    "427589": "Are these your reconstructions? They look pretty decent to me, but it always helps to plot against the original points. Is your autoencoder able to generate a full curve (with arbitrary number of points)?\nAlso, you used embeddings to get around multiple bands, but how are you getting around the unevenly-spaced input problem?\nHere are some of the experiments I had with autoencoders.\n\n![Reconstruction][1]\n\n\n  [1]: https://i.imgur.com/8DhyPix.png",
    "427771": "Regular time interval (highnoon and midnight, twice a day) estimations from autoencoder model. I see lots of spikes which may indicate I need some regularizer enforcing continuous results.\n\n![target 6][1]\n![target 42][2]\n![target 53][3]\n![target 88][4]\n\n\n  [1]: https://i.imgur.com/mopR4ye.png \"target 6\"\n  [2]: https://i.imgur.com/fekYexg.png\n  [3]: https://i.imgur.com/0GpxZBR.png\n  [4]: https://i.imgur.com/Nn5Eai8.png",
    "430231": "Nice work recreating these light curves.  Any luck with using it as a feature extractor?",
    "427714": "This looks great.  Does it help your modeling?"
  }
}