{
  "id": 67860,
  "title": "Multiple observations per (object_id, passband, mjd) triplet",
  "url": "/competitions/PLAsTiCC-2018/discussion/67860",
  "author_name": "",
  "post_date": "2018-10-06T13:03:22.070497700Z",
  "votes": null,
  "comment_count": 4,
  "views": 0,
  "content": "<p>There seems to be some (<code>object_id</code>, <code>passband</code>, <code>mjd</code>) triplets that have multiple observations. This is quite easy to observe.</p>\n\n<pre><code>train = pd.read_csv('data/training_set.csv')\nlcs.query('object_id == 730 and passband == 2')\\\n   .plot(x='mjd', y='flux', kind='scatter')\n</code></pre>\n\n<p><a href=\"https://imgur.com/a/utYEgAn\">https://imgur.com/a/utYEgAn</a></p>\n\n<p>Most of you have probably already noticed this. From what I understood the <em>in silico</em> telescope which is simulating the data can take multiple measurements per day. I'm guessing that this is an issue because the detail of the timestamp (the <code>mja</code> column) is too coarse. In other words we don't have the hour and minute of when the observation took place.</p>\n\n<p>Am I missing something? I was going to average the data per day but I want to make I haven't missed something.</p>",
  "messages": [
    {
      "id": "399671",
      "postDate": "10/06/2018 13:03:22",
      "content": "<p>There seems to be some (<code>object_id</code>, <code>passband</code>, <code>mjd</code>) triplets that have multiple observations. This is quite easy to observe.</p>\n\n<pre><code>train = pd.read_csv('data/training_set.csv')\nlcs.query('object_id == 730 and passband == 2')\\\n   .plot(x='mjd', y='flux', kind='scatter')\n</code></pre>\n\n<p><a href=\"https://imgur.com/a/utYEgAn\">https://imgur.com/a/utYEgAn</a></p>\n\n<p>Most of you have probably already noticed this. From what I understood the <em>in silico</em> telescope which is simulating the data can take multiple measurements per day. I'm guessing that this is an issue because the detail of the timestamp (the <code>mja</code> column) is too coarse. In other words we don't have the hour and minute of when the observation took place.</p>\n\n<p>Am I missing something? I was going to average the data per day but I want to make I haven't missed something.</p>",
      "rawMarkdown": "There seems to be some (`object_id`, `passband`, `mjd`) triplets that have multiple observations. This is quite easy to observe.\n\n    train = pd.read_csv('data/training_set.csv')\n    lcs.query('object_id == 730 and passband == 2')\\\n       .plot(x='mjd', y='flux', kind='scatter')\n\n[https://imgur.com/a/utYEgAn](https://imgur.com/a/utYEgAn)\n\nMost of you have probably already noticed this. From what I understood the *in silico* telescope which is simulating the data can take multiple measurements per day. I'm guessing that this is an issue because the detail of the timestamp (the `mja` column) is too coarse. In other words we don't have the hour and minute of when the observation took place.\n\nAm I missing something? I was going to average the data per day but I want to make I haven't missed something.",
      "votes": null
    },
    {
      "id": "399694",
      "postDate": "10/06/2018 14:06:07",
      "content": "<p>taking that average per day on the test set might take awhile, let us know how it goes</p>",
      "rawMarkdown": "taking that average per day on the test set might take awhile, let us know how it goes",
      "votes": null
    },
    {
      "id": "399763",
      "postDate": "10/06/2018 17:56:36",
      "content": "<p>Both the real telescope and the simulations can take multiple measurements per day. <code>mjd</code> is in units of days and shows these quantities to 4 places of decimals. 0.001 day is roughly 14 seconds. So, you should be able to get hour and minute of the observation, but not seconds or milli-seconds.  This was thought to be quite reasonable,  we also have a fixed exposure time that is already greater than 14 seconds for all the observations.   </p>\n\n<p>But you are correct in that you don't have arbitrary precision and have not missed something there!</p>\n\n<ul>\n<li>For the PLAsTiCC team.</li>\n</ul>",
      "rawMarkdown": "Both the real telescope and the simulations can take multiple measurements per day. `mjd` is in units of days and shows these quantities to 4 places of decimals. 0.001 day is roughly 14 seconds. So, you should be able to get hour and minute of the observation, but not seconds or milli-seconds.  This was thought to be quite reasonable,  we also have a fixed exposure time that is already greater than 14 seconds for all the observations.   \n\nBut you are correct in that you don't have arbitrary precision and have not missed something there!\n\n- For the PLAsTiCC team.",
      "votes": null
    },
    {
      "id": "399765",
      "postDate": "10/06/2018 18:05:15",
      "content": "<p>Thanks for the clarification.</p>",
      "rawMarkdown": "Thanks for the clarification.",
      "votes": null
    },
    {
      "id": "404156",
      "postDate": "10/15/2018 10:45:14",
      "content": "<p><a href=\"/rbiswas4\">@rbiswas4</a>, do I understand correctly that there is a single fixed exposure time for all measurements in train and test?</p>",
      "rawMarkdown": "rbiswas4, do I understand correctly that there is a single fixed exposure time for all measurements in train and test?",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 399694,
      "author_name": "abednadir",
      "author_url": "",
      "post_date": "10/06/2018 14:06:07",
      "content": "<p>taking that average per day on the test set might take awhile, let us know how it goes</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 399763,
      "author_name": "rbiswas4",
      "author_url": "",
      "post_date": "10/06/2018 17:56:36",
      "content": "<p>Both the real telescope and the simulations can take multiple measurements per day. <code>mjd</code> is in units of days and shows these quantities to 4 places of decimals. 0.001 day is roughly 14 seconds. So, you should be able to get hour and minute of the observation, but not seconds or milli-seconds.  This was thought to be quite reasonable,  we also have a fixed exposure time that is already greater than 14 seconds for all the observations.   </p>\n\n<p>But you are correct in that you don't have arbitrary precision and have not missed something there!</p>\n\n<ul>\n<li>For the PLAsTiCC team.</li>\n</ul>",
      "votes": null,
      "replies": [
        {
          "id": 399765,
          "author_name": "maxhalford",
          "author_url": "",
          "post_date": "10/06/2018 18:05:15",
          "content": "<p>Thanks for the clarification.</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 404156,
          "author_name": "alexfir",
          "author_url": "",
          "post_date": "10/15/2018 10:45:14",
          "content": "<p><a href=\"/rbiswas4\">@rbiswas4</a>, do I understand correctly that there is a single fixed exposure time for all measurements in train and test?</p>",
          "votes": null,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "399671": "There seems to be some (`object_id`, `passband`, `mjd`) triplets that have multiple observations. This is quite easy to observe.\n\n    train = pd.read_csv('data/training_set.csv')\n    lcs.query('object_id == 730 and passband == 2')\\\n       .plot(x='mjd', y='flux', kind='scatter')\n\n[https://imgur.com/a/utYEgAn](https://imgur.com/a/utYEgAn)\n\nMost of you have probably already noticed this. From what I understood the *in silico* telescope which is simulating the data can take multiple measurements per day. I'm guessing that this is an issue because the detail of the timestamp (the `mja` column) is too coarse. In other words we don't have the hour and minute of when the observation took place.\n\nAm I missing something? I was going to average the data per day but I want to make I haven't missed something.",
    "399694": "taking that average per day on the test set might take awhile, let us know how it goes",
    "399763": "Both the real telescope and the simulations can take multiple measurements per day. `mjd` is in units of days and shows these quantities to 4 places of decimals. 0.001 day is roughly 14 seconds. So, you should be able to get hour and minute of the observation, but not seconds or milli-seconds.  This was thought to be quite reasonable,  we also have a fixed exposure time that is already greater than 14 seconds for all the observations.   \n\nBut you are correct in that you don't have arbitrary precision and have not missed something there!\n\n- For the PLAsTiCC team.",
    "399765": "Thanks for the clarification.",
    "404156": "rbiswas4, do I understand correctly that there is a single fixed exposure time for all measurements in train and test?"
  },
  "source": "meta"
}