{
  "id": 10274,
  "title": "'Technical Artefacts' in some channels and data files for patient 1 ?",
  "url": "/competitions/seizure-prediction/discussion/10274",
  "author_name": "",
  "post_date": "2014-09-09T11:57:53.387Z",
  "votes": 1,
  "comment_count": 14,
  "views": 3014,
  "content": "<p>At least for the preictal datsets I see in some files some strange artefacts.</p>\n<p>I added a file with a screenshot.</p>\n<p>Can I get&nbsp; a conformation, that there are some strange artefacts ?</p>\n\n<p>A further option is, that something went wrong during a data conversion, which I performed. Since in most files everything looks fine, this explanation does not seem to have a high probability to me.</p>\n<p>Thank you,</p>\n<p>Matthias</p>",
  "messages": [
    {
      "id": "53385",
      "postDate": "09/09/2014 11:57:53",
      "content": "<p>At least for the preictal datsets I see in some files some strange artefacts.</p>\n<p>I added a file with a screenshot.</p>\n<p>Can I get&nbsp; a conformation, that there are some strange artefacts ?</p>\n\n<p>A further option is, that something went wrong during a data conversion, which I performed. Since in most files everything looks fine, this explanation does not seem to have a high probability to me.</p>\n<p>Thank you,</p>\n<p>Matthias</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53389",
      "postDate": "09/09/2014 13:14:29",
      "content": "<p>What artifacts do you see in the picture you attached? (I am not an expert in reading EEGs)</p>\n<p>Also what are the blue annotations you have overlaid over the graphs?</p>\n\n<p>Thanks!</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53391",
      "postDate": "09/09/2014 13:52:10",
      "content": "<p>Hmm ... that is interesting Matthias. Any chance you can tell us which part of which clips from Patient 1 you see these in so that one of us can try and cross-verify if the raw .mat files show the same behavior?</p>\n<p>@zzspar: I'm not sure about the blue annotations, but if you look at trace RD_7 near the bottom, for instance, you'll see what I would consider a fairly normal signal. That is, it oscillates fairly evenly between positive and negative values (kind of looks like a seismograph). Compare that to LD_4 / LD_5, where you see there seems to be a positive / negative cutoff.</p>\n<p>I'm not sure why this would be, but if you look at LD_4 and LD_5, it looks like the there seems to be some kind of inverse correlation between the signals...</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53392",
      "postDate": "09/09/2014 14:00:54",
      "content": "<p>The graphics is made with an EEG review program. The blue vertical line in the left is used to mark a time point and is of no meaning hear. On the bottom on the right you see a scaling.</p>\n<p>The blue horizontal line represent 2 s. The vertical line the voltage scaling. But here the absolute values are not correct (I did not find a conversion factor to Voltage in the description.</p>\n<p>The file is: patient_1_preictal_segment_0003</p>\n<p>The signals clearly look strange in channels</p>\n<p>LD_4</p>\n<p>LD_5</p>\n<p>RD_7</p>\n<p>RD_8</p>\n<p>But also to some exten LD_1 and LD3</p>\n<p>In some way it looks like a clipping of the signal.</p>\n\n<p>I attached her a screendump of</p>\n\n<p>patient_1_preictal_segment_0007</p>\n\n<p>where also the channels seem to be normal to me.</p>\n\n<p>Thanks for your interest and support,</p>\n\n<p>Matthias</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53393",
      "postDate": "09/09/2014 14:02:31",
      "content": "<p>I'll try to give it a look when I get home tonight and have access to the data on my hard drive (if no one else can do it before then).</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53442",
      "postDate": "09/10/2014 13:55:15",
      "content": "<p>[quote=MatthiasD;53392]</p>\n<p>The graphics is made with an EEG review program. [/quote]</p>\n\n<p>Hi Matthias,</p>\n<p>Which EEG (re)viewer are you using?</p>\n<p>Thanx,</p>\n<p>Jules</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53443",
      "postDate": "09/10/2014 14:59:32",
      "content": "<p>I have used the ASA software package (www.ant-neuro.com). To review the data I converted them to the edf (European Data Format), a kind of standard for EEG data.</p>\n\n<p>You can download a four weeks trial licence for the software.</p>\n<p>Bye,</p>\n<p>Matthias</p>\n\n<p>P.S. Small parts of the data can also be imported in an ASCII based format like shown below</p>\n\n<p>#Averaged Spiked Tue Jan 09 10:15:12 2001</p>\n<p>NumberPositions= 4</p>\n<p>NumberTimeSteps= 7</p>\n<p>UnitTime ms</p>\n<p>UnitMeas uV</p>\n<p>TimeSteps 0(3.906250)</p>\n<p>ValuesTransposed</p>\n<p>-1.273162 0.067602 3.475845 3.785685</p>\n<p>-1.329496 0.011267 3.554713 3.774418</p>\n<p>-0.997122 -0.242239 3.605414 3.965956</p>\n<p>-1.036557 -0.512645 3.571613 3.385709</p>\n<p>-1.228094 -0.743617 3.757517 3.042068</p>\n<p>-0.884453 -0.388709 4.275796 3.250506</p>\n<p>-0.377442 -0.157737 5.013779 3.740617</p>\n<p>Labels</p>\n<p>Fp1 Fp2 F3 F4</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53448",
      "postDate": "09/10/2014 15:43:42",
      "content": "<p>Matthias, my apologies, but I did not have the time last night to check the clip. Is there any chance you can upload that particular clip (the original '.mat' file) here?&nbsp;I'm not sure if it will be above the size limit of a file upload, but if you can I should have time to look at it some time today. Otherwise, I should have time tonight to look at it.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53449",
      "postDate": "09/10/2014 16:11:06",
      "content": "<p>I uploaded patient_1_preictal_segment_0003</p>\n<p>The signals clearly look strange in channels</p>\n<p>LD_4</p>\n<p>LD_5</p>\n<p>RD_7</p>\n<p>RD_8</p>\n<p>But also to some exten LD_1 and LD3</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53454",
      "postDate": "09/10/2014 16:55:44",
      "content": "<p>Hmm, I am not sure what to think. When I loaded the .mat into Python and plotted the first 10 seconds of data for LD_4 (down-sampled by a factor of 100), I got the attached image (LD_4-10sec-100FoldDownSample.png),&nbsp;which at least doesn't seem to have any clipping issues. Can you confirm that the plot you provided is definitely just the first 10 seconds of data, and not 10 minutes? When I try the whole clip (10 minutes) downsampled by a factor of 10000, I get the second image (LD_4-10min-10000FoldDownSample.png), which also seems to be fairly clipping free (though hard to say for sure due to the downsampling).</p>\n<p>Based on my lack of familiarity with ASA and the&nbsp;humble amount of experience I have in this area, I'm not sure what to tell you outside of two things:</p>\n<p>1) Whatever algorithm the program uses to process the data for display is resulting in the clipping (even in a topographical sense, our graphs look very different).</p>\n<p>2) There really is some kind of error going on when importing the file format?</p>\n<p>Sorry I wasn't able to be of more help :(</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53546",
      "postDate": "09/12/2014 08:50:18",
      "content": "<p>I looked again at the data in Matlab. Indeed there is some clipping due to a conversion to int16.</p>\n\n<p>Nevertheless the signal looks more like a slightly modulated 50 Hz sine wave than an EEG signal.</p>\n\n<p>Matthias</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53551",
      "postDate": "09/12/2014 11:43:23",
      "content": "<p>Attaches is the spectrum of the first 4096 samples of</p>\n<p>patient_1_preictal_segment_0003</p>\n<p>channel</p>\n<p>LD</p>\n<p>It shows a strong peak at 50 Hz or 60 Hz and smaller peaks for multiples of it.</p>\n\n<p>Also the screendumps from LAD do not look for me like EEG signals. It could only be hidden in the sinuoidal activity.</p>\n\n<p>Matthias</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53671",
      "postDate": "09/14/2014 01:51:12",
      "content": "<p>I have noticed that some channels of some samples have a great amount of line noise (60hz) and others have some DC interference (0-0.5hz). These are common artifacts in EEG readings taken from the scalp. I have no experience (outside of this competition) with iEEG data. Thus, I am not sure why some samples exhibit line noise, and others do not.</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53677",
      "postDate": "09/14/2014 03:42:01",
      "content": "<p>Well the dogs have implanted sensors that are read from a battery powered unit, the human patients have their recordings taken only during surgery and probably have a line powered unit doing the recording. would that account for the discrepancy ?</p>",
      "rawMarkdown": "",
      "votes": null
    },
    {
      "id": "53774",
      "postDate": "09/15/2014 11:21:54",
      "content": "<p>O.k. I will try to leave out channels/datasets with hughe artefact contaminations from training as fas as I am able to identify them.</p>\n\n<p>Matthias</p>",
      "rawMarkdown": "",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 53389,
      "author_name": "udibr1",
      "author_url": "",
      "post_date": "09/09/2014 13:14:29",
      "content": "<p>What artifacts do you see in the picture you attached? (I am not an expert in reading EEGs)</p>\n<p>Also what are the blue annotations you have overlaid over the graphs?</p>\n\n<p>Thanks!</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53391,
      "author_name": "lad215562",
      "author_url": "",
      "post_date": "09/09/2014 13:52:10",
      "content": "<p>Hmm ... that is interesting Matthias. Any chance you can tell us which part of which clips from Patient 1 you see these in so that one of us can try and cross-verify if the raw .mat files show the same behavior?</p>\n<p>@zzspar: I'm not sure about the blue annotations, but if you look at trace RD_7 near the bottom, for instance, you'll see what I would consider a fairly normal signal. That is, it oscillates fairly evenly between positive and negative values (kind of looks like a seismograph). Compare that to LD_4 / LD_5, where you see there seems to be a positive / negative cutoff.</p>\n<p>I'm not sure why this would be, but if you look at LD_4 and LD_5, it looks like the there seems to be some kind of inverse correlation between the signals...</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53392,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/09/2014 14:00:54",
      "content": "<p>The graphics is made with an EEG review program. The blue vertical line in the left is used to mark a time point and is of no meaning hear. On the bottom on the right you see a scaling.</p>\n<p>The blue horizontal line represent 2 s. The vertical line the voltage scaling. But here the absolute values are not correct (I did not find a conversion factor to Voltage in the description.</p>\n<p>The file is: patient_1_preictal_segment_0003</p>\n<p>The signals clearly look strange in channels</p>\n<p>LD_4</p>\n<p>LD_5</p>\n<p>RD_7</p>\n<p>RD_8</p>\n<p>But also to some exten LD_1 and LD3</p>\n<p>In some way it looks like a clipping of the signal.</p>\n\n<p>I attached her a screendump of</p>\n\n<p>patient_1_preictal_segment_0007</p>\n\n<p>where also the channels seem to be normal to me.</p>\n\n<p>Thanks for your interest and support,</p>\n\n<p>Matthias</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53393,
      "author_name": "lad215562",
      "author_url": "",
      "post_date": "09/09/2014 14:02:31",
      "content": "<p>I'll try to give it a look when I get home tonight and have access to the data on my hard drive (if no one else can do it before then).</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53442,
      "author_name": "julesvanligtenberg",
      "author_url": "",
      "post_date": "09/10/2014 13:55:15",
      "content": "<p>[quote=MatthiasD;53392]</p>\n<p>The graphics is made with an EEG review program. [/quote]</p>\n\n<p>Hi Matthias,</p>\n<p>Which EEG (re)viewer are you using?</p>\n<p>Thanx,</p>\n<p>Jules</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53443,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/10/2014 14:59:32",
      "content": "<p>I have used the ASA software package (www.ant-neuro.com). To review the data I converted them to the edf (European Data Format), a kind of standard for EEG data.</p>\n\n<p>You can download a four weeks trial licence for the software.</p>\n<p>Bye,</p>\n<p>Matthias</p>\n\n<p>P.S. Small parts of the data can also be imported in an ASCII based format like shown below</p>\n\n<p>#Averaged Spiked Tue Jan 09 10:15:12 2001</p>\n<p>NumberPositions= 4</p>\n<p>NumberTimeSteps= 7</p>\n<p>UnitTime ms</p>\n<p>UnitMeas uV</p>\n<p>TimeSteps 0(3.906250)</p>\n<p>ValuesTransposed</p>\n<p>-1.273162 0.067602 3.475845 3.785685</p>\n<p>-1.329496 0.011267 3.554713 3.774418</p>\n<p>-0.997122 -0.242239 3.605414 3.965956</p>\n<p>-1.036557 -0.512645 3.571613 3.385709</p>\n<p>-1.228094 -0.743617 3.757517 3.042068</p>\n<p>-0.884453 -0.388709 4.275796 3.250506</p>\n<p>-0.377442 -0.157737 5.013779 3.740617</p>\n<p>Labels</p>\n<p>Fp1 Fp2 F3 F4</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53448,
      "author_name": "lad215562",
      "author_url": "",
      "post_date": "09/10/2014 15:43:42",
      "content": "<p>Matthias, my apologies, but I did not have the time last night to check the clip. Is there any chance you can upload that particular clip (the original '.mat' file) here?&nbsp;I'm not sure if it will be above the size limit of a file upload, but if you can I should have time to look at it some time today. Otherwise, I should have time tonight to look at it.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53449,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/10/2014 16:11:06",
      "content": "<p>I uploaded patient_1_preictal_segment_0003</p>\n<p>The signals clearly look strange in channels</p>\n<p>LD_4</p>\n<p>LD_5</p>\n<p>RD_7</p>\n<p>RD_8</p>\n<p>But also to some exten LD_1 and LD3</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53454,
      "author_name": "lad215562",
      "author_url": "",
      "post_date": "09/10/2014 16:55:44",
      "content": "<p>Hmm, I am not sure what to think. When I loaded the .mat into Python and plotted the first 10 seconds of data for LD_4 (down-sampled by a factor of 100), I got the attached image (LD_4-10sec-100FoldDownSample.png),&nbsp;which at least doesn't seem to have any clipping issues. Can you confirm that the plot you provided is definitely just the first 10 seconds of data, and not 10 minutes? When I try the whole clip (10 minutes) downsampled by a factor of 10000, I get the second image (LD_4-10min-10000FoldDownSample.png), which also seems to be fairly clipping free (though hard to say for sure due to the downsampling).</p>\n<p>Based on my lack of familiarity with ASA and the&nbsp;humble amount of experience I have in this area, I'm not sure what to tell you outside of two things:</p>\n<p>1) Whatever algorithm the program uses to process the data for display is resulting in the clipping (even in a topographical sense, our graphs look very different).</p>\n<p>2) There really is some kind of error going on when importing the file format?</p>\n<p>Sorry I wasn't able to be of more help :(</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53546,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/12/2014 08:50:18",
      "content": "<p>I looked again at the data in Matlab. Indeed there is some clipping due to a conversion to int16.</p>\n\n<p>Nevertheless the signal looks more like a slightly modulated 50 Hz sine wave than an EEG signal.</p>\n\n<p>Matthias</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53551,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/12/2014 11:43:23",
      "content": "<p>Attaches is the spectrum of the first 4096 samples of</p>\n<p>patient_1_preictal_segment_0003</p>\n<p>channel</p>\n<p>LD</p>\n<p>It shows a strong peak at 50 Hz or 60 Hz and smaller peaks for multiples of it.</p>\n\n<p>Also the screendumps from LAD do not look for me like EEG signals. It could only be hidden in the sinuoidal activity.</p>\n\n<p>Matthias</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53671,
      "author_name": "kevinjos",
      "author_url": "",
      "post_date": "09/14/2014 01:51:12",
      "content": "<p>I have noticed that some channels of some samples have a great amount of line noise (60hz) and others have some DC interference (0-0.5hz). These are common artifacts in EEG readings taken from the scalp. I have no experience (outside of this competition) with iEEG data. Thus, I am not sure why some samples exhibit line noise, and others do not.</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53677,
      "author_name": "franklyn",
      "author_url": "",
      "post_date": "09/14/2014 03:42:01",
      "content": "<p>Well the dogs have implanted sensors that are read from a battery powered unit, the human patients have their recordings taken only during surgery and probably have a line powered unit doing the recording. would that account for the discrepancy ?</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 53774,
      "author_name": "matthiasd",
      "author_url": "",
      "post_date": "09/15/2014 11:21:54",
      "content": "<p>O.k. I will try to leave out channels/datasets with hughe artefact contaminations from training as fas as I am able to identify them.</p>\n\n<p>Matthias</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "53385": "",
    "53389": "",
    "53391": "",
    "53392": "",
    "53393": "",
    "53442": "",
    "53443": "",
    "53448": "",
    "53449": "",
    "53454": "",
    "53546": "",
    "53551": "",
    "53671": "",
    "53677": "",
    "53774": ""
  },
  "source": "meta"
}