{ "id": 10128, "title": "Fields in Matlab file - can't find 'data' etc", "url": "/competitions/seizure-prediction/discussion/10128", "author_name": "", "post_date": "2014-08-27T19:23:33.173Z", "votes": 1, "comment_count": 13, "views": 4713, "content": "

After I import the files into Python, I can't find the field names 'Data' , 'data_length_sec', 'sampling_frequency' etc etc. There's an array with

\n

'preictal_segment_1': array

\n

etc

\n

but I can't access by the field names given in data description. Any idea? I can see fields in R but not Python

", "messages": [ { "id": "52537", "postDate": "08/27/2014 19:23:33", "content": "

After I import the files into Python, I can't find the field names 'Data' , 'data_length_sec', 'sampling_frequency' etc etc. There's an array with

\n

'preictal_segment_1': array

\n

etc

\n

but I can't access by the field names given in data description. Any idea? I can see fields in R but not Python

", "rawMarkdown": "", "votes": null }, { "id": "52556", "postDate": "08/28/2014 07:02:16", "content": "

I hope it's help.

", "rawMarkdown": "", "votes": null }, { "id": "52560", "postDate": "08/28/2014 08:29:02", "content": "

BTW, it's seems we spend many GB for zeroes ) All data traces saved in DBL (double), but they appears in SHORT (i16)... At least the examples I tried..

", "rawMarkdown": "", "votes": null }, { "id": "52570", "postDate": "08/28/2014 13:04:40", "content": "

 If you use scipy.io.loadmat() then you will have all data you mentioned in a python dictionary , but nested in other data structures. If you print the result you can figure out how to access each field.

\n

For example ,if you have

\n

data_struct = scipy.io.loadmat('Dog_1_interictal_segment_0008.mat')

\n

with code:

\n

data_struct['interictal_segment_8'][0][0][0][ch][samp]

\n

you can extract the sample with index 'samp' from chanel  'ch'   from file 'Dog_1_interictal_segment_0008.mat'

", "rawMarkdown": "", "votes": null }, { "id": "52598", "postDate": "08/28/2014 19:39:25", "content": "

[quote=claudiu1989;52570]

\n

 If you use scipy.io.loadmat() then you will have all data you mentioned in a python dictionary , but nested in other data structures. If you print the result you can figure out how to access each field.

\n

For example ,if you have

\n

data_struct = scipy.io.loadmat('Dog_1_interictal_segment_0008.mat')

\n

with code:

\n

data_struct['interictal_segment_8'][0][0][0][ch][samp]

\n

you can extract the sample with index 'samp' from chanel  'ch'   from file 'Dog_1_interictal_segment_0008.mat'

\n

[/quote]

\n

Expanding on Claudiu1989's great response:

\n

data_struct[sample][0][0] is your 'base' level to access the clip's information. From there:

\n\n

EDIT: For the last bullet, you should only expect to have a '4' value for the third index for preictal and interictal files. Since we are not given the test files in the context of an hour long series, they are not provided with this field. Thanks to Lawrence for pointing out the lack of clarity! :)

", "rawMarkdown": "", "votes": null }, { "id": "52600", "postDate": "08/28/2014 20:36:37", "content": "

In case if you prefer R:

\n

library(R.matlab)
pat <- readMat('Patient_1\\\\Patient_1_interictal_segment_0018.mat')
#pat[[1]][[2]] == 600
#pat[[1]][[3]] == 5000
df <- data.frame(t(pat[[1]][[1]]))
names(df) <- unlist(pat[[1]][[4]])
head(df)

", "rawMarkdown": "", "votes": null }, { "id": "53073", "postDate": "09/03/2014 21:09:21", "content": "

Thanks all. Can I confirm that for each segment (eg Patient 1), I would end up with a 300000 * 15 data frame (in R)? ta

", "rawMarkdown": "", "votes": null }, { "id": "53203", "postDate": "09/05/2014 20:04:45", "content": "

I do not see any values for the third index set to "4" as LAD wrote, and as a sanity check: take 

\n

data_length_sec =  data_struct[sample][0][0][1][0][0]      (e.g. 600 seconds) 

\n

                       multiplied by 

\n

sampling_frequency = data_struct[sample][0][0][2][0][0]    (e.g. 399.61 Hz)

\n

to see that it matches the amount of data in the clip for that electrode:

\n

len( data_struct[sample][0][0][0][0] )    (e.g. 239766)

", "rawMarkdown": "", "votes": null }, { "id": "53206", "postDate": "09/05/2014 20:50:06", "content": "

Are you missing the '4' in third index for any interictal or priectal files? I should have been more explicit in the original post I made, but I don't believe you should expect one for the test files, since we are not given them in the context of an hour long segment (as we are for preictal and interictal).

\n

I only have Dog_5 locally on this computer to verify, but some quick scanning seems to confirm this to be true for substituent data files (i.e. preictal and interictal files have a '4' vale for third index, test files do not).

", "rawMarkdown": "", "votes": null }, { "id": "53207", "postDate": "09/05/2014 21:07:26", "content": "

Sorry, my mistake. It was missing in a test file.

", "rawMarkdown": "", "votes": null }, { "id": "53209", "postDate": "09/05/2014 21:09:17", "content": "

No worries! It was good that you pointed it out, I should have been more explicit in the original post (I'll go back and edit it now for clarity) :)

", "rawMarkdown": "", "votes": null }, { "id": "53327", "postDate": "09/08/2014 06:43:43", "content": "

[quote=Vadym Gnatkovsky;52560]

\n

BTW, it's seems we spend many GB for zeroes ) All data traces saved in DBL (double), but they appears in SHORT (i16)... At least the examples I tried..

\n

[/quote]

\n\n

I have only looked at Dog_5 (the short one). Can anyone confirm that these are all i16's taken straight from the ADC, so I can safely shrink them?

", "rawMarkdown": "", "votes": null }, { "id": "56108", "postDate": "10/16/2014 00:04:02", "content": "

Is anybody having issues loading some of the files using scipy.io.loadmat. So far 2 files are giving me: "IOError:could not read bytes" (currently checking all files)

\n

1) Dog_1_test_segment_0088.mat

\n

2) Dog_2_interictal_segment_0111.mat

\n\n

Anyone else experiencing this and are matlab/octave people seeing this?

", "rawMarkdown": "", "votes": null }, { "id": "216263", "postDate": "08/25/2017 03:03:29", "content": "

could anyone share a scrip for reading the data using scipy.io.loadmat? it's would be great

", "rawMarkdown": "could anyone share a scrip for reading the data using scipy.io.loadmat? it's would be great", "votes": null } ], "comments": [ { "id": 52556, "author_name": "vadymgnatkovsky", "author_url": "", "post_date": "08/28/2014 07:02:16", "content": "

I hope it's help.

", "votes": null, "replies": [] }, { "id": 52560, "author_name": "vadymgnatkovsky", "author_url": "", "post_date": "08/28/2014 08:29:02", "content": "

BTW, it's seems we spend many GB for zeroes ) All data traces saved in DBL (double), but they appears in SHORT (i16)... At least the examples I tried..

", "votes": null, "replies": [] }, { "id": 52570, "author_name": "claudiu1989", "author_url": "", "post_date": "08/28/2014 13:04:40", "content": "

 If you use scipy.io.loadmat() then you will have all data you mentioned in a python dictionary , but nested in other data structures. If you print the result you can figure out how to access each field.

\n

For example ,if you have

\n

data_struct = scipy.io.loadmat('Dog_1_interictal_segment_0008.mat')

\n

with code:

\n

data_struct['interictal_segment_8'][0][0][0][ch][samp]

\n

you can extract the sample with index 'samp' from chanel  'ch'   from file 'Dog_1_interictal_segment_0008.mat'

", "votes": null, "replies": [] }, { "id": 52598, "author_name": "lad215562", "author_url": "", "post_date": "08/28/2014 19:39:25", "content": "

[quote=claudiu1989;52570]

\n

 If you use scipy.io.loadmat() then you will have all data you mentioned in a python dictionary , but nested in other data structures. If you print the result you can figure out how to access each field.

\n

For example ,if you have

\n

data_struct = scipy.io.loadmat('Dog_1_interictal_segment_0008.mat')

\n

with code:

\n

data_struct['interictal_segment_8'][0][0][0][ch][samp]

\n

you can extract the sample with index 'samp' from chanel  'ch'   from file 'Dog_1_interictal_segment_0008.mat'

\n

[/quote]

\n

Expanding on Claudiu1989's great response:

\n

data_struct[sample][0][0] is your 'base' level to access the clip's information. From there:

\n\n

EDIT: For the last bullet, you should only expect to have a '4' value for the third index for preictal and interictal files. Since we are not given the test files in the context of an hour long series, they are not provided with this field. Thanks to Lawrence for pointing out the lack of clarity! :)

", "votes": null, "replies": [] }, { "id": 52600, "author_name": "spimenov", "author_url": "", "post_date": "08/28/2014 20:36:37", "content": "

In case if you prefer R:

\n

library(R.matlab)
pat <- readMat('Patient_1\\\\Patient_1_interictal_segment_0018.mat')
#pat[[1]][[2]] == 600
#pat[[1]][[3]] == 5000
df <- data.frame(t(pat[[1]][[1]]))
names(df) <- unlist(pat[[1]][[4]])
head(df)

", "votes": null, "replies": [] }, { "id": 53073, "author_name": "domcastro", "author_url": "", "post_date": "09/03/2014 21:09:21", "content": "

Thanks all. Can I confirm that for each segment (eg Patient 1), I would end up with a 300000 * 15 data frame (in R)? ta

", "votes": null, "replies": [] }, { "id": 53203, "author_name": "lawrencechernin", "author_url": "", "post_date": "09/05/2014 20:04:45", "content": "

I do not see any values for the third index set to "4" as LAD wrote, and as a sanity check: take 

\n

data_length_sec =  data_struct[sample][0][0][1][0][0]      (e.g. 600 seconds) 

\n

                       multiplied by 

\n

sampling_frequency = data_struct[sample][0][0][2][0][0]    (e.g. 399.61 Hz)

\n

to see that it matches the amount of data in the clip for that electrode:

\n

len( data_struct[sample][0][0][0][0] )    (e.g. 239766)

", "votes": null, "replies": [] }, { "id": 53206, "author_name": "lad215562", "author_url": "", "post_date": "09/05/2014 20:50:06", "content": "

Are you missing the '4' in third index for any interictal or priectal files? I should have been more explicit in the original post I made, but I don't believe you should expect one for the test files, since we are not given them in the context of an hour long segment (as we are for preictal and interictal).

\n

I only have Dog_5 locally on this computer to verify, but some quick scanning seems to confirm this to be true for substituent data files (i.e. preictal and interictal files have a '4' vale for third index, test files do not).

", "votes": null, "replies": [] }, { "id": 53207, "author_name": "lawrencechernin", "author_url": "", "post_date": "09/05/2014 21:07:26", "content": "

Sorry, my mistake. It was missing in a test file.

", "votes": null, "replies": [] }, { "id": 53209, "author_name": "lad215562", "author_url": "", "post_date": "09/05/2014 21:09:17", "content": "

No worries! It was good that you pointed it out, I should have been more explicit in the original post (I'll go back and edit it now for clarity) :)

", "votes": null, "replies": [] }, { "id": 53327, "author_name": "eumeswil0", "author_url": "", "post_date": "09/08/2014 06:43:43", "content": "

[quote=Vadym Gnatkovsky;52560]

\n

BTW, it's seems we spend many GB for zeroes ) All data traces saved in DBL (double), but they appears in SHORT (i16)... At least the examples I tried..

\n

[/quote]

\n\n

I have only looked at Dog_5 (the short one). Can anyone confirm that these are all i16's taken straight from the ADC, so I can safely shrink them?

", "votes": null, "replies": [] }, { "id": 56108, "author_name": "corticalharmony", "author_url": "", "post_date": "10/16/2014 00:04:02", "content": "

Is anybody having issues loading some of the files using scipy.io.loadmat. So far 2 files are giving me: "IOError:could not read bytes" (currently checking all files)

\n

1) Dog_1_test_segment_0088.mat

\n

2) Dog_2_interictal_segment_0111.mat

\n\n

Anyone else experiencing this and are matlab/octave people seeing this?

", "votes": null, "replies": [] }, { "id": 216263, "author_name": "ngduyanhece", "author_url": "", "post_date": "08/25/2017 03:03:29", "content": "

could anyone share a scrip for reading the data using scipy.io.loadmat? it's would be great

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "52537": "", "52556": "", "52560": "", "52570": "", "52598": "", "52600": "", "53073": "", "53203": "", "53206": "", "53207": "", "53209": "", "53327": "", "56108": "", "216263": "could anyone share a scrip for reading the data using scipy.io.loadmat? it's would be great" }, "source": "meta" }