{ "id": 8311, "title": "Evaluation criteria written by google translate?", "url": "/competitions/seizure-detection/discussion/8311", "author_name": "", "post_date": "2014-05-29T04:20:36.987Z", "votes": null, "comment_count": 14, "views": 3050, "content": "

Can somebody please tell me what on earth this means?:

\n\n

"Secondly, you must predict the probability that the clip is within the first 15 seconds its respective seizure"

", "messages": [ { "id": "47222", "postDate": "05/29/2014 04:20:36", "content": "

Can somebody please tell me what on earth this means?:

\n\n

"Secondly, you must predict the probability that the clip is within the first 15 seconds its respective seizure"

", "rawMarkdown": "", "votes": null }, { "id": "47223", "postDate": "05/29/2014 04:44:50", "content": "

The first clip of the seizure will have a latency of 0 seconds (the seizure onset). The next clip will have a latency of 1 second, and so on. Any clips that you predict to be of a seizure you'll also have to predict if it's within the first 15 seconds of the seizure onset.

", "rawMarkdown": "", "votes": null }, { "id": "47225", "postDate": "05/29/2014 05:04:24", "content": "

Do you mean that there will literally be a series of clips that are 0 sec latency, 1 sec latency, 2 sec latency etc in a pattern? 

", "rawMarkdown": "", "votes": null }, { "id": "47226", "postDate": "05/29/2014 05:17:10", "content": "

Yes. Take a peek at the latency variable in the training data. You will see that it is an increasing finite sequence of integers. Note, I haven't really dug into the data a lot, but it looks like there are several distinct latency sequences in the data (different sequences for different seizures). Take dog 2 for example. The first latency sequence is 46,47,48,49,50,51. The 2nd sequence is 0,1,2,3, and so on.

", "rawMarkdown": "", "votes": null }, { "id": "47287", "postDate": "05/29/2014 21:24:57", "content": "

I don't really see what you're saying about dog 1.  In the ictal data for dog 1, I see five seizures.  Each one has a series of latencies from 0 to N-1 where N is the length of the seizure.  Records 1 to 31 are the first seizure, 32 to 66 are the second seizure, 67 to 99 are the third, 100 to 140 are the fourth, and 141 to 178 are the fifth and final.

", "rawMarkdown": "", "votes": null }, { "id": "47289", "postDate": "05/29/2014 21:59:58", "content": "

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

", "rawMarkdown": "", "votes": null }, { "id": "47290", "postDate": "05/29/2014 22:01:23", "content": "

You guys obviously have fairly conflicting interpretations of the structure of the data.

\n\n

The idea that there are separate "clips" that start, respectively, at 0,1,2 sec latency within a given seizure seems utterly silly. That would be highly rendundant data.

\n

Now, a set of collections of sensor readings each assigned to a given latency - now THAT would make sense.

\n

And, given the latency variable, the interpretation is clear - we've got some time series data with an auxilliary "latency" variable which acts as a second y variable - we need to classify them by the range they fall in in addition to whether or not the time series contains a seizure. Sounds good.

", "rawMarkdown": "", "votes": null }, { "id": "47292", "postDate": "05/29/2014 22:08:58", "content": "

[quote=RReynoldson;47289]

\n

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

\n

[/quote]

\n

OK, but I would say something similar about dog 2.  Its ictal records consist of three seizures: records 1 to 53, records 54 to 105 and records 106 to 172.  In each case, the first record has latency 0 and the last record has latency N-1.  There's no seizure that starts at latency 46 (which is what I took you to be saying before).

", "rawMarkdown": "", "votes": null }, { "id": "47293", "postDate": "05/29/2014 22:11:35", "content": "

[quote=Phillip Chilton Adkins;47290]

\n

The idea that there are separate "clips" that start, respectively, at 0,1,2 sec latency within a given seizure seems utterly silly. That would be highly rendundant data.

\n

[/quote]

\n

This is exactly what we have, as far as I can tell.  Not sure why you think it's silly or redundant.  Each clip is one second, so there is no overlap between clips.

", "rawMarkdown": "", "votes": null }, { "id": "47294", "postDate": "05/29/2014 22:17:35", "content": "

DELETED

", "rawMarkdown": "", "votes": null }, { "id": "47295", "postDate": "05/29/2014 22:19:29", "content": "

[quote=Eric Jackson;47292]

\n

[quote=RReynoldson;47289]

\n

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

\n

[/quote]

\n

OK, but I would say something similar about dog 2.  Its ictal records consist of three seizures: records 1 to 53, records 54 to 105 and records 106 to 172.  In each case, the first record has latency 0 and the last record has latency N-1.  There's no seizure that starts at latency 46 (which is what I took you to be saying before).

\n

[/quote]

\n

Okay, I see the problem. Looks like I'm reading the files alpha-numerically by name, so the first one is Dog_2_ictal_segment_100.mat, the 2nd is Dog_2_ictal_segment_101.mat and so on. Guess that's why my latency variables are out of order (i.e., starts at 46 instead of 0). Thanks again.

", "rawMarkdown": "", "votes": null }, { "id": "47297", "postDate": "05/29/2014 22:31:21", "content": "

[quote=Phillip Chilton Adkins;47290]

\n

You guys obviously have fairly conflicting interpretations of the structure of the data.

\n

[/quote]

\n

No, I think we're in agreement. The only issue was that I didn't read the files in the correct order. When you load the files in the proper order you'll get exactly what Eric Jackson described.

", "rawMarkdown": "", "votes": null }, { "id": "48878", "postDate": "06/08/2014 23:30:52", "content": "

I'm still a bit confused on the predictions we need to make (my confusion being solidified by my dismal LB results).

\n\n

I believe I'm clear on the first prediction: for each clip, predict whether a seizure is currently taking place. Continuous probabilities are desirable.

\n\n

For the second, do I predict in a purely non-conditional manner the probability that each clip has been recorded in the first 15 seconds of a sequence of clips in which a seizure is occurring? OR, in a conditional sense, do I predict the probability that each clip has been recorded in the first 15 second of a sequence of clips given that a seizure is in fact occurring?

\n\n

In other words, do we have:

\n\n

A - No Seizure

\n

B - Early Seizure

\n

C - Late Seizure

\n\n

or 

\n\n

A - No Seizure

\n

B - Seizure (Early or Late)

\n\n

When I first started I was using discrete probabilities, so this seemed much clearer: only predict for if clip is within_15 if you've already predicted that a seizure is in fact occurring. Now that I've learned that we should use continuous probabilities, I'm not so sure..

\n\n

Thanks!

\n\n

Will

", "rawMarkdown": "", "votes": null }, { "id": "48900", "postDate": "06/09/2014 12:52:16", "content": "

Answered here previously - https://www.kaggle.com/c/seizure-detection/forums/t/8232/clarification-on-evaluation-metric/44947#post44947

", "rawMarkdown": "", "votes": null }, { "id": "48901", "postDate": "06/09/2014 12:56:50", "content": "

Sweet, many thanks.

", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 47223, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 04:44:50", "content": "

The first clip of the seizure will have a latency of 0 seconds (the seizure onset). The next clip will have a latency of 1 second, and so on. Any clips that you predict to be of a seizure you'll also have to predict if it's within the first 15 seconds of the seizure onset.

", "votes": null, "replies": [] }, { "id": 47225, "author_name": "phillipadkins", "author_url": "", "post_date": "05/29/2014 05:04:24", "content": "

Do you mean that there will literally be a series of clips that are 0 sec latency, 1 sec latency, 2 sec latency etc in a pattern? 

", "votes": null, "replies": [] }, { "id": 47226, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 05:17:10", "content": "

Yes. Take a peek at the latency variable in the training data. You will see that it is an increasing finite sequence of integers. Note, I haven't really dug into the data a lot, but it looks like there are several distinct latency sequences in the data (different sequences for different seizures). Take dog 2 for example. The first latency sequence is 46,47,48,49,50,51. The 2nd sequence is 0,1,2,3, and so on.

", "votes": null, "replies": [] }, { "id": 47287, "author_name": "onemillionmonkeys", "author_url": "", "post_date": "05/29/2014 21:24:57", "content": "

I don't really see what you're saying about dog 1.  In the ictal data for dog 1, I see five seizures.  Each one has a series of latencies from 0 to N-1 where N is the length of the seizure.  Records 1 to 31 are the first seizure, 32 to 66 are the second seizure, 67 to 99 are the third, 100 to 140 are the fourth, and 141 to 178 are the fifth and final.

", "votes": null, "replies": [] }, { "id": 47289, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 21:59:58", "content": "

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

", "votes": null, "replies": [] }, { "id": 47290, "author_name": "phillipadkins", "author_url": "", "post_date": "05/29/2014 22:01:23", "content": "

You guys obviously have fairly conflicting interpretations of the structure of the data.

\n\n

The idea that there are separate "clips" that start, respectively, at 0,1,2 sec latency within a given seizure seems utterly silly. That would be highly rendundant data.

\n

Now, a set of collections of sensor readings each assigned to a given latency - now THAT would make sense.

\n

And, given the latency variable, the interpretation is clear - we've got some time series data with an auxilliary "latency" variable which acts as a second y variable - we need to classify them by the range they fall in in addition to whether or not the time series contains a seizure. Sounds good.

", "votes": null, "replies": [] }, { "id": 47292, "author_name": "onemillionmonkeys", "author_url": "", "post_date": "05/29/2014 22:08:58", "content": "

[quote=RReynoldson;47289]

\n

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

\n

[/quote]

\n

OK, but I would say something similar about dog 2.  Its ictal records consist of three seizures: records 1 to 53, records 54 to 105 and records 106 to 172.  In each case, the first record has latency 0 and the last record has latency N-1.  There's no seizure that starts at latency 46 (which is what I took you to be saying before).

", "votes": null, "replies": [] }, { "id": 47293, "author_name": "onemillionmonkeys", "author_url": "", "post_date": "05/29/2014 22:11:35", "content": "

[quote=Phillip Chilton Adkins;47290]

\n

The idea that there are separate "clips" that start, respectively, at 0,1,2 sec latency within a given seizure seems utterly silly. That would be highly rendundant data.

\n

[/quote]

\n

This is exactly what we have, as far as I can tell.  Not sure why you think it's silly or redundant.  Each clip is one second, so there is no overlap between clips.

", "votes": null, "replies": [] }, { "id": 47294, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 22:17:35", "content": "

DELETED

", "votes": null, "replies": [] }, { "id": 47295, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 22:19:29", "content": "

[quote=Eric Jackson;47292]

\n

[quote=RReynoldson;47289]

\n

Eric Jackson: Nice catch. I should have wrote dog 2, not dog 1.

\n

[/quote]

\n

OK, but I would say something similar about dog 2.  Its ictal records consist of three seizures: records 1 to 53, records 54 to 105 and records 106 to 172.  In each case, the first record has latency 0 and the last record has latency N-1.  There's no seizure that starts at latency 46 (which is what I took you to be saying before).

\n

[/quote]

\n

Okay, I see the problem. Looks like I'm reading the files alpha-numerically by name, so the first one is Dog_2_ictal_segment_100.mat, the 2nd is Dog_2_ictal_segment_101.mat and so on. Guess that's why my latency variables are out of order (i.e., starts at 46 instead of 0). Thanks again.

", "votes": null, "replies": [] }, { "id": 47297, "author_name": "robreynoldson", "author_url": "", "post_date": "05/29/2014 22:31:21", "content": "

[quote=Phillip Chilton Adkins;47290]

\n

You guys obviously have fairly conflicting interpretations of the structure of the data.

\n

[/quote]

\n

No, I think we're in agreement. The only issue was that I didn't read the files in the correct order. When you load the files in the proper order you'll get exactly what Eric Jackson described.

", "votes": null, "replies": [] }, { "id": 48878, "author_name": "cavaunpeu", "author_url": "", "post_date": "06/08/2014 23:30:52", "content": "

I'm still a bit confused on the predictions we need to make (my confusion being solidified by my dismal LB results).

\n\n

I believe I'm clear on the first prediction: for each clip, predict whether a seizure is currently taking place. Continuous probabilities are desirable.

\n\n

For the second, do I predict in a purely non-conditional manner the probability that each clip has been recorded in the first 15 seconds of a sequence of clips in which a seizure is occurring? OR, in a conditional sense, do I predict the probability that each clip has been recorded in the first 15 second of a sequence of clips given that a seizure is in fact occurring?

\n\n

In other words, do we have:

\n\n

A - No Seizure

\n

B - Early Seizure

\n

C - Late Seizure

\n\n

or 

\n\n

A - No Seizure

\n

B - Seizure (Early or Late)

\n\n

When I first started I was using discrete probabilities, so this seemed much clearer: only predict for if clip is within_15 if you've already predicted that a seizure is in fact occurring. Now that I've learned that we should use continuous probabilities, I'm not so sure..

\n\n

Thanks!

\n\n

Will

", "votes": null, "replies": [] }, { "id": 48900, "author_name": "wcukierski", "author_url": "", "post_date": "06/09/2014 12:52:16", "content": "

Answered here previously - https://www.kaggle.com/c/seizure-detection/forums/t/8232/clarification-on-evaluation-metric/44947#post44947

", "votes": null, "replies": [] }, { "id": 48901, "author_name": "cavaunpeu", "author_url": "", "post_date": "06/09/2014 12:56:50", "content": "

Sweet, many thanks.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "47222": "", "47223": "", "47225": "", "47226": "", "47287": "", "47289": "", "47290": "", "47292": "", "47293": "", "47294": "", "47295": "", "47297": "", "48878": "", "48900": "", "48901": "" }, "source": "meta" }