![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1920073%2Fd1ddcfaf4e28aa3a6ea6018d5aa7cae1%2F2020-06-19%2013-03-38.png?generation=1592561329187928&alt=media\")
Why in example birds field has empty values?
\n\n> For each row_id/time window, you need to provide a space separated list of the set of unique birds that made a call beginning or ending in that time window. If there are no bird calls in a time window, use the code nocall.
> Submissions will be evaluated based on their row-wise micro averaged F1 score.
\n\nRow-wise means that amecro amerob will use as unique class for calculating f1?
what about semi-prediction? how your metric consider this cases: \n1. amecro amerob vs amecro\n2. amecro amerob vs amecro amerob banswa\n3. amecro amerob vs banswa
I think case 1 is not equal case 3 for evaluation. Obviously model in case 1 is better than model in case 3.
\n\n@stefankahl @sohier @tomdenton
", "messages": [ { "id": "893019", "postDate": "06/19/2020 10:22:43", "content": "Why in example birds field has empty values?
\n\n> For each row_id/time window, you need to provide a space separated list of the set of unique birds that made a call beginning or ending in that time window. If there are no bird calls in a time window, use the code nocall.
> Submissions will be evaluated based on their row-wise micro averaged F1 score.
\n\nRow-wise means that amecro amerob will use as unique class for calculating f1?
what about semi-prediction? how your metric consider this cases: \n1. amecro amerob vs amecro\n2. amecro amerob vs amecro amerob banswa\n3. amecro amerob vs banswa
I think case 1 is not equal case 3 for evaluation. Obviously model in case 1 is better than model in case 3.
\n\n@stefankahl @sohier @tomdenton
", "rawMarkdown": "# Question 1\n\nWhy in example birds field has empty values?\n\n> For each `row_id/time` window, you need to provide a space separated list of the set of unique `birds` that made a call beginning or ending in that time window. If there are no bird calls in a time window, use the code `nocall`.\n\n\n\n# Question 2\n\n> Submissions will be evaluated based on their row-wise micro averaged F1 score.\n\nRow-wise means that `amecro amerob` will use as unique class for calculating f1? \n\n\n# Question 3\nwhat about semi-prediction? how your metric consider this cases: \n1. `amecro amerob` vs `amecro`\n2. `amecro amerob` vs `amecro amerob banswa`\n3. `amecro amerob` vs `banswa`\n\nI think case 1 is not equal case 3 for evaluation. Obviously model in case 1 is better than model in case 3.\n\n@stefankahl @sohier @tomdenton", "votes": null }, { "id": "893120", "postDate": "06/19/2020 12:05:38", "content": "Q2 amecro and amerob will be treated two positive classes and F1 will calculated using your predicted labels for example if you predict amecro banswa your recall is 0.5 and precision is 0.5 so F1 for this row 0.5
Q3 case 1 and case3 are not equal as calculated in above maner.
", "rawMarkdown": "Q2 `amecro` and `amerob` will be treated two positive classes and F1 will calculated using your predicted labels for example if you predict `amecro banswa` your recall is 0.5 and precision is 0.5 so F1 for this row 0.5\n\nQ3 case 1 and case3 are not equal as calculated in above maner.", "votes": null }, { "id": "893131", "postDate": "06/19/2020 12:16:40", "content": "Thank you for answer @dhananjay3 Hope you are right!
\n\nI think we need working method with evaluation f1-score with approving from host. If you are right this method is not easy: need some preprocessing with extending and matching all labels.
\n\nAlso don't exclude that hosts make a mistake as he wrote about row-wise for f1-score.
i apologize if the question is sillly , but all audio files with channels == '2 (stereo)' are also mono , does this mean it was converted to mono?
", "rawMarkdown": "i apologize if the question is sillly , but all audio files with channels == '2 (stereo)' are also mono , does this mean it was converted to mono?", "votes": null }, { "id": "896784", "postDate": "06/22/2020 12:34:36", "content": "Yes, that might be the case. We used metadata provided by Xeno-canto with no further processing and some technical metadata might not be accurate (which also applies for many other data provided by the recordists).
", "rawMarkdown": "Yes, that might be the case. We used metadata provided by Xeno-canto with no further processing and some technical metadata might not be accurate (which also applies for many other data provided by the recordists).", "votes": null }, { "id": "918240", "postDate": "07/07/2020 05:37:34", "content": "@dhananjay3 I am still puzzled as to how this idea is being extended for the full submission file? We get a micro-avg value for a row taking each entry in the row as a positive class, but do we end up averaging all the F1 scores for each row? Plus, F1 score of each row does not make sense for some reason. You will have at most one positive value from each class mentioned in a row at most, so F1 score does not really make sense for one particular row.
\n\neg: amecro amerob vs amecro amecro amecro amerob (not a valid row)
\n\nI think what @shonenkov is speculating may be right. Correct me if I am heading in the wrong direction.
", "rawMarkdown": "dhananjay3 I am still puzzled as to how this idea is being extended for the full submission file? We get a micro-avg value for a row taking each entry in the row as a positive class, but do we end up averaging all the F1 scores for each row? Plus, F1 score of each row does not make sense for some reason. You will have at most one positive value from each class mentioned in a row at most, so F1 score does not really make sense for one particular row.\n\neg: **amecro amerob** vs **amecro amecro amecro** amerob (not a valid row)\n\nI think what @shonenkov is speculating may be right. Correct me if I am heading in the wrong direction.", "votes": null } ], "comments": [ { "id": 893120, "author_name": "dhananjay3", "author_url": "", "post_date": "06/19/2020 12:05:38", "content": "Q2 amecro and amerob will be treated two positive classes and F1 will calculated using your predicted labels for example if you predict amecro banswa your recall is 0.5 and precision is 0.5 so F1 for this row 0.5
Q3 case 1 and case3 are not equal as calculated in above maner.
", "votes": null, "replies": [ { "id": 893131, "author_name": "shonenkov", "author_url": "", "post_date": "06/19/2020 12:16:40", "content": "Thank you for answer @dhananjay3 Hope you are right!
\n\nI think we need working method with evaluation f1-score with approving from host. If you are right this method is not easy: need some preprocessing with extending and matching all labels.
\n\nAlso don't exclude that hosts make a mistake as he wrote about row-wise for f1-score.
@dhananjay3 I am still puzzled as to how this idea is being extended for the full submission file? We get a micro-avg value for a row taking each entry in the row as a positive class, but do we end up averaging all the F1 scores for each row? Plus, F1 score of each row does not make sense for some reason. You will have at most one positive value from each class mentioned in a row at most, so F1 score does not really make sense for one particular row.
\n\neg: amecro amerob vs amecro amecro amecro amerob (not a valid row)
\n\nI think what @shonenkov is speculating may be right. Correct me if I am heading in the wrong direction.
", "votes": null, "replies": [] } ] }, { "id": 896463, "author_name": "kumarselvakumaran", "author_url": "", "post_date": "06/22/2020 07:43:36", "content": "i apologize if the question is sillly , but all audio files with channels == '2 (stereo)' are also mono , does this mean it was converted to mono?
", "votes": null, "replies": [ { "id": 896784, "author_name": "stefankahl", "author_url": "", "post_date": "06/22/2020 12:34:36", "content": "Yes, that might be the case. We used metadata provided by Xeno-canto with no further processing and some technical metadata might not be accurate (which also applies for many other data provided by the recordists).
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "893019": "# Question 1\n\nWhy in example birds field has empty values?\n\n> For each `row_id/time` window, you need to provide a space separated list of the set of unique `birds` that made a call beginning or ending in that time window. If there are no bird calls in a time window, use the code `nocall`.\n\n\n\n# Question 2\n\n> Submissions will be evaluated based on their row-wise micro averaged F1 score.\n\nRow-wise means that `amecro amerob` will use as unique class for calculating f1? \n\n\n# Question 3\nwhat about semi-prediction? how your metric consider this cases: \n1. `amecro amerob` vs `amecro`\n2. `amecro amerob` vs `amecro amerob banswa`\n3. `amecro amerob` vs `banswa`\n\nI think case 1 is not equal case 3 for evaluation. Obviously model in case 1 is better than model in case 3.\n\n@stefankahl @sohier @tomdenton", "893120": "Q2 `amecro` and `amerob` will be treated two positive classes and F1 will calculated using your predicted labels for example if you predict `amecro banswa` your recall is 0.5 and precision is 0.5 so F1 for this row 0.5\n\nQ3 case 1 and case3 are not equal as calculated in above maner.", "893131": "Thank you for answer @dhananjay3 Hope you are right!\n\nI think we need working method with evaluation f1-score with approving from host. If you are right this method is not easy: need some preprocessing with extending and matching all labels. \n\nAlso don't exclude that hosts make a mistake as he wrote about `row-wise` for f1-score.", "896463": "i apologize if the question is sillly , but all audio files with channels == '2 (stereo)' are also mono , does this mean it was converted to mono?", "896784": "Yes, that might be the case. We used metadata provided by Xeno-canto with no further processing and some technical metadata might not be accurate (which also applies for many other data provided by the recordists).", "918240": "dhananjay3 I am still puzzled as to how this idea is being extended for the full submission file? We get a micro-avg value for a row taking each entry in the row as a positive class, but do we end up averaging all the F1 scores for each row? Plus, F1 score of each row does not make sense for some reason. You will have at most one positive value from each class mentioned in a row at most, so F1 score does not really make sense for one particular row.\n\neg: **amecro amerob** vs **amecro amecro amecro** amerob (not a valid row)\n\nI think what @shonenkov is speculating may be right. Correct me if I am heading in the wrong direction." }, "source": "meta" }