![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4256010%2F1067f0f9a7afbdf47d41ff107d2a1481%2FScreenshot%20from%202020-12-23%2017-49-07.png?generation=1608745854599500&alt=media\")
Here are some notes I made on the competition metric in my preferred learning format: an equation-first walkthrough.
\nI begin by dissecting label-ranking average precision, and finish up with a minor tweak made to get label-weighted label-ranking average precision.
\nSorry I needed to paste an image as the markdown here doesn't support inline equations. I haven't tweaked it much from what it looks like in my own notes so feel free to ask if something doesn't make sense.
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4256010%2F0a90d873b6f86cc713969f2f6f69140c%2FScreenshot%20from%202020-12-23%2017-50-29.png?generation=1608745865435766&alt=media\")
FYI |X| refers to cardinality of set X, meaning the number of elements in that set. And ||y||_0 is the 0-norm of y, meaning the number of non-zero elements of y.
", "messages": [ { "id": "1124160", "postDate": "12/23/2020 17:51:37", "content": "Here are some notes I made on the competition metric in my preferred learning format: an equation-first walkthrough.
\nI begin by dissecting label-ranking average precision, and finish up with a minor tweak made to get label-weighted label-ranking average precision.
\nSorry I needed to paste an image as the markdown here doesn't support inline equations. I haven't tweaked it much from what it looks like in my own notes so feel free to ask if something doesn't make sense.
\n
FYI |X| refers to cardinality of set X, meaning the number of elements in that set. And ||y||_0 is the 0-norm of y, meaning the number of non-zero elements of y.
", "rawMarkdown": "Here are some notes I made on the competition metric in my preferred learning format: an equation-first walkthrough.\n\nI begin by dissecting label-ranking average precision, and finish up with a minor tweak made to get label-weighted label-ranking average precision.\n\nSorry I needed to paste an image as the markdown here doesn't support inline equations. I haven't tweaked it much from what it looks like in my own notes so feel free to ask if something doesn't make sense.\n\n---\n\n\n\n\n\nFYI |X| refers to cardinality of set X, meaning the number of elements in that set. And ||y||_0 is the 0-norm of y, meaning the number of non-zero elements of y.", "votes": null }, { "id": "1155633", "postDate": "01/16/2021 14:21:52", "content": "This has enough upvotes now that I should comment: I'm probably wrong about \"a metric of 0.5 would mean that the model is being random\". I noticed this when I actually got into the competition and always saw my score start at around 0.2 ish during training.
\nLet's think about that, going back on my statement that \"we're asking for the fraction of labels scored at least as high as this one, that are indeed positive\". So we'll use a 5 class dataset, where the classes are evenly balanced. Now let's say we randomly scatter our scores on a number line, we pick a label, and we color all ground truths for that label red. Next, pick a random red point. The question is then: what proportion of scores from here and above are red? Answer: with a random scatter we would have an expectation value of 0.2.
\nSo to fix my statement I should say that \"a metric of 1/N_classes would mean the model is being totally random (given an evenly balanced dataset)\". I feel a bit lazy so I'm not going to think about the non-evenly balanced dataset.
", "rawMarkdown": "This has enough upvotes now that I should comment: I'm probably wrong about \"a metric of 0.5 would mean that the model is being random\". I noticed this when I actually got into the competition and always saw my score start at around 0.2 ish during training.\n\nLet's think about that, going back on my statement that \"we're asking for the fraction of labels scored at least as high as this one, that are indeed positive\". So we'll use a 5 class dataset, where the classes are evenly balanced. Now let's say we randomly scatter our scores on a number line, we pick a label, and we color all ground truths for that label red. Next, pick a random red point. The question is then: what proportion of scores from here and above are red? Answer: with a random scatter we would have an expectation value of 0.2.\n\nSo to fix my statement I should say that **\"a metric of 1/N_classes would mean the model is being totally random (given an evenly balanced dataset)\"**. I feel a bit lazy so I'm not going to think about the non-evenly balanced dataset.", "votes": null }, { "id": "1163154", "postDate": "01/21/2021 15:08:14", "content": "Now, I understood the LRAP metric.
\nThank you!
", "rawMarkdown": "Now, I understood the LRAP metric.\n\nThank you!", "votes": null } ], "comments": [ { "id": 1155633, "author_name": "alexandersoare", "author_url": "", "post_date": "01/16/2021 14:21:52", "content": "This has enough upvotes now that I should comment: I'm probably wrong about \"a metric of 0.5 would mean that the model is being random\". I noticed this when I actually got into the competition and always saw my score start at around 0.2 ish during training.
\nLet's think about that, going back on my statement that \"we're asking for the fraction of labels scored at least as high as this one, that are indeed positive\". So we'll use a 5 class dataset, where the classes are evenly balanced. Now let's say we randomly scatter our scores on a number line, we pick a label, and we color all ground truths for that label red. Next, pick a random red point. The question is then: what proportion of scores from here and above are red? Answer: with a random scatter we would have an expectation value of 0.2.
\nSo to fix my statement I should say that \"a metric of 1/N_classes would mean the model is being totally random (given an evenly balanced dataset)\". I feel a bit lazy so I'm not going to think about the non-evenly balanced dataset.
", "votes": null, "replies": [] }, { "id": 1163154, "author_name": "joshi98kishan", "author_url": "", "post_date": "01/21/2021 15:08:14", "content": "Now, I understood the LRAP metric.
\nThank you!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1124160": "Here are some notes I made on the competition metric in my preferred learning format: an equation-first walkthrough.\n\nI begin by dissecting label-ranking average precision, and finish up with a minor tweak made to get label-weighted label-ranking average precision.\n\nSorry I needed to paste an image as the markdown here doesn't support inline equations. I haven't tweaked it much from what it looks like in my own notes so feel free to ask if something doesn't make sense.\n\n---\n\n\n\n\n\nFYI |X| refers to cardinality of set X, meaning the number of elements in that set. And ||y||_0 is the 0-norm of y, meaning the number of non-zero elements of y.", "1155633": "This has enough upvotes now that I should comment: I'm probably wrong about \"a metric of 0.5 would mean that the model is being random\". I noticed this when I actually got into the competition and always saw my score start at around 0.2 ish during training.\n\nLet's think about that, going back on my statement that \"we're asking for the fraction of labels scored at least as high as this one, that are indeed positive\". So we'll use a 5 class dataset, where the classes are evenly balanced. Now let's say we randomly scatter our scores on a number line, we pick a label, and we color all ground truths for that label red. Next, pick a random red point. The question is then: what proportion of scores from here and above are red? Answer: with a random scatter we would have an expectation value of 0.2.\n\nSo to fix my statement I should say that **\"a metric of 1/N_classes would mean the model is being totally random (given an evenly balanced dataset)\"**. I feel a bit lazy so I'm not going to think about the non-evenly balanced dataset.", "1163154": "Now, I understood the LRAP metric.\n\nThank you!" }, "source": "meta" }