Hi, Kagglers.
\nApologies if this has already been discussed in previous discussions.
\nAfter conducting some EDA, I discovered data where the votes are tied. In such cases, how was the Expert Consensus(final targets) ultimately determined? Additionally, would it be quite challenging for a model to make these subtle judgments?
\nI'm looking for a discussion on this matter.\"
\n![](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8386305%2Fb63bbc64c1403755a3e5f1b08dee6f99%2F2024-02-23%2018.56.56.png?generation=1708682288500579&alt=media\")
I made this discovery using the following code:
\ndef filter_rows():\n [TARGETS]\n non_zero_values [v v if v ]\n if non_zero_values:\n \n max_value (non_zero_values)\n unique_values (non_zero_values)\n (non_zero_values.() max_value unique_values)\n\nfiltered_df train[train.apply(filter_rows, axis)]\nfiltered_df\nThank you first.
", "messages": [ { "id": "2664942", "postDate": "02/23/2024 09:59:53", "content": "Hi, Kagglers.
\nApologies if this has already been discussed in previous discussions.
\nAfter conducting some EDA, I discovered data where the votes are tied. In such cases, how was the Expert Consensus(final targets) ultimately determined? Additionally, would it be quite challenging for a model to make these subtle judgments?
\nI'm looking for a discussion on this matter.\"
\n
I made this discovery using the following code:
\ndef filter_rows():\n [TARGETS]\n non_zero_values [v v if v ]\n if non_zero_values:\n \n max_value (non_zero_values)\n unique_values (non_zero_values)\n (non_zero_values.() max_value unique_values)\n\nfiltered_df train[train.apply(filter_rows, axis)]\nfiltered_df\nThank you first.
", "rawMarkdown": "Hi, Kagglers.\nApologies if this has already been discussed in previous discussions.\nAfter conducting some EDA, I discovered data where the votes are tied. In such cases, how was the Expert Consensus(final targets) ultimately determined? Additionally, would it be quite challenging for a model to make these subtle judgments?\nI'm looking for a discussion on this matter.\"\n\n\n\nI made this discovery using the following code:\n```\ndef filter_rows(row):\n values = row[TARGETS]\n non_zero_values = [v for v in values if v != 0]\n if not non_zero_values:\n return False\n max_value = max(non_zero_values)\n unique_values = set(non_zero_values)\n return any(non_zero_values.count(value) > 1 and max_value == value for value in unique_values)\n\nfiltered_df = train[train.apply(filter_rows, axis=1)]\nfiltered_df\n```\n\nThank you first.", "votes": null }, { "id": "2665202", "postDate": "02/23/2024 14:11:23", "content": "['lpd_vote', 'gpd_vote'] : LPD 165
\n['gpd_vote', 'grda_vote'] : GPD 58
\n['gpd_vote', 'lrda_vote'] : GPD 3
\n['lpd_vote', 'lrda_vote'] : LPD 60
\n['lrda_vote', 'grda_vote'] : LRDA 6
\n['seizure_vote', 'gpd_vote'] : Seizure 56
\n['seizure_vote', 'lpd_vote'] : Seizure 90
\n['seizure_vote', 'grda_vote'] : Seizure 3
\n['seizure_vote', 'lrda_vote'] : Seizure 6
\n['seizure_vote', 'other_vote'] : Seizure 147
\n(others always lose)
\netc … inside [] is kinds of votes, after \":\" is expert consensus number.
\nso if u do compare the strength SEIZURE>LPD>GPD>LRDA>GRDA>OTHERS
\nwill make notebook about this soon
Expert_consensus is just argmax(*_vote).
\nargmax in case of equality of votes always returns the very first vote, i.e. voice with \"lower index\".
\nIt's easy to check:
df = pd.read_csv()\n\ntars = {: , : , : , : , : , : }\ntargets = [, , , , , ]\n\ndf[] = df.expert_consensus.(tars)\ndf[] = np.argmax(df[targets].values, axis=)\n\n(df[] != df.target).()\n0
", "rawMarkdown": "Expert_consensus is just **argmax**(*_vote).\nargmax in case of equality of votes always returns the **very first vote**, i.e. voice with \"lower index\".\nIt's easy to check:\n\n```python\ndf = pd.read_csv('/kaggle/input/hms-harmful-brain-activity-classification/train.csv')\n\ntars = {'Seizure': 0, 'LPD': 1, 'GPD': 2, 'LRDA': 3, 'GRDA': 4, 'Other': 5}\ntargets = ['seizure_vote', 'lpd_vote', 'gpd_vote', 'lrda_vote', 'grda_vote', 'other_vote']\n\ndf['target'] = df.expert_consensus.map(tars)\ndf['argmax_vote'] = np.argmax(df[targets].values, axis=1)\n\n(df['argmax_vote'] != df.target).sum()\n```\n 0", "votes": null }, { "id": "2665520", "postDate": "02/23/2024 17:43:44", "content": "\" Additionally, would it be quite challenging for a model to make these subtle judgments?\" Not much. The model has to predict the probabilities, not the consensus.
", "rawMarkdown": "\" Additionally, would it be quite challenging for a model to make these subtle judgments?\" Not much. The model has to predict the probabilities, not the consensus.", "votes": null }, { "id": "2665609", "postDate": "02/23/2024 18:39:01", "content": "Nice to know. Thanks
\nFor the data set I am using I found around 4000+ rows of ties. When chasing another idea last week I had created a column in train for the top 2 vote getters. With a bit of coding I changed the expert consensus to the second tie getter on a extracted dataframe. I than merged the 4000+ into my train. Just getting things running now, but one issue was that I was using eeg_id and eeg_sub_id later for some grouping. My 4000+ addition were duplicates of course and messed up those results. So I added 1000 to the eeg_sub_id before I merged the extracted dataframe into train.
\nI am mostly using the probabilities for my models rather than the expert_consenus but a couple of models do use the expert_consenus. Will see if it helps those models.
", "rawMarkdown": "Nice to know. Thanks\n\nFor the data set I am using I found around 4000+ rows of ties. When chasing another idea last week I had created a column in train for the top 2 vote getters. With a bit of coding I changed the expert consensus to the second tie getter on a extracted dataframe. I than merged the 4000+ into my train. Just getting things running now, but one issue was that I was using eeg_id and eeg_sub_id later for some grouping. My 4000+ addition were duplicates of course and messed up those results. So I added 1000 to the eeg_sub_id before I merged the extracted dataframe into train.\n\nI am mostly using the probabilities for my models rather than the expert_consenus but a couple of models do use the expert_consenus. Will see if it helps those models.", "votes": null }, { "id": "2665957", "postDate": "02/24/2024 02:44:50", "content": "Thank you for reply.
\nI didn’t know that.
\nI was not curious about “expert consensus” because our making model predicts probability.
Thank you for reply.
\n「I am mostly using the probabilities for my models rather than the expert_consenus but a couple of models do use the expert_consenus. Will see if it helps those models.」
\n→I was thinking same things. Basically, in this competition, we use probabilities that model output, but I was thinking using 'expert_consenus' helps.
\nAnd, I was thinking of trying to make a bit change to targets probabilities which has tie votes based on expert_consenus.
In reality, there are 5 IIIC pattern classes not 6, the 'other' class (non-IIIC patterns) contains segments that were noisy and too poor in quality to classify in one of the 5 classes. you can read more about the process of annotation here.
\nIt would be interesting If you can find a tie between two of the five classes except others
['lpd_vote', 'gpd_vote'] : LPD 165
\n['gpd_vote', 'grda_vote'] : GPD 58
\n['gpd_vote', 'lrda_vote'] : GPD 3
\n['lpd_vote', 'lrda_vote'] : LPD 60
\n['lrda_vote', 'grda_vote'] : LRDA 6
\n['seizure_vote', 'gpd_vote'] : Seizure 56
\n['seizure_vote', 'lpd_vote'] : Seizure 90
\n['seizure_vote', 'grda_vote'] : Seizure 3
\n['seizure_vote', 'lrda_vote'] : Seizure 6
\n['seizure_vote', 'other_vote'] : Seizure 147
\n(others always lose)
\netc … inside [] is kinds of votes, after \":\" is expert consensus number.
\nso if u do compare the strength SEIZURE>LPD>GPD>LRDA>GRDA>OTHERS
\nwill make notebook about this soon
Expert_consensus is just argmax(*_vote).
\nargmax in case of equality of votes always returns the very first vote, i.e. voice with \"lower index\".
\nIt's easy to check:
df = pd.read_csv()\n\ntars = {: , : , : , : , : , : }\ntargets = [, , , , , ]\n\ndf[] = df.expert_consensus.(tars)\ndf[] = np.argmax(df[targets].values, axis=)\n\n(df[] != df.target).()\n0
", "votes": null, "replies": [ { "id": 2665957, "author_name": "haruki741", "author_url": "", "post_date": "02/24/2024 02:44:50", "content": "Thank you for reply.
\nI didn’t know that.
\nI was not curious about “expert consensus” because our making model predicts probability.
\" Additionally, would it be quite challenging for a model to make these subtle judgments?\" Not much. The model has to predict the probabilities, not the consensus.
", "votes": null, "replies": [] }, { "id": 2665609, "author_name": "pcjimmmy", "author_url": "", "post_date": "02/23/2024 18:39:01", "content": "Nice to know. Thanks
\nFor the data set I am using I found around 4000+ rows of ties. When chasing another idea last week I had created a column in train for the top 2 vote getters. With a bit of coding I changed the expert consensus to the second tie getter on a extracted dataframe. I than merged the 4000+ into my train. Just getting things running now, but one issue was that I was using eeg_id and eeg_sub_id later for some grouping. My 4000+ addition were duplicates of course and messed up those results. So I added 1000 to the eeg_sub_id before I merged the extracted dataframe into train.
\nI am mostly using the probabilities for my models rather than the expert_consenus but a couple of models do use the expert_consenus. Will see if it helps those models.
", "votes": null, "replies": [ { "id": 2666094, "author_name": "haruki741", "author_url": "", "post_date": "02/24/2024 06:31:29", "content": "Thank you for reply.
\n「I am mostly using the probabilities for my models rather than the expert_consenus but a couple of models do use the expert_consenus. Will see if it helps those models.」
\n→I was thinking same things. Basically, in this competition, we use probabilities that model output, but I was thinking using 'expert_consenus' helps.
\nAnd, I was thinking of trying to make a bit change to targets probabilities which has tie votes based on expert_consenus.
In reality, there are 5 IIIC pattern classes not 6, the 'other' class (non-IIIC patterns) contains segments that were noisy and too poor in quality to classify in one of the 5 classes. you can read more about the process of annotation here.
\nIt would be interesting If you can find a tie between two of the five classes except others