{
  "id": 513135,
  "title": "How do: output feature engineering",
  "url": "/competitions/rsna-2024-lumbar-spine-degenerative-classification/discussion/513135",
  "author_name": "",
  "post_date": "2024-06-18T17:47:04.407393800Z",
  "votes": 5,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I am curious about what approaches people are using. The things to cover that I have figured so far are:</p>\n<ul>\n<li>Handedness: for foraminal narrowing and subarticular stenosis. </li>\n<li>Multi-labeling: since each vertebral joint gets a separate label</li>\n<li>Ordinality: the labels are ordinal scores rather than distinct classes</li>\n</ul>\n<p>And some approaches I have figured:</p>\n<ul>\n<li><p><strong>Continuous score per joint</strong>: With handed features, this would be size 10 and 5 for spinal canal narrowing. This would mean labeling classes as {0.16, 0.5, 0.83} and using {0.33, 0.66} as inference thresholds.<br>\nPros: it's relatively easy to implement, can use just a single head. Captures ordinality.<br>\nCons: Have to get elaborate with the loss function or the model will not train.</p></li>\n<li><p><strong>One-hot encoded 1D tensor</strong>: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], concat in the order of joints.<br>\nPros: also easy to implement and can use a single head.<br>\nCons: Ignores ordinality but the head can probably learn it in the previous layer.</p></li>\n<li><p><strong>Multi-hot encoded 1D tensor</strong>: encode labels as [0, 0], [1, 0] or [1, 1], concat in the order of joints.<br>\nPros: it explicitly captures the relationship between moderate and severe labels.  </p></li>\n<li><p><strong>One-hot encoded 2D tensor</strong>: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], feed separately per joint.<br>\nPros: can use separate losses per joint, more options as to which function to use.<br>\nCons: need to use separate heads which can increase training and inference times.</p></li>\n<li><p><strong>Multi-hot encoded 2D tensor</strong>: encode labels as [0, 0], [0, 1] or [1. 1], feed separately per joint.<br>\nPros: more explicit way to capture ordinality, same pros as above.<br>\nCons: Increased training time same as above.</p></li>\n</ul>\n<p>Any comments on how to do handedness with the later approaches? I have only been experimenting with the T2/STIR series data so far so did not get a chance to test things out with the handed series.</p>",
  "messages": [
    {
      "id": "2878059",
      "postDate": "06/18/2024 17:47:04",
      "content": "<p>I am curious about what approaches people are using. The things to cover that I have figured so far are:</p>\n<ul>\n<li>Handedness: for foraminal narrowing and subarticular stenosis. </li>\n<li>Multi-labeling: since each vertebral joint gets a separate label</li>\n<li>Ordinality: the labels are ordinal scores rather than distinct classes</li>\n</ul>\n<p>And some approaches I have figured:</p>\n<ul>\n<li><p><strong>Continuous score per joint</strong>: With handed features, this would be size 10 and 5 for spinal canal narrowing. This would mean labeling classes as {0.16, 0.5, 0.83} and using {0.33, 0.66} as inference thresholds.<br>\nPros: it's relatively easy to implement, can use just a single head. Captures ordinality.<br>\nCons: Have to get elaborate with the loss function or the model will not train.</p></li>\n<li><p><strong>One-hot encoded 1D tensor</strong>: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], concat in the order of joints.<br>\nPros: also easy to implement and can use a single head.<br>\nCons: Ignores ordinality but the head can probably learn it in the previous layer.</p></li>\n<li><p><strong>Multi-hot encoded 1D tensor</strong>: encode labels as [0, 0], [1, 0] or [1, 1], concat in the order of joints.<br>\nPros: it explicitly captures the relationship between moderate and severe labels.  </p></li>\n<li><p><strong>One-hot encoded 2D tensor</strong>: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], feed separately per joint.<br>\nPros: can use separate losses per joint, more options as to which function to use.<br>\nCons: need to use separate heads which can increase training and inference times.</p></li>\n<li><p><strong>Multi-hot encoded 2D tensor</strong>: encode labels as [0, 0], [0, 1] or [1. 1], feed separately per joint.<br>\nPros: more explicit way to capture ordinality, same pros as above.<br>\nCons: Increased training time same as above.</p></li>\n</ul>\n<p>Any comments on how to do handedness with the later approaches? I have only been experimenting with the T2/STIR series data so far so did not get a chance to test things out with the handed series.</p>",
      "rawMarkdown": "I am curious about what approaches people are using. The things to cover that I have figured so far are:\n- Handedness: for foraminal narrowing and subarticular stenosis. \n- Multi-labeling: since each vertebral joint gets a separate label\n- Ordinality: the labels are ordinal scores rather than distinct classes\n\nAnd some approaches I have figured:\n\n- **Continuous score per joint**: With handed features, this would be size 10 and 5 for spinal canal narrowing. This would mean labeling classes as {0.16, 0.5, 0.83} and using {0.33, 0.66} as inference thresholds.\nPros: it's relatively easy to implement, can use just a single head. Captures ordinality.\nCons: Have to get elaborate with the loss function or the model will not train.\n\n- **One-hot encoded 1D tensor**: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], concat in the order of joints.\nPros: also easy to implement and can use a single head.\nCons: Ignores ordinality but the head can probably learn it in the previous layer.\n\n- **Multi-hot encoded 1D tensor**: encode labels as [0, 0], [1, 0] or [1, 1], concat in the order of joints.\nPros: it explicitly captures the relationship between moderate and severe labels.  \n\n- **One-hot encoded 2D tensor**: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], feed separately per joint.\nPros: can use separate losses per joint, more options as to which function to use.\nCons: need to use separate heads which can increase training and inference times.\n\n- **Multi-hot encoded 2D tensor**: encode labels as [0, 0], [0, 1] or [1. 1], feed separately per joint.\nPros: more explicit way to capture ordinality, same pros as above.\nCons: Increased training time same as above.\n\nAny comments on how to do handedness with the later approaches? I have only been experimenting with the T2/STIR series data so far so did not get a chance to test things out with the handed series.",
      "votes": null
    },
    {
      "id": "2882134",
      "postDate": "06/21/2024 07:10:11",
      "content": "<p>I use multihead which the output shape is (L, C) where L=# of levels(l1/12, l2/l3…) and C=# of condition(normal/mod/serv)</p>",
      "rawMarkdown": "I use multihead which the output shape is (L, C) where L=# of levels(l1/12, l2/l3...) and C=# of condition(normal/mod/serv)",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2882134,
      "author_name": "sakurayuyuko",
      "author_url": "",
      "post_date": "06/21/2024 07:10:11",
      "content": "<p>I use multihead which the output shape is (L, C) where L=# of levels(l1/12, l2/l3…) and C=# of condition(normal/mod/serv)</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2878059": "I am curious about what approaches people are using. The things to cover that I have figured so far are:\n- Handedness: for foraminal narrowing and subarticular stenosis. \n- Multi-labeling: since each vertebral joint gets a separate label\n- Ordinality: the labels are ordinal scores rather than distinct classes\n\nAnd some approaches I have figured:\n\n- **Continuous score per joint**: With handed features, this would be size 10 and 5 for spinal canal narrowing. This would mean labeling classes as {0.16, 0.5, 0.83} and using {0.33, 0.66} as inference thresholds.\nPros: it's relatively easy to implement, can use just a single head. Captures ordinality.\nCons: Have to get elaborate with the loss function or the model will not train.\n\n- **One-hot encoded 1D tensor**: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], concat in the order of joints.\nPros: also easy to implement and can use a single head.\nCons: Ignores ordinality but the head can probably learn it in the previous layer.\n\n- **Multi-hot encoded 1D tensor**: encode labels as [0, 0], [1, 0] or [1, 1], concat in the order of joints.\nPros: it explicitly captures the relationship between moderate and severe labels.  \n\n- **One-hot encoded 2D tensor**: encode labels as [1, 0, 0], [0, 1, 0] or [0, 0, 1], feed separately per joint.\nPros: can use separate losses per joint, more options as to which function to use.\nCons: need to use separate heads which can increase training and inference times.\n\n- **Multi-hot encoded 2D tensor**: encode labels as [0, 0], [0, 1] or [1. 1], feed separately per joint.\nPros: more explicit way to capture ordinality, same pros as above.\nCons: Increased training time same as above.\n\nAny comments on how to do handedness with the later approaches? I have only been experimenting with the T2/STIR series data so far so did not get a chance to test things out with the handed series.",
    "2882134": "I use multihead which the output shape is (L, C) where L=# of levels(l1/12, l2/l3...) and C=# of condition(normal/mod/serv)"
  },
  "source": "meta"
}