{
  "id": 402875,
  "title": "Has anybody tried MinkowskiEngine?",
  "url": "/competitions/icecube-neutrinos-in-deep-ice/discussion/402875",
  "author_name": "slime",
  "post_date": "2023-04-20T03:28:07.383000",
  "votes": 0,
  "comment_count": 0,
  "views": 0,
  "content": "<p>We tried various things, - GNNs, transformers with pulses as tokens and the most interesting thing - <a href=\"https://github.com/NVIDIA/MinkowskiEngine\" target=\"_blank\">MinkowskiEngine</a>, this library allows for use of 4D (xyzt) sparse convolutions, I thought that it could be the key to this type of problem, because neutrino direction is depended on the shape of the event in time-space, however I couldn't make resnet-like model train better than a 10-layer transformer with 9 emedded features (6-&gt;64).</p>\n<p>The good things is that the library allows for input tensors to be sparse as well, so during training consumed amount of VRAM is reasonable, however I found that transformer with the same representational ability trained way faster anyways.  </p>",
  "messages": [
    {
      "id": 2227813,
      "postDate": "2023-04-20T03:28:07.383Z",
      "content": "<p>We tried various things, - GNNs, transformers with pulses as tokens and the most interesting thing - <a href=\"https://github.com/NVIDIA/MinkowskiEngine\" target=\"_blank\">MinkowskiEngine</a>, this library allows for use of 4D (xyzt) sparse convolutions, I thought that it could be the key to this type of problem, because neutrino direction is depended on the shape of the event in time-space, however I couldn't make resnet-like model train better than a 10-layer transformer with 9 emedded features (6-&gt;64).</p>\n<p>The good things is that the library allows for input tensors to be sparse as well, so during training consumed amount of VRAM is reasonable, however I found that transformer with the same representational ability trained way faster anyways.  </p>",
      "rawMarkdown": "We tried various things, - GNNs, transformers with pulses as tokens and the most interesting thing - [MinkowskiEngine](https://github.com/NVIDIA/MinkowskiEngine), this library allows for use of 4D (xyzt) sparse convolutions, I thought that it could be the key to this type of problem, because neutrino direction is depended on the shape of the event in time-space, however I couldn't make resnet-like model train better than a 10-layer transformer with 9 emedded features (6->64).\n\nThe good things is that the library allows for input tensors to be sparse as well, so during training consumed amount of VRAM is reasonable, however I found that transformer with the same representational ability trained way faster anyways.  "
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2227813": "We tried various things, - GNNs, transformers with pulses as tokens and the most interesting thing - [MinkowskiEngine](https://github.com/NVIDIA/MinkowskiEngine), this library allows for use of 4D (xyzt) sparse convolutions, I thought that it could be the key to this type of problem, because neutrino direction is depended on the shape of the event in time-space, however I couldn't make resnet-like model train better than a 10-layer transformer with 9 emedded features (6->64).\n\nThe good things is that the library allows for input tensors to be sparse as well, so during training consumed amount of VRAM is reasonable, however I found that transformer with the same representational ability trained way faster anyways.  "
  }
}