{
  "id": 567360,
  "title": "Understanding the Competition Data",
  "url": "/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/567360",
  "author_name": "SSS",
  "post_date": "2025-03-09T22:24:30.327000",
  "votes": 69,
  "comment_count": 42,
  "views": 0,
  "content": "<p>There are <strong>648</strong> unique subdirectories in the <code>train</code> folder, each corresponding to a specific tomogram. Each train tomogram subdirectory contains <code>2D</code> slices, which can be stacked to reconstruct a <code>3D</code> tomogram.  </p>\n<p>In total, there are <strong>269,194</strong> <code>.jpg</code> slices across all train tomogram subdirectories. Some <code>3D</code> tomograms can be storage-intensive, reaching up to <strong>1.7GB</strong> in size. When unpacked, the total volume of all train examples amounts to approximately <strong>236GB</strong> (<code>uint8</code> format).  </p>\n<hr>\n<h2>Explanation and Schema</h2>\n<p>The dataset consists of standard <strong>train</strong> and <strong>test</strong> folders. The <strong>test</strong> folder contains three directories of dummy test tomograms, while the actual rerun test dataset includes approximately <strong>900</strong> tomograms. <strong>Test data only contains tomograms with either one or zero motors.</strong>  </p>\n<p>The <strong>train</strong> folder contains <strong>648</strong> tomograms, each sliced into <code>.jpg</code> files. The <code>train_labels.csv</code> file provides the <code>z, y, x</code> coordinates of the target of interest. If no target is present in a tomogram, the coordinates are set to <code>-1, -1, -1</code>. Similarly, during submission, <code>-1, -1, -1</code> should be used when predicting <code>0</code> targets.  </p>\n<h2>Below is the descriptive data schema and a visualization of a tomogram:  </h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F156fe5b03a449127cefe3cfae561c6c9%2FUnderstanding%20comp%20data.png?generation=1741558489474098&amp;alt=media\" alt=\"\"></p>\n<hr>\n<h3>Have Fun!</h3>\n<p><a href=\"https://www.kaggle.com/code/sergiosaharovskiy/byu-2025-eda-viz-pseudo-leaks\" target=\"_blank\">Code</a> - slowly working through.<br>\n<a href=\"https://www.kaggle.com/datasets/sergiosaharovskiy/2025-byu-locating-bacterial-motors-public-repo\" target=\"_blank\">Repo</a> - external scripts &amp; data</p>",
  "messages": [
    {
      "id": 3145500,
      "postDate": "2025-03-09T22:24:30.327Z",
      "content": "<p>There are <strong>648</strong> unique subdirectories in the <code>train</code> folder, each corresponding to a specific tomogram. Each train tomogram subdirectory contains <code>2D</code> slices, which can be stacked to reconstruct a <code>3D</code> tomogram.  </p>\n<p>In total, there are <strong>269,194</strong> <code>.jpg</code> slices across all train tomogram subdirectories. Some <code>3D</code> tomograms can be storage-intensive, reaching up to <strong>1.7GB</strong> in size. When unpacked, the total volume of all train examples amounts to approximately <strong>236GB</strong> (<code>uint8</code> format).  </p>\n<hr>\n<h2>Explanation and Schema</h2>\n<p>The dataset consists of standard <strong>train</strong> and <strong>test</strong> folders. The <strong>test</strong> folder contains three directories of dummy test tomograms, while the actual rerun test dataset includes approximately <strong>900</strong> tomograms. <strong>Test data only contains tomograms with either one or zero motors.</strong>  </p>\n<p>The <strong>train</strong> folder contains <strong>648</strong> tomograms, each sliced into <code>.jpg</code> files. The <code>train_labels.csv</code> file provides the <code>z, y, x</code> coordinates of the target of interest. If no target is present in a tomogram, the coordinates are set to <code>-1, -1, -1</code>. Similarly, during submission, <code>-1, -1, -1</code> should be used when predicting <code>0</code> targets.  </p>\n<h2>Below is the descriptive data schema and a visualization of a tomogram:  </h2>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F156fe5b03a449127cefe3cfae561c6c9%2FUnderstanding%20comp%20data.png?generation=1741558489474098&amp;alt=media\" alt=\"\"></p>\n<hr>\n<h3>Have Fun!</h3>\n<p><a href=\"https://www.kaggle.com/code/sergiosaharovskiy/byu-2025-eda-viz-pseudo-leaks\" target=\"_blank\">Code</a> - slowly working through.<br>\n<a href=\"https://www.kaggle.com/datasets/sergiosaharovskiy/2025-byu-locating-bacterial-motors-public-repo\" target=\"_blank\">Repo</a> - external scripts &amp; data</p>",
      "rawMarkdown": "There are **648** unique subdirectories in the `train` folder, each corresponding to a specific tomogram. Each train tomogram subdirectory contains `2D` slices, which can be stacked to reconstruct a `3D` tomogram.  \n\nIn total, there are **269,194** `.jpg` slices across all train tomogram subdirectories. Some `3D` tomograms can be storage-intensive, reaching up to **1.7GB** in size. When unpacked, the total volume of all train examples amounts to approximately **236GB** (`uint8` format).  \n\n---\n\n## Explanation and Schema  \n\nThe dataset consists of standard **train** and **test** folders. The **test** folder contains three directories of dummy test tomograms, while the actual rerun test dataset includes approximately **900** tomograms. **Test data only contains tomograms with either one or zero motors.**  \n\nThe **train** folder contains **648** tomograms, each sliced into `.jpg` files. The `train_labels.csv` file provides the `z, y, x` coordinates of the target of interest. If no target is present in a tomogram, the coordinates are set to `-1, -1, -1`. Similarly, during submission, `-1, -1, -1` should be used when predicting `0` targets.  \n\nBelow is the descriptive data schema and a visualization of a tomogram:  \n---\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F156fe5b03a449127cefe3cfae561c6c9%2FUnderstanding%20comp%20data.png?generation=1741558489474098&alt=media)\n\n---\n### Have Fun!\n[Code](https://www.kaggle.com/code/sergiosaharovskiy/byu-2025-eda-viz-pseudo-leaks) - slowly working through.\n[Repo](https://www.kaggle.com/datasets/sergiosaharovskiy/2025-byu-locating-bacterial-motors-public-repo) - external scripts & data",
      "votes": 68
    },
    {
      "id": 3174438,
      "postDate": "2025-04-09T05:35:07.113Z",
      "content": "<p>Thanks for the detailed explanation! Really helps to understand how the dataset is structured. Rebuilding the 3D tomograms from slices and predicting the target location sounds challenging but also very interesting. Looking forward to working on it!</p>",
      "rawMarkdown": "Thanks for the detailed explanation! Really helps to understand how the dataset is structured. Rebuilding the 3D tomograms from slices and predicting the target location sounds challenging but also very interesting. Looking forward to working on it!",
      "votes": 1
    },
    {
      "id": 3147938,
      "postDate": "2025-03-12T15:16:23.763Z",
      "content": "<p>What about conversion to Zarr, this dataset is outrageously large?</p>",
      "rawMarkdown": "What about conversion to Zarr, this dataset is outrageously large?",
      "votes": 1,
      "replies": [
        {
          "id": 3147962,
          "postDate": "2025-03-12T15:44:40.970Z",
          "content": "<p>Hi, I've never really seen someone take that direction and convert to Zarr. Ultimately, we’ll end up with NumPy arrays and then tensors for modeling. Zarr can help with I/O and caching, but it adds some extra complexity and a learning curve. What I usually do—when space permits—is convert everything to .npy files to speed up I/O, compromising a bit on storage space. But honestly, who cares if you’ve got a good NVMe SSD anyway? The GPU VRAM though, is another story, which is more challenging sometimes.</p>",
          "rawMarkdown": "Hi, I've never really seen someone take that direction and convert to Zarr. Ultimately, we’ll end up with NumPy arrays and then tensors for modeling. Zarr can help with I/O and caching, but it adds some extra complexity and a learning curve. What I usually do—when space permits—is convert everything to .npy files to speed up I/O, compromising a bit on storage space. But honestly, who cares if you’ve got a good NVMe SSD anyway? The GPU VRAM though, is another story, which is more challenging sometimes.",
          "votes": 4
        }
      ]
    },
    {
      "id": 3147064,
      "postDate": "2025-03-11T15:40:50.900Z",
      "content": "<p>to how much extent  can we can we decrease the unpacked data size (by under sampling and other techniques) without hurting the performance , can we reach below 80 GB?</p>",
      "rawMarkdown": "to how much extent  can we can we decrease the unpacked data size (by under sampling and other techniques) without hurting the performance , can we reach below 80 GB?",
      "votes": 2,
      "replies": [
        {
          "id": 3147085,
          "postDate": "2025-03-11T16:20:45.243Z",
          "content": "<p>There are some:</p>\n<ol>\n<li>Maybe try Sparse sampling and take odd slices only.</li>\n<li>Maybe resize the images in the beginning to test the pipeline.</li>\n<li>Maybe crop the images and take only specific ROI.</li>\n<li>Maybe discard samples with more than 1 motors since in the test there is going to be only zero and one motors examples.</li>\n<li>FP16 training</li>\n</ol>\n<p>There are more maybe(s) but at the moment I have not started with the experimenting yet.</p>",
          "rawMarkdown": "There are some:\n\n1. Maybe try Sparse sampling and take odd slices only.\n2. Maybe resize the images in the beginning to test the pipeline.\n3. Maybe crop the images and take only specific ROI.\n4. Maybe discard samples with more than 1 motors since in the test there is going to be only zero and one motors examples.\n5. FP16 training\n\nThere are more maybe(s) but at the moment I have not started with the experimenting yet.",
          "votes": 6,
          "replies": [
            {
              "id": 3147963,
              "postDate": "2025-03-12T15:48:08.527Z",
              "content": "<p>4 is guaranteed ?</p>",
              "rawMarkdown": "4 is guaranteed ?",
              "votes": 2
            },
            {
              "id": 3147970,
              "postDate": "2025-03-12T15:57:02.277Z",
              "content": "<p>Hi Tom, yes, according to the data page</p>\n<blockquote>\n  <p>test/: Directory with 3 directories of dummy test tomograms; the rerun test dataset contains approximately 900 tomograms. <strong><em>The test data only contain tomograms with one or zero motors.</em></strong> </p>\n</blockquote>\n<p>Though we can ask the host, hi <a href=\"https://www.kaggle.com/braxtonowens\" target=\"_blank\">@braxtonowens</a>, <a href=\"https://www.kaggle.com/jacksonpond\" target=\"_blank\">@jacksonpond</a> can you confirm that all 900 test tomograms only contain one or zero motors?</p>",
              "rawMarkdown": "Hi Tom, yes, according to the data page\n>test/: Directory with 3 directories of dummy test tomograms; the rerun test dataset contains approximately 900 tomograms. ***The test data only contain tomograms with one or zero motors.*** \n\nThough we can ask the host, hi @braxtonowens, @jacksonpond can you confirm that all 900 test tomograms only contain one or zero motors?",
              "votes": 4
            },
            {
              "id": 3147983,
              "postDate": "2025-03-12T16:14:37.233Z",
              "content": "<p>The competition metric also requires one row per tomography id, so you can make only a prediction for one motor location or no motor at all. A confirmation that they only contain tomograms with one or zero motors would be great though!</p>",
              "rawMarkdown": "The competition metric also requires one row per tomography id, so you can make only a prediction for one motor location or no motor at all. A confirmation that they only contain tomograms with one or zero motors would be great though!",
              "votes": 3
            },
            {
              "id": 3148361,
              "postDate": "2025-03-13T03:20:49.520Z",
              "content": "<p>So we can just use a linear layer to tell whether the image contains motor and its coordinate. <code>[cls, z, y, x]</code>.<br>\nI think that's a good alternative approach.</p>",
              "rawMarkdown": "So we can just use a linear layer to tell whether the image contains motor and its coordinate. `[cls, z, y, x]`.\nI think that's a good alternative approach.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 3146994,
      "postDate": "2025-03-11T14:39:23.303Z",
      "content": "<p>Hi, I have a question about the height and width of .jpg files. <br>\nTake tomo_00e047 as an example. In train_labels.csv, Array shape (axis 1) = 959 (width of each slice), Array shape (axis 2) = 928 (height of each slice). However, the .jpg file has a height of 959 and a width of 928.</p>",
      "rawMarkdown": "Hi, I have a question about the height and width of .jpg files. \nTake tomo_00e047 as an example. In train_labels.csv, Array shape (axis 1) = 959 (width of each slice), Array shape (axis 2) = 928 (height of each slice). However, the .jpg file has a height of 959 and a width of 928.",
      "votes": 2,
      "replies": [
        {
          "id": 3147039,
          "postDate": "2025-03-11T15:18:56.067Z",
          "content": "<p>Hi,</p>\n<p>The <code>train_labels.csv</code> file provides the <code>z, y, x</code> w.r.t axis-0, axis-1, axis2, which means (num_slices, height, width). Here is the data page snippet:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F3538235b52ff17bc2ee86628e02a47ce%2FMonosnap%20BYU%20-%20Locating%20Bacterial%20Flagellar%20Motors.png?generation=1741706319127052&amp;alt=media\" alt=\"\"></p>\n<p>Hope, that helps.</p>",
          "rawMarkdown": "Hi,\n\nThe `train_labels.csv` file provides the `z, y, x` w.r.t axis-0, axis-1, axis2, which means (num_slices, height, width). Here is the data page snippet:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F3538235b52ff17bc2ee86628e02a47ce%2FMonosnap%20BYU%20-%20Locating%20Bacterial%20Flagellar%20Motors.png?generation=1741706319127052&alt=media)\n\nHope, that helps.",
          "votes": 2,
          "replies": [
            {
              "id": 3147043,
              "postDate": "2025-03-11T15:21:04.173Z",
              "content": "<p>Array shape axis 1: y-axis \"width\" of each slice.</p>",
              "rawMarkdown": "Array shape axis 1: y-axis \"width\" of each slice.",
              "votes": 1
            },
            {
              "id": 3147051,
              "postDate": "2025-03-11T15:28:10.100Z",
              "content": "<p>yikes, it broke everything in my head now, hidden parallel world, illuminati!</p>",
              "rawMarkdown": "yikes, it broke everything in my head now, hidden parallel world, illuminati!"
            }
          ]
        }
      ]
    },
    {
      "id": 3146634,
      "postDate": "2025-03-11T04:44:16.230Z",
      "content": "<p>Can a single 4090 win this competition?</p>",
      "rawMarkdown": "Can a single 4090 win this competition?",
      "votes": 2,
      "replies": [
        {
          "id": 3146698,
          "postDate": "2025-03-11T06:58:06.693Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 3146810,
          "postDate": "2025-03-11T09:22:16.197Z",
          "content": "<p>:) Let’s assume it takes 24 hours for the training to run all examples split into 5 folds. Top contenders run more than 500 experiments up to 1500 experiments. So good question - probably not if you are up to the number of experiments and full dataset runs. Though if you experiment on the small curated subset and then run the whole dataset then one 4090 is doable. Some folks managed to win such competitions by only using kaggle notebooks. P.S. I am sure you knew the answer.</p>",
          "rawMarkdown": ":) Let’s assume it takes 24 hours for the training to run all examples split into 5 folds. Top contenders run more than 500 experiments up to 1500 experiments. So good question - probably not if you are up to the number of experiments and full dataset runs. Though if you experiment on the small curated subset and then run the whole dataset then one 4090 is doable. Some folks managed to win such competitions by only using kaggle notebooks. P.S. I am sure you knew the answer.",
          "votes": 3,
          "replies": [
            {
              "id": 3146867,
              "postDate": "2025-03-11T10:50:01.213Z",
              "content": "<p>I guess. It is not enough. Will external dataset help in this competition?</p>",
              "rawMarkdown": "I guess. It is not enough. Will external dataset help in this competition?"
            },
            {
              "id": 3146881,
              "postDate": "2025-03-11T11:01:32.657Z",
              "content": "<p>Yes, but only if you run your 4090 in turbo mode while chanting 'ReLU' three times under a full moon. Also, don't forget to overclock your CPU by applying deep learning principles to your cooling system.</p>\n<p>though, honestly, idk</p>",
              "rawMarkdown": "Yes, but only if you run your 4090 in turbo mode while chanting 'ReLU' three times under a full moon. Also, don't forget to overclock your CPU by applying deep learning principles to your cooling system.\n\nthough, honestly, idk",
              "votes": 3
            }
          ]
        },
        {
          "id": 3146840,
          "postDate": "2025-03-11T10:11:30.267Z",
          "content": "<p><a href=\"https://www.kaggle.com/yuanzhezhou\" target=\"_blank\">@yuanzhezhou</a> I think fork notebook is enough</p>",
          "rawMarkdown": "@yuanzhezhou I think fork notebook is enough",
          "votes": 1,
          "replies": [
            {
              "id": 3147071,
              "postDate": "2025-03-11T15:53:07.367Z",
              "content": "<p><a href=\"https://www.kaggle.com/tom99763\" target=\"_blank\">@tom99763</a> <a href=\"https://www.kaggle.com/andreizamfir\" target=\"_blank\">@andreizamfir</a> <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> I am completely relying on GPU quota provided by kaggle .Is there still a chance to score good in this competition using kaggle GPU quota only ?</p>",
              "rawMarkdown": "@tom99763 @andreizamfir @sergiosaharovskiy I am completely relying on GPU quota provided by kaggle .Is there still a chance to score good in this competition using kaggle GPU quota only ?",
              "votes": 1,
              "isDeleted": true
            },
            {
              "id": 3147080,
              "postDate": "2025-03-11T16:03:19.980Z",
              "content": "<p>Tough but possible (don't let it to discourage you). Smart data sampling, efficient training, and selective runs can help. Some have won using only Kaggle's free tier, but full dataset runs will be a challenge. Make every experiment count.</p>\n<p>p.s. traditionally, deep learning competitions were compute hungry, and this is one of them.</p>",
              "rawMarkdown": "Tough but possible (don't let it to discourage you). Smart data sampling, efficient training, and selective runs can help. Some have won using only Kaggle's free tier, but full dataset runs will be a challenge. Make every experiment count.\n\np.s. traditionally, deep learning competitions were compute hungry, and this is one of them.",
              "votes": 3
            },
            {
              "id": 3147082,
              "postDate": "2025-03-11T16:10:01.193Z",
              "content": "<p>I am only working with Kaggle GPU. While it's harder, and for some competitions certainly impossible (don't think it's impossible here), still doable to get good results.</p>",
              "rawMarkdown": "I am only working with Kaggle GPU. While it's harder, and for some competitions certainly impossible (don't think it's impossible here), still doable to get good results.",
              "votes": 6
            },
            {
              "id": 3160525,
              "postDate": "2025-03-26T22:03:33.243Z",
              "content": "<p>Wow, thank you for sharing.</p>",
              "rawMarkdown": "Wow, thank you for sharing.",
              "votes": 11
            }
          ]
        },
        {
          "id": 3147115,
          "postDate": "2025-03-11T16:54:30.510Z",
          "content": "<p>Hi, do you guys know where to get cheap gpu (VMs)? Google Colab is very expensive</p>",
          "rawMarkdown": "Hi, do you guys know where to get cheap gpu (VMs)? Google Colab is very expensive",
          "replies": [
            {
              "id": 3147126,
              "postDate": "2025-03-11T17:14:54.697Z",
              "content": "<p><a href=\"https://vast.ai/\" target=\"_blank\">https://vast.ai/</a><br>\n<a href=\"https://www.runpod.io/\" target=\"_blank\">https://www.runpod.io/</a></p>",
              "rawMarkdown": "https://vast.ai/\nhttps://www.runpod.io/",
              "votes": 4,
              "isDeleted": true
            }
          ]
        },
        {
          "id": 3150185,
          "postDate": "2025-03-15T06:24:38.840Z",
          "content": "<p><a href=\"https://www.kaggle.com/yuanzhezhou\" target=\"_blank\">@yuanzhezhou</a> 2 RTX4090 take 2 hours run for training 3DUnet. Submission on kaggle takes 90min.</p>",
          "rawMarkdown": "@yuanzhezhou 2 RTX4090 take 2 hours run for training 3DUnet. Submission on kaggle takes 90min.",
          "votes": 2,
          "replies": [
            {
              "id": 3151234,
              "postDate": "2025-03-16T13:01:48.020Z",
              "content": "<p>Is 24GB not enough for training?</p>",
              "rawMarkdown": "Is 24GB not enough for training?"
            },
            {
              "id": 3151759,
              "postDate": "2025-03-17T04:22:12.047Z",
              "content": "<p>It seems that the main problem is the shake up … Data size is not that problematic.</p>",
              "rawMarkdown": "It seems that the main problem is the shake up ... Data size is not that problematic.",
              "votes": 2
            },
            {
              "id": 3151766,
              "postDate": "2025-03-17T04:26:01.347Z",
              "content": "<p>But I think object detection will outperform segmentation in this comp. The ground truth is so sparse. Post processing for segmentation model is really really hard</p>",
              "rawMarkdown": "But I think object detection will outperform segmentation in this comp. The ground truth is so sparse. Post processing for segmentation model is really really hard",
              "votes": 2
            }
          ]
        }
      ]
    },
    {
      "id": 3149409,
      "postDate": "2025-03-14T07:04:56.013Z",
      "content": "<p>Do we know the actual quantity of test images ? I men the hidden one</p>",
      "rawMarkdown": "Do we know the actual quantity of test images ? I men the hidden one\n",
      "votes": 1,
      "isDeleted": true,
      "replies": [
        {
          "id": 3149412,
          "postDate": "2025-03-14T07:09:36.133Z",
          "content": "<blockquote>\n  <p>the rerun test dataset contains approximately 900 tomograms</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/data\" target=\"_blank\">https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/data</a></p>\n<blockquote>\n  <p>This leaderboard is calculated with approximately 30% of the test data. The final results will be based on the other 70%, so the final standings may be different.</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/leaderboard\" target=\"_blank\">https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/leaderboard</a></p>",
          "rawMarkdown": ">the rerun test dataset contains approximately 900 tomograms\n\nhttps://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/data\n\n>This leaderboard is calculated with approximately 30% of the test data. The final results will be based on the other 70%, so the final standings may be different.\n\nhttps://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/leaderboard",
          "votes": 3,
          "replies": [
            {
              "id": 3151796,
              "postDate": "2025-03-17T05:38:46.360Z",
              "content": "<p>Thanks the give a fair idea on the submission strategy …</p>",
              "rawMarkdown": "Thanks the give a fair idea on the submission strategy ...",
              "votes": 1,
              "isDeleted": true
            }
          ]
        }
      ]
    },
    {
      "id": 3146410,
      "postDate": "2025-03-10T20:32:05.357Z",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> Thanks for your helpful description. </p>\n<p>Lets take example of <strong>tomo_00e047</strong> consists</p>\n<p>Motor axis 0 = 169<br>\nMotor axis 1 = 546<br>\nMotor axis 2 = 603</p>\n<p>Array shape (axis 0) = 300<br>\nArray shape (axis 1) = 959<br>\nArray shape (axis 2) = 928</p>\n<p>This particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. <strong>Motor axis 0</strong> means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ? </p>\n<p>Also what is the meaning of voxel spacing ?</p>",
      "rawMarkdown": "Hi @sergiosaharovskiy Thanks for your helpful description. \n\nLets take example of **tomo_00e047** consists\n\nMotor axis 0 = 169\nMotor axis 1 = 546\nMotor axis 2 = 603\n\nArray shape (axis 0) = 300\nArray shape (axis 1) = 959\nArray shape (axis 2) = 928\n\nThis particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. **Motor axis 0** means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ? \n\nAlso what is the meaning of voxel spacing ?\n\n\n\n",
      "votes": 3,
      "isDeleted": true,
      "replies": [
        {
          "id": 3146421,
          "postDate": "2025-03-10T20:56:29.393Z",
          "content": "<p>Hi, </p>\n<blockquote>\n  <p>This particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. Motor axis 0 means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ?</p>\n</blockquote>\n<p>This is correct</p>\n<blockquote>\n  <p>Also what is the meaning of voxel spacing ?</p>\n</blockquote>\n<p>Voxel spacing refers to the physical distance between adjacent voxels in a 3D volume (voxels are small cubes )<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Fde793a087e31ce68a4f752cadcb1c0b4%2Fvox%20spacing.png?generation=1741640137562554&amp;alt=media\" alt=\"\"></p>\n<p><a href=\"https://www.researchgate.net/figure/The-image-shows-the-26-voxel-neighborhood-used-for-the-pseudo-3D-key-point-calculation_fig5_281337843\" target=\"_blank\">image source</a></p>",
          "rawMarkdown": "Hi, \n\n>This particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. Motor axis 0 means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ?\n\nThis is correct\n\n>Also what is the meaning of voxel spacing ?\n\nVoxel spacing refers to the physical distance between adjacent voxels in a 3D volume (voxels are small cubes )\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Fde793a087e31ce68a4f752cadcb1c0b4%2Fvox%20spacing.png?generation=1741640137562554&alt=media)\n\n[image source](https://www.researchgate.net/figure/The-image-shows-the-26-voxel-neighborhood-used-for-the-pseudo-3D-key-point-calculation_fig5_281337843)",
          "votes": 8,
          "replies": [
            {
              "id": 3148610,
              "postDate": "2025-03-13T10:39:45.960Z",
              "content": "<p><a href=\"https://www.kaggle.com/andreizamfir\" target=\"_blank\">@andreizamfir</a> <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> Again taking example of tomo_id = 'tomo_00e047 '. There is flagella motor present at slice 169. But if you watch slice number 165 to 175 you can still notice flagella motor. Is this because actual image is in 3D format and we have given 2D slices and instead of taking circle at 1000 angstrom it is actually a cube of 1000 angstrom ?</p>",
              "rawMarkdown": "@andreizamfir @sergiosaharovskiy Again taking example of tomo_id = 'tomo_00e047 '. There is flagella motor present at slice 169. But if you watch slice number 165 to 175 you can still notice flagella motor. Is this because actual image is in 3D format and we have given 2D slices and instead of taking circle at 1000 angstrom it is actually a cube of 1000 angstrom ?",
              "votes": 5,
              "isDeleted": true
            },
            {
              "id": 3148626,
              "postDate": "2025-03-13T10:57:19.667Z",
              "content": "<p>This is a good question. Yes, the nature of the tomogram is 3d. The flagella motor is spatial object so it can span through multiple spices. So when you create your labels you might want to span it over multiple adjacent slices!</p>",
              "rawMarkdown": "This is a good question. Yes, the nature of the tomogram is 3d. The flagella motor is spatial object so it can span through multiple spices. So when you create your labels you might want to span it over multiple adjacent slices!",
              "votes": 2
            },
            {
              "id": 3148813,
              "postDate": "2025-03-13T14:50:49.597Z",
              "content": "<p>Yes, the tomogram is a 3d reconstruction of 2d projections of cryo-electron microscopies, at different inclination angles. There's a good intuition of the process on <a href=\"https://cryoem101.org/chapter-4-et/\" target=\"_blank\">this</a> page with a slider that lets you play around with the tilt angle, and of course, a comprehensive explanation of the methodology and implications.</p>\n<p>As the motor is 3d, it will span through the depth dimension, which is provided through different neighbouring slices. As the labels are supposed to be the centroids of the structures, you get the coordinates of the slice in the Z dimension that belongs to the center, but that doesn't mean that you won't have the motor on other neighbouring slices as well. It's just as you move around the x or y axis next to the vicinity of the center, these pixels still depict the center. This raises interesting questions about the way we want to define our labels for training, which is a crucial part for any supervised learning problem.</p>\n<p>There's also a discussion <a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/567067\" target=\"_blank\">topic</a> addressing this, started by <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a> where the dimensions of the motor are taken into consideration.</p>\n<p>The 1000 angstroms are not to be confused with the size of the motor, it's just the distance threshold defined in the competition metric for a particular prediction to be considered valid. 1e-9 meters = 1 nanometer = 10 angstroms.</p>",
              "rawMarkdown": "Yes, the tomogram is a 3d reconstruction of 2d projections of cryo-electron microscopies, at different inclination angles. There's a good intuition of the process on [this](https://cryoem101.org/chapter-4-et/) page with a slider that lets you play around with the tilt angle, and of course, a comprehensive explanation of the methodology and implications.\n\nAs the motor is 3d, it will span through the depth dimension, which is provided through different neighbouring slices. As the labels are supposed to be the centroids of the structures, you get the coordinates of the slice in the Z dimension that belongs to the center, but that doesn't mean that you won't have the motor on other neighbouring slices as well. It's just as you move around the x or y axis next to the vicinity of the center, these pixels still depict the center. This raises interesting questions about the way we want to define our labels for training, which is a crucial part for any supervised learning problem.\n\nThere's also a discussion [topic](https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/567067) addressing this, started by @gunesevitan where the dimensions of the motor are taken into consideration.\n\nThe 1000 angstroms are not to be confused with the size of the motor, it's just the distance threshold defined in the competition metric for a particular prediction to be considered valid. 1e-9 meters = 1 nanometer = 10 angstroms.",
              "votes": 2
            },
            {
              "id": 3148840,
              "postDate": "2025-03-13T15:13:04.563Z",
              "content": "<p>So something like building gaussian balls might works well.</p>",
              "rawMarkdown": "So something like building gaussian balls might works well.",
              "votes": 2
            },
            {
              "id": 3148946,
              "postDate": "2025-03-13T16:57:44.207Z",
              "content": "<p>I might share the code for that, if you can be so generous?;)</p>",
              "rawMarkdown": "I might share the code for that, if you can be so generous?;)"
            }
          ]
        },
        {
          "id": 3146423,
          "postDate": "2025-03-10T21:04:49.477Z",
          "content": "<p>The motor will be present in slice 169 from 300, at 546 (out of 959), 603 (out of 928). Voxel spacing gives you the reference units of dimension, it’s expressed in angstroms (0.1 nm or 1e-10 meters). </p>\n<p>You basically read it as, for this particular tomography, there’s voxel_spacing angstroms between 2 pixels on the same axis; if you multiply the dimensions of the tomography with the voxel spacing, you get its size in angstroms.</p>\n<p>This is directly linked with the competition metric threshold of 1000 angstroms for a prediction to be considered valid.</p>",
          "rawMarkdown": "The motor will be present in slice 169 from 300, at 546 (out of 959), 603 (out of 928). Voxel spacing gives you the reference units of dimension, it’s expressed in angstroms (0.1 nm or 1e-10 meters). \n\nYou basically read it as, for this particular tomography, there’s voxel_spacing angstroms between 2 pixels on the same axis; if you multiply the dimensions of the tomography with the voxel spacing, you get its size in angstroms.\n\nThis is directly linked with the competition metric threshold of 1000 angstroms for a prediction to be considered valid.",
          "votes": 5
        }
      ]
    },
    {
      "id": 3207747,
      "postDate": "2025-05-23T07:30:56.703Z",
      "content": "<p>Thanks for posting!</p>",
      "rawMarkdown": "Thanks for posting!",
      "votes": 2
    },
    {
      "id": 3153157,
      "postDate": "2025-03-18T13:13:20.113Z",
      "content": "<p>Thank you for sharing</p>",
      "rawMarkdown": "Thank you for sharing",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 3174438,
      "author_name": "M.Ashhad Ur Rehman khan",
      "author_url": "",
      "post_date": "2025-04-09T05:35:07.113000",
      "content": "<p>Thanks for the detailed explanation! Really helps to understand how the dataset is structured. Rebuilding the 3D tomograms from slices and predicting the target location sounds challenging but also very interesting. Looking forward to working on it!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3147938,
      "author_name": "MK245",
      "author_url": "",
      "post_date": "2025-03-12T15:16:23.763000",
      "content": "<p>What about conversion to Zarr, this dataset is outrageously large?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3147962,
          "author_name": "SSS",
          "author_url": "",
          "post_date": "2025-03-12T15:44:40.970000",
          "content": "<p>Hi, I've never really seen someone take that direction and convert to Zarr. Ultimately, we’ll end up with NumPy arrays and then tensors for modeling. Zarr can help with I/O and caching, but it adds some extra complexity and a learning curve. What I usually do—when space permits—is convert everything to .npy files to speed up I/O, compromising a bit on storage space. But honestly, who cares if you’ve got a good NVMe SSD anyway? The GPU VRAM though, is another story, which is more challenging sometimes.</p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 3147064,
      "author_name": "Taha_Alshatiri",
      "author_url": "",
      "post_date": "2025-03-11T15:40:50.900000",
      "content": "<p>to how much extent  can we can we decrease the unpacked data size (by under sampling and other techniques) without hurting the performance , can we reach below 80 GB?</p>",
      "votes": 2,
      "replies": [
        {
          "id": 3147085,
          "author_name": "SSS",
          "author_url": "",
          "post_date": "2025-03-11T16:20:45.243000",
          "content": "<p>There are some:</p>\n<ol>\n<li>Maybe try Sparse sampling and take odd slices only.</li>\n<li>Maybe resize the images in the beginning to test the pipeline.</li>\n<li>Maybe crop the images and take only specific ROI.</li>\n<li>Maybe discard samples with more than 1 motors since in the test there is going to be only zero and one motors examples.</li>\n<li>FP16 training</li>\n</ol>\n<p>There are more maybe(s) but at the moment I have not started with the experimenting yet.</p>",
          "votes": 6,
          "replies": [
            {
              "id": 3147963,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-03-12T15:48:08.527000",
              "content": "<p>4 is guaranteed ?</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3147970,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-12T15:57:02.277000",
              "content": "<p>Hi Tom, yes, according to the data page</p>\n<blockquote>\n  <p>test/: Directory with 3 directories of dummy test tomograms; the rerun test dataset contains approximately 900 tomograms. <strong><em>The test data only contain tomograms with one or zero motors.</em></strong> </p>\n</blockquote>\n<p>Though we can ask the host, hi <a href=\"https://www.kaggle.com/braxtonowens\" target=\"_blank\">@braxtonowens</a>, <a href=\"https://www.kaggle.com/jacksonpond\" target=\"_blank\">@jacksonpond</a> can you confirm that all 900 test tomograms only contain one or zero motors?</p>",
              "votes": 4,
              "replies": []
            },
            {
              "id": 3147983,
              "author_name": "Andrei Zamfir",
              "author_url": "",
              "post_date": "2025-03-12T16:14:37.233000",
              "content": "<p>The competition metric also requires one row per tomography id, so you can make only a prediction for one motor location or no motor at all. A confirmation that they only contain tomograms with one or zero motors would be great though!</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 3148361,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-03-13T03:20:49.520000",
              "content": "<p>So we can just use a linear layer to tell whether the image contains motor and its coordinate. <code>[cls, z, y, x]</code>.<br>\nI think that's a good alternative approach.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3146994,
      "author_name": "smalltrain",
      "author_url": "",
      "post_date": "2025-03-11T14:39:23.303000",
      "content": "<p>Hi, I have a question about the height and width of .jpg files. <br>\nTake tomo_00e047 as an example. In train_labels.csv, Array shape (axis 1) = 959 (width of each slice), Array shape (axis 2) = 928 (height of each slice). However, the .jpg file has a height of 959 and a width of 928.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 3147039,
          "author_name": "SSS",
          "author_url": "",
          "post_date": "2025-03-11T15:18:56.067000",
          "content": "<p>Hi,</p>\n<p>The <code>train_labels.csv</code> file provides the <code>z, y, x</code> w.r.t axis-0, axis-1, axis2, which means (num_slices, height, width). Here is the data page snippet:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F3538235b52ff17bc2ee86628e02a47ce%2FMonosnap%20BYU%20-%20Locating%20Bacterial%20Flagellar%20Motors.png?generation=1741706319127052&amp;alt=media\" alt=\"\"></p>\n<p>Hope, that helps.</p>",
          "votes": 2,
          "replies": [
            {
              "id": 3147043,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-03-11T15:21:04.173000",
              "content": "<p>Array shape axis 1: y-axis \"width\" of each slice.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3147051,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-11T15:28:10.100000",
              "content": "<p>yikes, it broke everything in my head now, hidden parallel world, illuminati!</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3146634,
      "author_name": "yuanzhe zhou",
      "author_url": "",
      "post_date": "2025-03-11T04:44:16.230000",
      "content": "<p>Can a single 4090 win this competition?</p>",
      "votes": 2,
      "replies": [
        {
          "id": 3146698,
          "author_name": "",
          "author_url": "",
          "post_date": "2025-03-11T06:58:06.693000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3146810,
          "author_name": "SSS",
          "author_url": "",
          "post_date": "2025-03-11T09:22:16.197000",
          "content": "<p>:) Let’s assume it takes 24 hours for the training to run all examples split into 5 folds. Top contenders run more than 500 experiments up to 1500 experiments. So good question - probably not if you are up to the number of experiments and full dataset runs. Though if you experiment on the small curated subset and then run the whole dataset then one 4090 is doable. Some folks managed to win such competitions by only using kaggle notebooks. P.S. I am sure you knew the answer.</p>",
          "votes": 3,
          "replies": [
            {
              "id": 3146867,
              "author_name": "yuanzhe zhou",
              "author_url": "",
              "post_date": "2025-03-11T10:50:01.213000",
              "content": "<p>I guess. It is not enough. Will external dataset help in this competition?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3146881,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-11T11:01:32.657000",
              "content": "<p>Yes, but only if you run your 4090 in turbo mode while chanting 'ReLU' three times under a full moon. Also, don't forget to overclock your CPU by applying deep learning principles to your cooling system.</p>\n<p>though, honestly, idk</p>",
              "votes": 3,
              "replies": []
            }
          ]
        },
        {
          "id": 3146840,
          "author_name": "Tom",
          "author_url": "",
          "post_date": "2025-03-11T10:11:30.267000",
          "content": "<p><a href=\"https://www.kaggle.com/yuanzhezhou\" target=\"_blank\">@yuanzhezhou</a> I think fork notebook is enough</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3147071,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-03-11T15:53:07.367000",
              "content": "<p><a href=\"https://www.kaggle.com/tom99763\" target=\"_blank\">@tom99763</a> <a href=\"https://www.kaggle.com/andreizamfir\" target=\"_blank\">@andreizamfir</a> <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> I am completely relying on GPU quota provided by kaggle .Is there still a chance to score good in this competition using kaggle GPU quota only ?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3147080,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-11T16:03:19.980000",
              "content": "<p>Tough but possible (don't let it to discourage you). Smart data sampling, efficient training, and selective runs can help. Some have won using only Kaggle's free tier, but full dataset runs will be a challenge. Make every experiment count.</p>\n<p>p.s. traditionally, deep learning competitions were compute hungry, and this is one of them.</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 3147082,
              "author_name": "Andrei Zamfir",
              "author_url": "",
              "post_date": "2025-03-11T16:10:01.193000",
              "content": "<p>I am only working with Kaggle GPU. While it's harder, and for some competitions certainly impossible (don't think it's impossible here), still doable to get good results.</p>",
              "votes": 6,
              "replies": []
            },
            {
              "id": 3160525,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-03-26T22:03:33.243000",
              "content": "<p>Wow, thank you for sharing.</p>",
              "votes": 11,
              "replies": []
            }
          ]
        },
        {
          "id": 3147115,
          "author_name": "bagas.jwnt",
          "author_url": "",
          "post_date": "2025-03-11T16:54:30.510000",
          "content": "<p>Hi, do you guys know where to get cheap gpu (VMs)? Google Colab is very expensive</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3147126,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-03-11T17:14:54.697000",
              "content": "<p><a href=\"https://vast.ai/\" target=\"_blank\">https://vast.ai/</a><br>\n<a href=\"https://www.runpod.io/\" target=\"_blank\">https://www.runpod.io/</a></p>",
              "votes": 4,
              "replies": []
            }
          ]
        },
        {
          "id": 3150185,
          "author_name": "Tom",
          "author_url": "",
          "post_date": "2025-03-15T06:24:38.840000",
          "content": "<p><a href=\"https://www.kaggle.com/yuanzhezhou\" target=\"_blank\">@yuanzhezhou</a> 2 RTX4090 take 2 hours run for training 3DUnet. Submission on kaggle takes 90min.</p>",
          "votes": 2,
          "replies": [
            {
              "id": 3151234,
              "author_name": "guo dashuai",
              "author_url": "",
              "post_date": "2025-03-16T13:01:48.020000",
              "content": "<p>Is 24GB not enough for training?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3151759,
              "author_name": "yuanzhe zhou",
              "author_url": "",
              "post_date": "2025-03-17T04:22:12.047000",
              "content": "<p>It seems that the main problem is the shake up … Data size is not that problematic.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3151766,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-03-17T04:26:01.347000",
              "content": "<p>But I think object detection will outperform segmentation in this comp. The ground truth is so sparse. Post processing for segmentation model is really really hard</p>",
              "votes": 2,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3149409,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-03-14T07:04:56.013000",
      "content": "<p>Do we know the actual quantity of test images ? I men the hidden one</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3149412,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-03-14T07:09:36.133000",
          "content": "<blockquote>\n  <p>the rerun test dataset contains approximately 900 tomograms</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/data\" target=\"_blank\">https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/data</a></p>\n<blockquote>\n  <p>This leaderboard is calculated with approximately 30% of the test data. The final results will be based on the other 70%, so the final standings may be different.</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/leaderboard\" target=\"_blank\">https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/leaderboard</a></p>",
          "votes": 3,
          "replies": [
            {
              "id": 3151796,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-03-17T05:38:46.360000",
              "content": "<p>Thanks the give a fair idea on the submission strategy …</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3146410,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-03-10T20:32:05.357000",
      "content": "<p>Hi <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> Thanks for your helpful description. </p>\n<p>Lets take example of <strong>tomo_00e047</strong> consists</p>\n<p>Motor axis 0 = 169<br>\nMotor axis 1 = 546<br>\nMotor axis 2 = 603</p>\n<p>Array shape (axis 0) = 300<br>\nArray shape (axis 1) = 959<br>\nArray shape (axis 2) = 928</p>\n<p>This particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. <strong>Motor axis 0</strong> means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ? </p>\n<p>Also what is the meaning of voxel spacing ?</p>",
      "votes": 3,
      "replies": [
        {
          "id": 3146421,
          "author_name": "SSS",
          "author_url": "",
          "post_date": "2025-03-10T20:56:29.393000",
          "content": "<p>Hi, </p>\n<blockquote>\n  <p>This particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. Motor axis 0 means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ?</p>\n</blockquote>\n<p>This is correct</p>\n<blockquote>\n  <p>Also what is the meaning of voxel spacing ?</p>\n</blockquote>\n<p>Voxel spacing refers to the physical distance between adjacent voxels in a 3D volume (voxels are small cubes )<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2Fde793a087e31ce68a4f752cadcb1c0b4%2Fvox%20spacing.png?generation=1741640137562554&amp;alt=media\" alt=\"\"></p>\n<p><a href=\"https://www.researchgate.net/figure/The-image-shows-the-26-voxel-neighborhood-used-for-the-pseudo-3D-key-point-calculation_fig5_281337843\" target=\"_blank\">image source</a></p>",
          "votes": 8,
          "replies": [
            {
              "id": 3148610,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-03-13T10:39:45.960000",
              "content": "<p><a href=\"https://www.kaggle.com/andreizamfir\" target=\"_blank\">@andreizamfir</a> <a href=\"https://www.kaggle.com/sergiosaharovskiy\" target=\"_blank\">@sergiosaharovskiy</a> Again taking example of tomo_id = 'tomo_00e047 '. There is flagella motor present at slice 169. But if you watch slice number 165 to 175 you can still notice flagella motor. Is this because actual image is in 3D format and we have given 2D slices and instead of taking circle at 1000 angstrom it is actually a cube of 1000 angstrom ?</p>",
              "votes": 5,
              "replies": []
            },
            {
              "id": 3148626,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-13T10:57:19.667000",
              "content": "<p>This is a good question. Yes, the nature of the tomogram is 3d. The flagella motor is spatial object so it can span through multiple spices. So when you create your labels you might want to span it over multiple adjacent slices!</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3148813,
              "author_name": "Andrei Zamfir",
              "author_url": "",
              "post_date": "2025-03-13T14:50:49.597000",
              "content": "<p>Yes, the tomogram is a 3d reconstruction of 2d projections of cryo-electron microscopies, at different inclination angles. There's a good intuition of the process on <a href=\"https://cryoem101.org/chapter-4-et/\" target=\"_blank\">this</a> page with a slider that lets you play around with the tilt angle, and of course, a comprehensive explanation of the methodology and implications.</p>\n<p>As the motor is 3d, it will span through the depth dimension, which is provided through different neighbouring slices. As the labels are supposed to be the centroids of the structures, you get the coordinates of the slice in the Z dimension that belongs to the center, but that doesn't mean that you won't have the motor on other neighbouring slices as well. It's just as you move around the x or y axis next to the vicinity of the center, these pixels still depict the center. This raises interesting questions about the way we want to define our labels for training, which is a crucial part for any supervised learning problem.</p>\n<p>There's also a discussion <a href=\"https://www.kaggle.com/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/567067\" target=\"_blank\">topic</a> addressing this, started by <a href=\"https://www.kaggle.com/gunesevitan\" target=\"_blank\">@gunesevitan</a> where the dimensions of the motor are taken into consideration.</p>\n<p>The 1000 angstroms are not to be confused with the size of the motor, it's just the distance threshold defined in the competition metric for a particular prediction to be considered valid. 1e-9 meters = 1 nanometer = 10 angstroms.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3148840,
              "author_name": "Tom",
              "author_url": "",
              "post_date": "2025-03-13T15:13:04.563000",
              "content": "<p>So something like building gaussian balls might works well.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3148946,
              "author_name": "SSS",
              "author_url": "",
              "post_date": "2025-03-13T16:57:44.207000",
              "content": "<p>I might share the code for that, if you can be so generous?;)</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3146423,
          "author_name": "Andrei Zamfir",
          "author_url": "",
          "post_date": "2025-03-10T21:04:49.477000",
          "content": "<p>The motor will be present in slice 169 from 300, at 546 (out of 959), 603 (out of 928). Voxel spacing gives you the reference units of dimension, it’s expressed in angstroms (0.1 nm or 1e-10 meters). </p>\n<p>You basically read it as, for this particular tomography, there’s voxel_spacing angstroms between 2 pixels on the same axis; if you multiply the dimensions of the tomography with the voxel spacing, you get its size in angstroms.</p>\n<p>This is directly linked with the competition metric threshold of 1000 angstroms for a prediction to be considered valid.</p>",
          "votes": 5,
          "replies": []
        }
      ]
    },
    {
      "id": 3207747,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-05-23T07:30:56.703000",
      "content": "<p>Thanks for posting!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 3153157,
      "author_name": "Alpcan Cepik",
      "author_url": "",
      "post_date": "2025-03-18T13:13:20.113000",
      "content": "<p>Thank you for sharing</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3145500": "There are **648** unique subdirectories in the `train` folder, each corresponding to a specific tomogram. Each train tomogram subdirectory contains `2D` slices, which can be stacked to reconstruct a `3D` tomogram.  \n\nIn total, there are **269,194** `.jpg` slices across all train tomogram subdirectories. Some `3D` tomograms can be storage-intensive, reaching up to **1.7GB** in size. When unpacked, the total volume of all train examples amounts to approximately **236GB** (`uint8` format).  \n\n---\n\n## Explanation and Schema  \n\nThe dataset consists of standard **train** and **test** folders. The **test** folder contains three directories of dummy test tomograms, while the actual rerun test dataset includes approximately **900** tomograms. **Test data only contains tomograms with either one or zero motors.**  \n\nThe **train** folder contains **648** tomograms, each sliced into `.jpg` files. The `train_labels.csv` file provides the `z, y, x` coordinates of the target of interest. If no target is present in a tomogram, the coordinates are set to `-1, -1, -1`. Similarly, during submission, `-1, -1, -1` should be used when predicting `0` targets.  \n\nBelow is the descriptive data schema and a visualization of a tomogram:  \n---\n\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F156fe5b03a449127cefe3cfae561c6c9%2FUnderstanding%20comp%20data.png?generation=1741558489474098&alt=media)\n\n---\n### Have Fun!\n[Code](https://www.kaggle.com/code/sergiosaharovskiy/byu-2025-eda-viz-pseudo-leaks) - slowly working through.\n[Repo](https://www.kaggle.com/datasets/sergiosaharovskiy/2025-byu-locating-bacterial-motors-public-repo) - external scripts & data",
    "3174438": "Thanks for the detailed explanation! Really helps to understand how the dataset is structured. Rebuilding the 3D tomograms from slices and predicting the target location sounds challenging but also very interesting. Looking forward to working on it!",
    "3147938": "What about conversion to Zarr, this dataset is outrageously large?",
    "3147064": "to how much extent  can we can we decrease the unpacked data size (by under sampling and other techniques) without hurting the performance , can we reach below 80 GB?",
    "3146994": "Hi, I have a question about the height and width of .jpg files. \nTake tomo_00e047 as an example. In train_labels.csv, Array shape (axis 1) = 959 (width of each slice), Array shape (axis 2) = 928 (height of each slice). However, the .jpg file has a height of 959 and a width of 928.",
    "3146634": "Can a single 4090 win this competition?",
    "3149409": "Do we know the actual quantity of test images ? I men the hidden one\n",
    "3146410": "Hi @sergiosaharovskiy Thanks for your helpful description. \n\nLets take example of **tomo_00e047** consists\n\nMotor axis 0 = 169\nMotor axis 1 = 546\nMotor axis 2 = 603\n\nArray shape (axis 0) = 300\nArray shape (axis 1) = 959\nArray shape (axis 2) = 928\n\nThis particular interpretation means that tomo_00e047 consists 300 2D slices each of Y and X axis of 959 and 928. **Motor axis 0** means that I need to look slice number 169 from 300 and at (X.Y) of (928,959) the Bacterial Flagellar Motors is present ? \n\nAlso what is the meaning of voxel spacing ?\n\n\n\n",
    "3207747": "Thanks for posting!",
    "3153157": "Thank you for sharing"
  }
}