{ "id": 578461, "title": "Reduce the number of slices for inference acceleration", "url": "/competitions/byu-locating-bacterial-flagellar-motors-2025/discussion/578461", "author_name": "", "post_date": "2025-05-11T05:06:33.881186500Z", "votes": 15, "comment_count": 23, "views": 0, "content": "

Here's a simple little trick: If your model's inference time is too long, a straightforward approach is to reduce the number of slices exponentially. For example, if you have a sample of 300 images, you can traverse the files with a step of 2 or 3 before performing inference. Theoretically, this can cut the inference time by more than half. In terms of accuracy, missing a few images won't significantly impact the results, and the positioning will still remain accurate.

", "messages": [ { "id": "3199515", "postDate": "05/11/2025 05:06:33", "content": "

Here's a simple little trick: If your model's inference time is too long, a straightforward approach is to reduce the number of slices exponentially. For example, if you have a sample of 300 images, you can traverse the files with a step of 2 or 3 before performing inference. Theoretically, this can cut the inference time by more than half. In terms of accuracy, missing a few images won't significantly impact the results, and the positioning will still remain accurate.

", "rawMarkdown": "Here's a simple little trick: If your model's inference time is too long, a straightforward approach is to reduce the number of slices exponentially. For example, if you have a sample of 300 images, you can traverse the files with a step of 2 or 3 before performing inference. Theoretically, this can cut the inference time by more than half. In terms of accuracy, missing a few images won't significantly impact the results, and the positioning will still remain accurate.", "votes": null }, { "id": "3199599", "postDate": "05/11/2025 08:03:37", "content": "

Do you mean to predict only every second or third slice?

", "rawMarkdown": "Do you mean to predict only every second or third slice?", "votes": null }, { "id": "3199603", "postDate": "05/11/2025 08:08:52", "content": "

Yes, for example, each prediction z=2n+1

", "rawMarkdown": "Yes, for example, each prediction z=2n+1", "votes": null }, { "id": "3199613", "postDate": "05/11/2025 08:28:49", "content": "

@yyyy0201 How can you guarantee this also works in another 72% hidden test? Does anyone want to risk on scoring high LB by discarding slices?

", "rawMarkdown": "yyyy0201 How can you guarantee this also works in another 72% hidden test? Does anyone want to risk on scoring high LB by discarding slices?", "votes": null }, { "id": "3199621", "postDate": "05/11/2025 08:42:47", "content": "

Cannot guarantee, it's only observed to have minimal impact on the current public test set now. However, theoretically, such a large number of slices isn't necessary—half the quantity is already sufficient for precise localization and classification. The saved time can be used for other operations. Of course, those who aren't pressed for inference time don’t need to do this.

", "rawMarkdown": "Cannot guarantee, it's only observed to have minimal impact on the current public test set now. However, theoretically, such a large number of slices isn't necessary—half the quantity is already sufficient for precise localization and classification. The saved time can be used for other operations. Of course, those who aren't pressed for inference time don’t need to do this.", "votes": null }, { "id": "3199665", "postDate": "05/11/2025 10:01:58", "content": "

I have an idea, assume we have 3d model, what if tuning the model by constraint the correlations or overall predictions cross each slice equal to jump slice. Then we can use this model to inference test set by jumping slice.

\n

Or even adding time positional embedding and manipulate it.

", "rawMarkdown": "I have an idea, assume we have 3d model, what if tuning the model by constraint the correlations or overall predictions cross each slice equal to jump slice. Then we can use this model to inference test set by jumping slice. \n\nOr even adding time positional embedding and manipulate it.", "votes": null }, { "id": "3199715", "postDate": "05/11/2025 11:29:55", "content": "

nice trick, it should not decay the perfermance

", "rawMarkdown": "nice trick, it should not decay the perfermance", "votes": null }, { "id": "3199751", "postDate": "05/11/2025 12:35:04", "content": "

That's the way it is.

", "rawMarkdown": "That's the way it is.", "votes": null }, { "id": "3200213", "postDate": "05/12/2025 09:03:45", "content": "

for my rtdetr model, it reduce my lb by ~0.008 🥲

", "rawMarkdown": "for my rtdetr model, it reduce my lb by ~0.008 🥲", "votes": null }, { "id": "3200220", "postDate": "05/12/2025 09:15:10", "content": "

Different STEP values may result in different hyperparameters as well, so some additional adaptation is still required😭

", "rawMarkdown": "Different STEP values may result in different hyperparameters as well, so some additional adaptation is still required😭", "votes": null }, { "id": "3200384", "postDate": "05/12/2025 13:56:09", "content": "

Nice tip, @yyyy0201. I’ve tried it as well. Just to share: my single model score dropped from 0.819 to 0.818.
\nThe difference might be bigger on the private leaderboard, but I still think it’s worth it to reduce inference time and make room for more models in an ensemble submission

", "rawMarkdown": "Nice tip, @yyyy0201. I’ve tried it as well. Just to share: my single model score dropped from 0.819 to 0.818. \nThe difference might be bigger on the private leaderboard, but I still think it’s worth it to reduce inference time and make room for more models in an ensemble submission", "votes": null }, { "id": "3200387", "postDate": "05/12/2025 13:59:30", "content": "

Thanks for the feedback! Glad to hear it works—we've always been using a step size of 2.

", "rawMarkdown": "Thanks for the feedback! Glad to hear it works—we've always been using a step size of 2.", "votes": null }, { "id": "3200717", "postDate": "05/13/2025 00:26:37", "content": "

I initially tried this method and my score dropped significantly. I think my score dropped significantly because I lacked understanding of post-processing at the time. Scores should only drop slightly. But the inference time is theoretically cut in half(in step 2), so as @sersasj said, in the ensemble there is more selection.

\n

I will try again as well.
\n@yyy0201 @sersasj thanks for the report that inspires me to try again.

", "rawMarkdown": "I initially tried this method and my score dropped significantly. I think my score dropped significantly because I lacked understanding of post-processing at the time. Scores should only drop slightly. But the inference time is theoretically cut in half(in step 2), so as @sersasj said, in the ensemble there is more selection.\n\nI will try again as well.\n@yyy0201 @sersasj thanks for the report that inspires me to try again.", "votes": null }, { "id": "3200719", "postDate": "05/13/2025 00:30:48", "content": "

@yyyy0201
\nShare the result I did:
\n0.856(full slice)->0.846(half slice)
\n0.856(full slice)->0.856(cross slice loss constraint + half slice)
\nThis actually works.

", "rawMarkdown": "yyyy0201 \nShare the result I did: \n**0.856**(full slice)->**0.846**(half slice)\n**0.856**(full slice)->**0.856**(cross slice loss constraint + half slice)\nThis actually works.", "votes": null }, { "id": "3200724", "postDate": "05/13/2025 00:58:11", "content": "

you can improve it by your training data. (or loss)

", "rawMarkdown": "you can improve it by your training data. (or loss)", "votes": null }, { "id": "3200726", "postDate": "05/13/2025 00:59:39", "content": "

a better solution is ensemble/TTA using different slicing:
\nmodel1: slice 1,3,5, …
\nmodel2: slice 2,4,6, …

", "rawMarkdown": "a better solution is ensemble/TTA using different slicing:\nmodel1: slice 1,3,5, ...\nmodel2: slice 2,4,6, ...", "votes": null }, { "id": "3200765", "postDate": "05/13/2025 03:00:11", "content": "

Thanks for the feedback🫡

", "rawMarkdown": "Thanks for the feedback🫡", "votes": null }, { "id": "3200768", "postDate": "05/13/2025 03:03:19", "content": "

The biggest impact of dropping points after the change should be the post-processing hyperparameters, which can be optimized as well

", "rawMarkdown": "The biggest impact of dropping points after the change should be the post-processing hyperparameters, which can be optimized as well", "votes": null }, { "id": "3201093", "postDate": "05/13/2025 12:09:12", "content": "

Tried LB on an ensemble of multiple models
\n0.852(full slice) submit time 8.5H
\n0.850(half slice) submit time 4.5H
\nOnly post-processing parameters are changed
\nThis level of difference is acceptable considering the submit time.

\n

When I tried it in the past
\n0.727(full slice)
\n0.677(half slice)
\nPost-processing is the same.

", "rawMarkdown": "Tried LB on an ensemble of multiple models\n0.852(full slice) submit time 8.5H\n0.850(half slice) submit time 4.5H \nOnly post-processing parameters are changed \nThis level of difference is acceptable considering the submit time.\n\nWhen I tried it in the past\n0.727(full slice)\n0.677(half slice) \nPost-processing is the same.", "votes": null }, { "id": "3202198", "postDate": "05/15/2025 02:48:09", "content": "

@yyyy0201 The submission requirement for Kaggle's object detection/localization task is that the coordinates must be in the original image coordinate system? When predicting, I performed the same data processing steps on the prediction set as the training set so that my model can perform at its best. However, I found that after submitting the score, my score dropped significantly. Is this because I did not restore the image to its original state after using the model to predict the processed test set? What is the idea for solving this problem? Can anyone help me? I would be very grateful.

", "rawMarkdown": "yyyy0201 The submission requirement for Kaggle's object detection/localization task is that the coordinates must be in the original image coordinate system? When predicting, I performed the same data processing steps on the prediction set as the training set so that my model can perform at its best. However, I found that after submitting the score, my score dropped significantly. Is this because I did not restore the image to its original state after using the model to predict the processed test set? What is the idea for solving this problem? Can anyone help me? I would be very grateful.", "votes": null }, { "id": "3202211", "postDate": "05/15/2025 03:14:44", "content": "

You can treat some images from the training set as a \"test set,\" apply the same preprocessing and inference steps to them, and visualize the results to check if there are any significant issues.

", "rawMarkdown": "You can treat some images from the training set as a \"test set,\" apply the same preprocessing and inference steps to them, and visualize the results to check if there are any significant issues.", "votes": null }, { "id": "3202219", "postDate": "05/15/2025 03:45:40", "content": "

I understand your idea, thank you.

", "rawMarkdown": "I understand your idea, thank you.", "votes": null }, { "id": "3207060", "postDate": "05/22/2025 06:56:55", "content": "

What is the image size for your model?

", "rawMarkdown": "What is the image size for your model?", "votes": null }, { "id": "3207263", "postDate": "05/22/2025 13:54:42", "content": "

I use 640x640

", "rawMarkdown": "I use 640x640", "votes": null } ], "comments": [ { "id": 3199599, "author_name": "jankowalski2000", "author_url": "", "post_date": "05/11/2025 08:03:37", "content": "

Do you mean to predict only every second or third slice?

", "votes": null, "replies": [ { "id": 3199603, "author_name": "yyyy0201", "author_url": "", "post_date": "05/11/2025 08:08:52", "content": "

Yes, for example, each prediction z=2n+1

", "votes": null, "replies": [ { "id": 3207060, "author_name": "rustambazarbayev", "author_url": "", "post_date": "05/22/2025 06:56:55", "content": "

What is the image size for your model?

", "votes": null, "replies": [ { "id": 3207263, "author_name": "yyyy0201", "author_url": "", "post_date": "05/22/2025 13:54:42", "content": "

I use 640x640

", "votes": null, "replies": [] } ] } ] } ] }, { "id": 3199613, "author_name": "tom99763", "author_url": "", "post_date": "05/11/2025 08:28:49", "content": "

@yyyy0201 How can you guarantee this also works in another 72% hidden test? Does anyone want to risk on scoring high LB by discarding slices?

", "votes": null, "replies": [ { "id": 3199621, "author_name": "yyyy0201", "author_url": "", "post_date": "05/11/2025 08:42:47", "content": "

Cannot guarantee, it's only observed to have minimal impact on the current public test set now. However, theoretically, such a large number of slices isn't necessary—half the quantity is already sufficient for precise localization and classification. The saved time can be used for other operations. Of course, those who aren't pressed for inference time don’t need to do this.

", "votes": null, "replies": [ { "id": 3199665, "author_name": "tom99763", "author_url": "", "post_date": "05/11/2025 10:01:58", "content": "

I have an idea, assume we have 3d model, what if tuning the model by constraint the correlations or overall predictions cross each slice equal to jump slice. Then we can use this model to inference test set by jumping slice.

\n

Or even adding time positional embedding and manipulate it.

", "votes": null, "replies": [ { "id": 3199751, "author_name": "yyyy0201", "author_url": "", "post_date": "05/11/2025 12:35:04", "content": "

That's the way it is.

", "votes": null, "replies": [ { "id": 3200719, "author_name": "tom99763", "author_url": "", "post_date": "05/13/2025 00:30:48", "content": "

@yyyy0201
\nShare the result I did:
\n0.856(full slice)->0.846(half slice)
\n0.856(full slice)->0.856(cross slice loss constraint + half slice)
\nThis actually works.

", "votes": null, "replies": [ { "id": 3200765, "author_name": "yyyy0201", "author_url": "", "post_date": "05/13/2025 03:00:11", "content": "

Thanks for the feedback🫡

", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 3199715, "author_name": "yuanzhezhou", "author_url": "", "post_date": "05/11/2025 11:29:55", "content": "

nice trick, it should not decay the perfermance

", "votes": null, "replies": [] }, { "id": 3200213, "author_name": "i2nfinit3y", "author_url": "", "post_date": "05/12/2025 09:03:45", "content": "

for my rtdetr model, it reduce my lb by ~0.008 🥲

", "votes": null, "replies": [ { "id": 3200220, "author_name": "yyyy0201", "author_url": "", "post_date": "05/12/2025 09:15:10", "content": "

Different STEP values may result in different hyperparameters as well, so some additional adaptation is still required😭

", "votes": null, "replies": [] } ] }, { "id": 3200384, "author_name": "sersasj", "author_url": "", "post_date": "05/12/2025 13:56:09", "content": "

Nice tip, @yyyy0201. I’ve tried it as well. Just to share: my single model score dropped from 0.819 to 0.818.
\nThe difference might be bigger on the private leaderboard, but I still think it’s worth it to reduce inference time and make room for more models in an ensemble submission

", "votes": null, "replies": [ { "id": 3200387, "author_name": "yyyy0201", "author_url": "", "post_date": "05/12/2025 13:59:30", "content": "

Thanks for the feedback! Glad to hear it works—we've always been using a step size of 2.

", "votes": null, "replies": [] }, { "id": 3200717, "author_name": "minfuka", "author_url": "", "post_date": "05/13/2025 00:26:37", "content": "

I initially tried this method and my score dropped significantly. I think my score dropped significantly because I lacked understanding of post-processing at the time. Scores should only drop slightly. But the inference time is theoretically cut in half(in step 2), so as @sersasj said, in the ensemble there is more selection.

\n

I will try again as well.
\n@yyy0201 @sersasj thanks for the report that inspires me to try again.

", "votes": null, "replies": [ { "id": 3200724, "author_name": "hengck23", "author_url": "", "post_date": "05/13/2025 00:58:11", "content": "

you can improve it by your training data. (or loss)

", "votes": null, "replies": [] }, { "id": 3200768, "author_name": "yyyy0201", "author_url": "", "post_date": "05/13/2025 03:03:19", "content": "

The biggest impact of dropping points after the change should be the post-processing hyperparameters, which can be optimized as well

", "votes": null, "replies": [] } ] }, { "id": 3201093, "author_name": "minfuka", "author_url": "", "post_date": "05/13/2025 12:09:12", "content": "

Tried LB on an ensemble of multiple models
\n0.852(full slice) submit time 8.5H
\n0.850(half slice) submit time 4.5H
\nOnly post-processing parameters are changed
\nThis level of difference is acceptable considering the submit time.

\n

When I tried it in the past
\n0.727(full slice)
\n0.677(half slice)
\nPost-processing is the same.

", "votes": null, "replies": [] } ] }, { "id": 3200726, "author_name": "hengck23", "author_url": "", "post_date": "05/13/2025 00:59:39", "content": "

a better solution is ensemble/TTA using different slicing:
\nmodel1: slice 1,3,5, …
\nmodel2: slice 2,4,6, …

", "votes": null, "replies": [] }, { "id": 3202198, "author_name": "uicyuhengye", "author_url": "", "post_date": "05/15/2025 02:48:09", "content": "

@yyyy0201 The submission requirement for Kaggle's object detection/localization task is that the coordinates must be in the original image coordinate system? When predicting, I performed the same data processing steps on the prediction set as the training set so that my model can perform at its best. However, I found that after submitting the score, my score dropped significantly. Is this because I did not restore the image to its original state after using the model to predict the processed test set? What is the idea for solving this problem? Can anyone help me? I would be very grateful.

", "votes": null, "replies": [ { "id": 3202211, "author_name": "yyyy0201", "author_url": "", "post_date": "05/15/2025 03:14:44", "content": "

You can treat some images from the training set as a \"test set,\" apply the same preprocessing and inference steps to them, and visualize the results to check if there are any significant issues.

", "votes": null, "replies": [ { "id": 3202219, "author_name": "uicyuhengye", "author_url": "", "post_date": "05/15/2025 03:45:40", "content": "

I understand your idea, thank you.

", "votes": null, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3199515": "Here's a simple little trick: If your model's inference time is too long, a straightforward approach is to reduce the number of slices exponentially. For example, if you have a sample of 300 images, you can traverse the files with a step of 2 or 3 before performing inference. Theoretically, this can cut the inference time by more than half. In terms of accuracy, missing a few images won't significantly impact the results, and the positioning will still remain accurate.", "3199599": "Do you mean to predict only every second or third slice?", "3199603": "Yes, for example, each prediction z=2n+1", "3199613": "yyyy0201 How can you guarantee this also works in another 72% hidden test? Does anyone want to risk on scoring high LB by discarding slices?", "3199621": "Cannot guarantee, it's only observed to have minimal impact on the current public test set now. However, theoretically, such a large number of slices isn't necessary—half the quantity is already sufficient for precise localization and classification. The saved time can be used for other operations. Of course, those who aren't pressed for inference time don’t need to do this.", "3199665": "I have an idea, assume we have 3d model, what if tuning the model by constraint the correlations or overall predictions cross each slice equal to jump slice. Then we can use this model to inference test set by jumping slice. \n\nOr even adding time positional embedding and manipulate it.", "3199715": "nice trick, it should not decay the perfermance", "3199751": "That's the way it is.", "3200213": "for my rtdetr model, it reduce my lb by ~0.008 🥲", "3200220": "Different STEP values may result in different hyperparameters as well, so some additional adaptation is still required😭", "3200384": "Nice tip, @yyyy0201. I’ve tried it as well. Just to share: my single model score dropped from 0.819 to 0.818. \nThe difference might be bigger on the private leaderboard, but I still think it’s worth it to reduce inference time and make room for more models in an ensemble submission", "3200387": "Thanks for the feedback! Glad to hear it works—we've always been using a step size of 2.", "3200717": "I initially tried this method and my score dropped significantly. I think my score dropped significantly because I lacked understanding of post-processing at the time. Scores should only drop slightly. But the inference time is theoretically cut in half(in step 2), so as @sersasj said, in the ensemble there is more selection.\n\nI will try again as well.\n@yyy0201 @sersasj thanks for the report that inspires me to try again.", "3200719": "yyyy0201 \nShare the result I did: \n**0.856**(full slice)->**0.846**(half slice)\n**0.856**(full slice)->**0.856**(cross slice loss constraint + half slice)\nThis actually works.", "3200724": "you can improve it by your training data. (or loss)", "3200726": "a better solution is ensemble/TTA using different slicing:\nmodel1: slice 1,3,5, ...\nmodel2: slice 2,4,6, ...", "3200765": "Thanks for the feedback🫡", "3200768": "The biggest impact of dropping points after the change should be the post-processing hyperparameters, which can be optimized as well", "3201093": "Tried LB on an ensemble of multiple models\n0.852(full slice) submit time 8.5H\n0.850(half slice) submit time 4.5H \nOnly post-processing parameters are changed \nThis level of difference is acceptable considering the submit time.\n\nWhen I tried it in the past\n0.727(full slice)\n0.677(half slice) \nPost-processing is the same.", "3202198": "yyyy0201 The submission requirement for Kaggle's object detection/localization task is that the coordinates must be in the original image coordinate system? When predicting, I performed the same data processing steps on the prediction set as the training set so that my model can perform at its best. However, I found that after submitting the score, my score dropped significantly. Is this because I did not restore the image to its original state after using the model to predict the processed test set? What is the idea for solving this problem? Can anyone help me? I would be very grateful.", "3202211": "You can treat some images from the training set as a \"test set,\" apply the same preprocessing and inference steps to them, and visualize the results to check if there are any significant issues.", "3202219": "I understand your idea, thank you.", "3207060": "What is the image size for your model?", "3207263": "I use 640x640" }, "source": "meta" }