![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4310004%2F6627841c58b70434ff5c698f41fffe0a%2Fgraphs.png?generation=1749125582409642&alt=media\")
Although I'm actually solo (with @daphne4sg ) in this competition, I want to thank all my teammates @goodcoder, @bigochampion and @cybersimar08 who help me in another LLM competition and share good things about personal life. I learn a lot from you guys. I also want to thank @andrewjdarley for hosting such amazing competition. This project gives a lot of helps for my job interview. I have high willingness to do further development because your goal is very interesting.
\nThis solution originates from the best project of my career, which I completed during my master’s degree a year ago. I collaborated with a director at Micron Technology through an industrial partnership. The project was extremely challenging—we were required to build a high-performance anomaly detector using only 88 HBM scanned images due to data privacy constraints.
\nAdditionally, the labels were a mix of AOI-judged results and human expert annotations. Our objective was to model the preferences of the human expert. Some parts of my BYU solution were adapted from this HBM project, such as keypoint generation and label inference from graph structures.
\nHowever, the most innovative aspect of my approach was leveraging few-shot prediction with a vision-language model (VLM) instead of traditional object detectors like YOLO. Essentially, I used prompts such as “this HBM is anomalous” or “HBM is OK” to represent the presence or absence of a target, allowing the model to make an initial classification guess.
\nMy solution calibrates the text-image embedding space using graph-based methods to enable the VLM to perform effectively for our specific task. As a result, I was able to train the model with only about 20 images, and it successfully detected nearly all anomalies identified by human experts in the HBM dataset.
\nThe initial idea can refer to the notebook where I made it public earlier. I want to produce a lot of points then refining them by building their relationship with labels. My strategy is sampling around the bacteria since motor existed in the head of it. If I aggregate all the keypoints and summarize them, the correct semantic might can be captured and get away from noisy labels. However, the biggest issue is how to represent the feature of each point, so I only use patch feature extracted from a simple CNN then building GNN to predict the location of motor. It turns out a really bad model.
\nIt's funny that I almost leaked my solution in this post two month ago. I had short discussion with @hengck23 about this development. After this discussion, I start to implement it and the idea becomes the pipeline showing below. So basically I use the C3K2 feature map of YOLO11 because the feature in each spatial position can represent refined object. Setting small confidence threshold (conf=0.05) can filter enough object locations determined from YOLO11. This design boosts the score, improving basic approach a lot.
\nFeature makes very big difference for motor prediction. I've tried almost every types of features like HoG, Daisy, EfficientNet, ResNet and different YOLO versions, all of them do not work well except YOLO11, but the score is still not good enough. After several days, I found AddRandomWalkPE on the PYG document. I suddenly realize this is the missing ingredient in 3d task. The score instantly boosts after I add it. So the node feature is C3K2 + RanadomWalkPE with walk_length=8. I think the main reason it improves the score is because it can represent the position of each point, implicitly modeling the position of motor.
YOLO successfully generates a lot of points by setting low confidence threshold, but it is still sparse in 3d space and has high possibility missing object existence. So my strategy for solving this issue is using the predicted point as center then uniformly sampling other points within a radius. I set radius and num_samples to control the range and density, respectively. I found using this strategy for training GNN also boosts score.
Since annotating a single point as motor can not completely represent the actual semantic (that's why host gives so much radius tolerance in the metric), I label a point as positive by the following conditions:
\nthr_sim.thr of ground truth location. ():\n label = train_labels[train_labels.tomo_id == tomo_id]\n (, label[[, , ]].values[])\n n_motors = label[].item()\n n_motors == :\n np.zeros(points.shape[], )\n d, h, w = label[[, , ]].values[]\n loc = label[[, , ]].values[]\n tz, ty, tx = loc.astype()\n fd, fh, fw = feat.shape[], feat.shape[], feat.shape[]\n z = tz.astype()\n y = ((ty/h) * fh).astype()\n x = ((tx/w) * fw).astype()\n \n extract_feat = extract_feat / np.linalg.norm(extract_feat, axis=-, keepdims=)\n target_feat = feat[z, :, y, x][, :]\n target_feat = target_feat / np.linalg.norm(target_feat, axis=-, keepdims=)\n sim = extract_feat @ target_feat.T\n bool1 = sim[:, ]>thr_sim\n\n \n dist = np.linalg.norm(points - loc[, :], axis=-) \n bool2 = dist<thr\n\n \n label = np.array(bool1 & bool2, dtype=)\n label\nI experimented with various graph structures and ultimately found that combining a k-NN graph with a Delaunay graph yields the best results. The k-NN graph provides high density, while the Delaunay graph establishes connections between distant clusters. By leveraging both properties, the combined approach results in a more generalized model. The experiment results below clearly explain why someone only take 16 submission attemps and go to sleep for a while.
\n| \n | Public Score | \nPrivate Score | \nCV | \n
|---|---|---|---|
| Knn Graph | \n0.835 | \n0.835 | \n0.968 | \n
| Radius Graph | \n0.826 | \n0.830 | \n0.966 | \n
| Delaunay Graph | \n0.801 | \n0.806 | \n0.962 | \n
| Knn + Radius | \n0.841 | \n0.835 | \n0.966 | \n
| Knn + Delaunay | \n0.856 | \n0.843 | \n0.966 | \n
First training a YOLO on the competition dataset. I don't use external dataset because I think the annotator is not the same person so it may gives differenet judgement for the motor existence. Then training the two GraphSages for each fold with Binary Cross Entropy with positive_weight=8. The ensemble function is maximum in my selected submission, but HDBSCAN is better in some scenarios. The training data for GNN is collected from the points filtered by YOLO which contains both positive and negative data. No extra dataset being used here.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4310004%2F82efe55faaa132866f20dbda41f235e7%2F123.png?generation=1749081901431650&alt=media\")
Unfortunately I'm trapped from my experiments and cannot make the right choice in the final minutes of deadline. YOLO11L predicts a lot of false positives which is good in this competition. But I didin't select it. This means I still have to improve my mindset and experience as a data scientist.
\n| \n | Public Score | \nPrivate Score | \nCV | \nSub Time | \n
|---|---|---|---|---|
| YOLO11L | \n0.856 | \n0.843 | \n0.968 | \n7HR | \n
| YOLO11X | \n0.865 | \n0.831 | \n0.983 | \n11HR | \n
yolo11l inference notebook (0.843 pb, best private score)
\nyolo11x interence notebook (0.831 pb, final submission)
\nGithub training code
My pipeline is quite complicated, verifying those methods take me a lot of engineering effort. Here is all the experiment records in the past three months: Google Sheet
\nIf you have any question please comment below or use email & linkin to contact me!
\nYou're welcome to share my solution on your blog—it's a great honor for me! I hope others will find it helpful and refer to it in future competitions.
Although I'm actually solo (with @daphne4sg ) in this competition, I want to thank all my teammates @goodcoder, @bigochampion and @cybersimar08 who help me in another LLM competition and share good things about personal life. I learn a lot from you guys. I also want to thank @andrewjdarley for hosting such amazing competition. This project gives a lot of helps for my job interview. I have high willingness to do further development because your goal is very interesting.
\nThis solution originates from the best project of my career, which I completed during my master’s degree a year ago. I collaborated with a director at Micron Technology through an industrial partnership. The project was extremely challenging—we were required to build a high-performance anomaly detector using only 88 HBM scanned images due to data privacy constraints.
\nAdditionally, the labels were a mix of AOI-judged results and human expert annotations. Our objective was to model the preferences of the human expert. Some parts of my BYU solution were adapted from this HBM project, such as keypoint generation and label inference from graph structures.
\nHowever, the most innovative aspect of my approach was leveraging few-shot prediction with a vision-language model (VLM) instead of traditional object detectors like YOLO. Essentially, I used prompts such as “this HBM is anomalous” or “HBM is OK” to represent the presence or absence of a target, allowing the model to make an initial classification guess.
\nMy solution calibrates the text-image embedding space using graph-based methods to enable the VLM to perform effectively for our specific task. As a result, I was able to train the model with only about 20 images, and it successfully detected nearly all anomalies identified by human experts in the HBM dataset.
\nThe initial idea can refer to the notebook where I made it public earlier. I want to produce a lot of points then refining them by building their relationship with labels. My strategy is sampling around the bacteria since motor existed in the head of it. If I aggregate all the keypoints and summarize them, the correct semantic might can be captured and get away from noisy labels. However, the biggest issue is how to represent the feature of each point, so I only use patch feature extracted from a simple CNN then building GNN to predict the location of motor. It turns out a really bad model.
\nIt's funny that I almost leaked my solution in this post two month ago. I had short discussion with @hengck23 about this development. After this discussion, I start to implement it and the idea becomes the pipeline showing below. So basically I use the C3K2 feature map of YOLO11 because the feature in each spatial position can represent refined object. Setting small confidence threshold (conf=0.05) can filter enough object locations determined from YOLO11. This design boosts the score, improving basic approach a lot.
\nFeature makes very big difference for motor prediction. I've tried almost every types of features like HoG, Daisy, EfficientNet, ResNet and different YOLO versions, all of them do not work well except YOLO11, but the score is still not good enough. After several days, I found AddRandomWalkPE on the PYG document. I suddenly realize this is the missing ingredient in 3d task. The score instantly boosts after I add it. So the node feature is C3K2 + RanadomWalkPE with walk_length=8. I think the main reason it improves the score is because it can represent the position of each point, implicitly modeling the position of motor.
YOLO successfully generates a lot of points by setting low confidence threshold, but it is still sparse in 3d space and has high possibility missing object existence. So my strategy for solving this issue is using the predicted point as center then uniformly sampling other points within a radius. I set radius and num_samples to control the range and density, respectively. I found using this strategy for training GNN also boosts score.
Since annotating a single point as motor can not completely represent the actual semantic (that's why host gives so much radius tolerance in the metric), I label a point as positive by the following conditions:
\nthr_sim.thr of ground truth location. ():\n label = train_labels[train_labels.tomo_id == tomo_id]\n (, label[[, , ]].values[])\n n_motors = label[].item()\n n_motors == :\n np.zeros(points.shape[], )\n d, h, w = label[[, , ]].values[]\n loc = label[[, , ]].values[]\n tz, ty, tx = loc.astype()\n fd, fh, fw = feat.shape[], feat.shape[], feat.shape[]\n z = tz.astype()\n y = ((ty/h) * fh).astype()\n x = ((tx/w) * fw).astype()\n \n extract_feat = extract_feat / np.linalg.norm(extract_feat, axis=-, keepdims=)\n target_feat = feat[z, :, y, x][, :]\n target_feat = target_feat / np.linalg.norm(target_feat, axis=-, keepdims=)\n sim = extract_feat @ target_feat.T\n bool1 = sim[:, ]>thr_sim\n\n \n dist = np.linalg.norm(points - loc[, :], axis=-) \n bool2 = dist<thr\n\n \n label = np.array(bool1 & bool2, dtype=)\n label\nI experimented with various graph structures and ultimately found that combining a k-NN graph with a Delaunay graph yields the best results. The k-NN graph provides high density, while the Delaunay graph establishes connections between distant clusters. By leveraging both properties, the combined approach results in a more generalized model. The experiment results below clearly explain why someone only take 16 submission attemps and go to sleep for a while.
\n| \n | Public Score | \nPrivate Score | \nCV | \n
|---|---|---|---|
| Knn Graph | \n0.835 | \n0.835 | \n0.968 | \n
| Radius Graph | \n0.826 | \n0.830 | \n0.966 | \n
| Delaunay Graph | \n0.801 | \n0.806 | \n0.962 | \n
| Knn + Radius | \n0.841 | \n0.835 | \n0.966 | \n
| Knn + Delaunay | \n0.856 | \n0.843 | \n0.966 | \n
First training a YOLO on the competition dataset. I don't use external dataset because I think the annotator is not the same person so it may gives differenet judgement for the motor existence. Then training the two GraphSages for each fold with Binary Cross Entropy with positive_weight=8. The ensemble function is maximum in my selected submission, but HDBSCAN is better in some scenarios. The training data for GNN is collected from the points filtered by YOLO which contains both positive and negative data. No extra dataset being used here.
Unfortunately I'm trapped from my experiments and cannot make the right choice in the final minutes of deadline. YOLO11L predicts a lot of false positives which is good in this competition. But I didin't select it. This means I still have to improve my mindset and experience as a data scientist.
\n| \n | Public Score | \nPrivate Score | \nCV | \nSub Time | \n
|---|---|---|---|---|
| YOLO11L | \n0.856 | \n0.843 | \n0.968 | \n7HR | \n
| YOLO11X | \n0.865 | \n0.831 | \n0.983 | \n11HR | \n
yolo11l inference notebook (0.843 pb, best private score)
\nyolo11x interence notebook (0.831 pb, final submission)
\nGithub training code
My pipeline is quite complicated, verifying those methods take me a lot of engineering effort. Here is all the experiment records in the past three months: Google Sheet
\nIf you have any question please comment below or use email & linkin to contact me!
\nYou're welcome to share my solution on your blog—it's a great honor for me! I hope others will find it helpful and refer to it in future competitions.
@mahdiseddigh Let's pray our big bird can show up.
", "rawMarkdown": "@mahdiseddigh Let's pray our big bird can show up.", "votes": 1 } ] }, { "id": 3217368, "postDate": "2025-06-05T00:31:25.433Z", "content": "Awesome looking solution. I didn't use a gnn setup but similarly used Yolo as a sort of region proposal system and then had a 3d classifier to reassess regions
", "rawMarkdown": "Awesome looking solution. I didn't use a gnn setup but similarly used Yolo as a sort of region proposal system and then had a 3d classifier to reassess regions", "votes": 1, "replies": [ { "id": 3217379, "postDate": "2025-06-05T00:56:00.700Z", "content": "@ryches I've tried region proposal method, but my experiments strongly show that sampling + graphs outperform it.
", "rawMarkdown": "@ryches I've tried region proposal method, but my experiments strongly show that sampling + graphs outperform it.", "votes": 1 } ] }, { "id": 3217846, "postDate": "2025-06-05T12:57:34.827Z", "content": "My first gold medal :(
\nNOOOOOOOOOOOOOOOOO
💀💀💀💀💀
", "rawMarkdown": "💀💀💀💀💀" } ] }, { "id": 3217519, "postDate": "2025-06-05T05:14:50.643Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3217672, "postDate": "2025-06-05T09:37:24.797Z", "content": "Thank you for your sharing
\nUpvoted
congrats, well designed approach with lot to learn from.
", "votes": 1, "replies": [ { "id": 3217699, "author_name": "Tom", "author_url": "", "post_date": "2025-06-05T10:20:09.353000", "content": "@mahdiseddigh Let's pray our big bird can show up.
", "votes": 1, "replies": [] } ] }, { "id": 3217368, "author_name": "ryches", "author_url": "", "post_date": "2025-06-05T00:31:25.433000", "content": "Awesome looking solution. I didn't use a gnn setup but similarly used Yolo as a sort of region proposal system and then had a 3d classifier to reassess regions
", "votes": 1, "replies": [ { "id": 3217379, "author_name": "Tom", "author_url": "", "post_date": "2025-06-05T00:56:00.700000", "content": "@ryches I've tried region proposal method, but my experiments strongly show that sampling + graphs outperform it.
", "votes": 1, "replies": [] } ] }, { "id": 3217846, "author_name": "daphne", "author_url": "", "post_date": "2025-06-05T12:57:34.827000", "content": "My first gold medal :(
\nNOOOOOOOOOOOOOOOOO
💀💀💀💀💀
", "votes": 0, "replies": [] } ] }, { "id": 3217519, "author_name": "", "author_url": "", "post_date": "2025-06-05T05:14:50.643000", "content": "", "votes": 0, "replies": [] }, { "id": 3217672, "author_name": "Yasir Hussein Shakir", "author_url": "", "post_date": "2025-06-05T09:37:24.797000", "content": "Thank you for your sharing
\nUpvoted