I'll sum up what I've tried and am going to try in this thread, please share your findings if you can.
\n\nWhat I've tried:
\n\nClassification with 5005 classes: not recommended, too easily overfit to class 0.
Removing class 0: Kinda worked, however a new problem arose which is how and where to insert the new_whale into your submission. Using a threshold worked ok so far, but it seems like it's just mimicking the ratio of new_whale on the public test set which is a bad thing.
Using bounding boxes increased the score by a 0.0x so there's no reason not to use it.
Ensembling helped, with 3-4 models it can give ~0.1 boost in score, using probabilities worked a bit better than ranking alone, but not by much. Gave diminished returns beyond 5 models in my experience.
Increasing image size helped, gave diminished returns beyond 384x384, not really sure about the number.
What I'll try:
\n\nBetter way to choose the new_whale threshold, see how far I can get with that approach.
Metrics learning, starting with @martinpiotte 's solution.
A different loss functions: Triplet loss, Magnet loss yadda yadda
Update 1
\n\nMartin's solution is ported here: https://www.kaggle.com/suicaokhoailang/martin-piotte-s-siamese-baseline . Very promising indeed.
\n\nUpdate 2
\n\nA few clues about high scoring solutions:
\n\nThe currently #1 solution used metrics learning: https://www.kaggle.com/c/humpback-whale-identification/discussion/74402#443160
My 0.898 submission was from a single siamese.
Classification is much better than I thought it would be, able to get 0.9+ LB: https://www.kaggle.com/c/humpback-whale-identification/discussion/75846#452220
Also, another approach on whether to include new_whale in your predictions using nearest neighbors: https://www.kaggle.com/iafoss/similarity-resnext50-0-602-lb
", "messages": [ { "id": 437446, "postDate": "2018-12-12T00:18:23.583Z", "content": "I'll sum up what I've tried and am going to try in this thread, please share your findings if you can.
\n\nWhat I've tried:
\n\nClassification with 5005 classes: not recommended, too easily overfit to class 0.
Removing class 0: Kinda worked, however a new problem arose which is how and where to insert the new_whale into your submission. Using a threshold worked ok so far, but it seems like it's just mimicking the ratio of new_whale on the public test set which is a bad thing.
Using bounding boxes increased the score by a 0.0x so there's no reason not to use it.
Ensembling helped, with 3-4 models it can give ~0.1 boost in score, using probabilities worked a bit better than ranking alone, but not by much. Gave diminished returns beyond 5 models in my experience.
Increasing image size helped, gave diminished returns beyond 384x384, not really sure about the number.
What I'll try:
\n\nBetter way to choose the new_whale threshold, see how far I can get with that approach.
Metrics learning, starting with @martinpiotte 's solution.
A different loss functions: Triplet loss, Magnet loss yadda yadda
Update 1
\n\nMartin's solution is ported here: https://www.kaggle.com/suicaokhoailang/martin-piotte-s-siamese-baseline . Very promising indeed.
\n\nUpdate 2
\n\nA few clues about high scoring solutions:
\n\nThe currently #1 solution used metrics learning: https://www.kaggle.com/c/humpback-whale-identification/discussion/74402#443160
My 0.898 submission was from a single siamese.
Classification is much better than I thought it would be, able to get 0.9+ LB: https://www.kaggle.com/c/humpback-whale-identification/discussion/75846#452220
Also, another approach on whether to include new_whale in your predictions using nearest neighbors: https://www.kaggle.com/iafoss/similarity-resnext50-0-602-lb
", "rawMarkdown": "I'll sum up what I've tried and am going to try in this thread, please share your findings if you can.\n\nWhat I've tried:\n\n- Classification with 5005 classes: not recommended, too easily overfit to class 0.\n\n- [Removing class 0][1]: Kinda worked, however a new problem arose which is how and where to insert the **new_whale** into your submission. Using a threshold worked ok so far, but it seems like it's just mimicking the ratio of **new_whale** on the public test set which is a bad thing.\n\n- [Using bounding boxes][2] increased the score by a 0.0x so there's no reason not to use it.\n\n- Ensembling helped, with 3-4 models it can give ~0.1 boost in score, [using probabilities][3] worked a bit better than [ranking][4] alone, but not by much. Gave diminished returns beyond 5 models in my experience.\n\n- Increasing image size helped, gave diminished returns beyond 384x384, not really sure about the number.\n\nWhat I'll try:\n\n- Better way to choose the **new_whale** threshold, see how far I can get with that approach.\n\n- Metrics learning, starting with @martinpiotte 's solution.\n\n- A different loss functions: Triplet loss, Magnet loss yadda yadda\n\n**Update 1**\n\nMartin's solution is ported here: https://www.kaggle.com/suicaokhoailang/martin-piotte-s-siamese-baseline . Very promising indeed.\n\n\n**Update 2**\n\nA few clues about high scoring solutions:\n\n- The currently #1 solution used metrics learning: https://www.kaggle.com/c/humpback-whale-identification/discussion/74402#443160\n\n- My 0.898 submission was from a single siamese. \n\n- Classification is much better than I thought it would be, able to get 0.9+ LB: https://www.kaggle.com/c/humpback-whale-identification/discussion/75846#452220\n\nAlso, another approach on whether to include **new_whale** in your predictions using nearest neighbors: https://www.kaggle.com/iafoss/similarity-resnext50-0-602-lb\n\n [1]: https://www.kaggle.com/suicaokhoailang/faster-and-better-resnet50-without-class-0-0-598\n [2]: https://www.kaggle.com/suicaokhoailang/resnet50-bounding-boxes-0-628-lb\n [3]: https://www.kaggle.com/suicaokhoailang/ensembling-with-averaged-probabilities-0-701-lb\n [4]: https://www.kaggle.com/matthewa313/ensembling-algorithm-for-average-precision-metric", "votes": 72 }, { "id": 453223, "postDate": "2019-01-09T21:45:38.893Z", "content": "I hope you guys do whale in the competition!
", "rawMarkdown": "I hope you guys do *whale* in the competition!", "votes": 12 }, { "id": 463464, "postDate": "2019-01-30T03:12:26.267Z", "content": "Update 3 : ）\nSome labelled data are shared by participants:\n- all training images' masks, labelled and predicted images are mixed ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78453 )\n- 1000 annotated fluke key points ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78699 )\n- 450 fluke masks ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78257 )
", "rawMarkdown": "Update 3 : ）\nSome labelled data are shared by participants:\n- all training images' masks, labelled and predicted images are mixed ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78453 )\n- 1000 annotated fluke key points ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78699 )\n- 450 fluke masks ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78257 )", "votes": 6, "replies": [ { "id": 464712, "postDate": "2019-02-01T10:43:22.860Z", "content": "Just to be sure, all these three links are annotations on the very same images of this competition, right? Or are these new images?
", "rawMarkdown": "Just to be sure, all these three links are annotations on the very same images of this competition, right? Or are these new images?" } ] }, { "id": 444476, "postDate": "2018-12-24T04:55:53.243Z", "content": "I am working with @martinpiotte 's solution. I've modified the lap to go with an imperfect answer instead of an exact answer. I ran 225 epochs at an hour per epoch with the initial implementation. Toady I changed the lap.lapjv to lapjv.lapjv and segmented the data into 6 segments running each in it's own thread. With these changes I am now running at 7-8 minutes per epoch.
", "rawMarkdown": "I am working with @martinpiotte 's solution. I've modified the lap to go with an imperfect answer instead of an exact answer. I ran 225 epochs at an hour per epoch with the initial implementation. Toady I changed the lap.lapjv to lapjv.lapjv and segmented the data into 6 segments running each in it's own thread. With these changes I am now running at 7-8 minutes per epoch.", "votes": 3, "replies": [ { "id": 444533, "postDate": "2018-12-24T07:49:50.463Z", "content": "Thanks for sharing Brian, I'm looking forward to seeing more of your findings in this competition. How is Atlas going?
", "rawMarkdown": "Thanks for sharing Brian, I'm looking forward to seeing more of your findings in this competition. How is Atlas going?" }, { "id": 444864, "postDate": "2018-12-25T02:31:49.020Z", "content": "I had to pause on things for a while, work took priority and I had no time left. Then I caught the flu. Finally getting back to things,I've got a model running now that hopefully I can get trained before things end.
\n\nAfter making the changes to the LAP yesterday things are looking good. In about 5 hours or so it will have completed the full 400 epochs in the original solution. Interestingly there is some gap in my local vs public LB. My .784 public score was .993 locally
", "rawMarkdown": "I had to pause on things for a while, work took priority and I had no time left. Then I caught the flu. Finally getting back to things,I've got a model running now that hopefully I can get trained before things end.\n\nAfter making the changes to the LAP yesterday things are looking good. In about 5 hours or so it will have completed the full 400 epochs in the original solution. Interestingly there is some gap in my local vs public LB. My .784 public score was .993 locally" }, { "id": 444951, "postDate": "2018-12-25T07:50:38.683Z", "content": "Funny, I caught a flu too, guess that's a global thing.
\n\nWhere did you get that lapjv package? I tried pip install lapjv but seems like its output format is different and not compatible with Martin's lap.
I never used to get sick, now with kids in school I get sick every year!
\n\nI am using 1.3.1 from:\nhttps://pypi.org/project/lapjv/
\n\nImporting as follows:
\n\nfrom lapjv import lapjv\n\n\n
In the generator here is how I do it multithreaded:
\n\nnum_threads = 6\n tmp = num_threads*[None]\n threads = []\n thread_input = num_threads*[None]\n thread_idx = 0\n batch = score.shape[0] // (num_threads-1)\n for start in range(0, score.shape[0], batch):\n end = min(score.shape[0], start + batch)\n thread_input[thread_idx] = self.score[start:end, start:end]\n thread_idx += 1\n\n def worker(data_idx):\n x,_,_ = lapjv(thread_input[data_idx]) \n tmp[data_idx] = x\n\n print(\"Start worker threads\")\n for i in range(num_threads):\n t = threading.Thread(target=worker, args=(i,), daemon=True)\n t.start()\n threads.append(t)\n for t in threads:\n if t is not None:\n t.join()\n x = np.concatenate(tmp)\n print(\"LAP completed\")\n\n\n
Basically lap.lapjv would return ,,x, and lapjv.lapjv returns the x first.
", "rawMarkdown": "I never used to get sick, now with kids in school I get sick every year!\n\nI am using 1.3.1 from:\nhttps://pypi.org/project/lapjv/\n\nImporting as follows:\nfrom lapjv import lapjv\n\n\nIn the generator here is how I do it multithreaded:\n
num_threads = 6\n tmp = num_threads*[None]\n threads = []\n thread_input = num_threads*[None]\n thread_idx = 0\n batch = score.shape[0] // (num_threads-1)\n for start in range(0, score.shape[0], batch):\n end = min(score.shape[0], start + batch)\n thread_input[thread_idx] = self.score[start:end, start:end]\n thread_idx += 1\n \n def worker(data_idx):\n x,_,_ = lapjv(thread_input[data_idx]) \n tmp[data_idx] = x\n\n print(\"Start worker threads\")\n for i in range(num_threads):\n t = threading.Thread(target=worker, args=(i,), daemon=True)\n t.start()\n threads.append(t)\n for t in threads:\n if t is not None:\n t.join()\n x = np.concatenate(tmp)\n print(\"LAP completed\")\n\n\nBasically lap.lapjv would return _,_,x, and lapjv.lapjv returns the x first.\n\n", "votes": 7 }, { "id": 445049, "postDate": "2018-12-25T13:18:53.013Z", "content": "
Thanks, I'll try it out. Btw what is your hardware specs? My 1080Ti also only needs 7-8 minutes per epoch with the original implementation.
", "rawMarkdown": "Thanks, I'll try it out. Btw what is your hardware specs? My 1080Ti also only needs 7-8 minutes per epoch with the original implementation." }, { "id": 445204, "postDate": "2018-12-26T00:01:33.323Z", "content": "Running this one on a RTX 2080, at the same time running HPA on a Titan X maxwell give epochs in the 7-10 minute range. If I am only running this one on the RTX it will get down to 6 minutes.
", "rawMarkdown": "Running this one on a RTX 2080, at the same time running HPA on a Titan X maxwell give epochs in the 7-10 minute range. If I am only running this one on the RTX it will get down to 6 minutes." }, { "id": 445842, "postDate": "2018-12-27T05:36:49.083Z", "content": "Noob question - what does lapjv do (I can see it's some sort of solver), and how should it be used with this dataset?
From my understanding, It is how the whales and non whales are matched up. It takes the two sets and returns the best matches. I am still trying to fully understand this myself. From my standpoint, I just found a faster way to do it than the original.
", "rawMarkdown": "From my understanding, It is how the whales and non whales are matched up. It takes the two sets and returns the best matches. I am still trying to fully understand this myself. From my standpoint, I just found a faster way to do it than the original.", "votes": 2 }, { "id": 447551, "postDate": "2018-12-30T03:50:20.063Z", "content": "@Brian Does your approach affect LB score compared to the original? I just switched to what you suggested and the training accuracy suddenly jumped up.
", "rawMarkdown": "@Brian Does your approach affect LB score compared to the original? I just switched to what you suggested and the training accuracy suddenly jumped up." }, { "id": 448047, "postDate": "2018-12-31T06:05:58.303Z", "content": "I switched midway through my training. I didn't notice much of a jump but I do have high local LB than public. 0.985 local resulted in 0.862 on the public.
", "rawMarkdown": "I switched midway through my training. I didn't notice much of a jump but I do have high local LB than public. 0.985 local resulted in 0.862 on the public." }, { "id": 448049, "postDate": "2018-12-31T06:15:29.307Z", "content": "The model is very unstable for me, jumping between 0.76 and 0.84 LB, yes local is always much higher than LB also.
", "rawMarkdown": "The model is very unstable for me, jumping between 0.76 and 0.84 LB, yes local is always much higher than LB also. ", "votes": 1 }, { "id": 451295, "postDate": "2019-01-06T19:36:12.840Z", "content": "The kaggle community is so sweet. Hope you get better, Brian.
", "rawMarkdown": "The kaggle community is so sweet. Hope you get better, Brian." }, { "id": 451535, "postDate": "2019-01-07T08:36:37.103Z", "content": "@Brain I am trying to understand 'lapjv'. I got a question here. The lap.lapjv() would return cost,x,y and lapjv() returns x,y,cost which means your code is using 'x' and Martin's code is using 'y'. Is there any difference between using x and y?
", "rawMarkdown": "@Brain I am trying to understand 'lapjv'. I got a question here. The lap.lapjv() would return cost,x,y and lapjv() returns x,y,cost which means your code is using 'x' and Martin's code is using 'y'. Is there any difference between using x and y?" }, { "id": 455039, "postDate": "2019-01-12T20:03:31.457Z", "content": "Thanks @XiokangWang. What a rough month December was. January meant back to work and I finally have some time to work on this again.
\n\n@philipgao1, That is a good catch, looks like I made a mistake. My intention was to use X there not Y. The input matrix X and Y are the same in most cases so I assume that is why it still works using Y. My background is in programming and I am using these competitions to learn the math for ML tasks.
\n\nEdit:\nLooking at the example here: https://github.com/src-d/lapjv
\n\nTo me that looks like it is returning y, x, cost as column is typically referred to as X. The documentation with lapjv.lapjv is pretty sparse.
", "rawMarkdown": "Thanks @XiokangWang. What a rough month December was. January meant back to work and I finally have some time to work on this again.\n\n@philipgao1, That is a good catch, looks like I made a mistake. My intention was to use X there not Y. The input matrix X and Y are the same in most cases so I assume that is why it still works using Y. My background is in programming and I am using these competitions to learn the math for ML tasks.\n\nEdit:\nLooking at the example here: https://github.com/src-d/lapjv\n\nTo me that looks like it is returning y, x, cost as column is typically referred to as X. The documentation with lapjv.lapjv is pretty sparse.", "votes": 2 }, { "id": 456787, "postDate": "2019-01-16T14:49:12.513Z", "content": "For me too its very inestable, could you share if you did it more stable and how?
", "rawMarkdown": "For me too its very inestable, could you share if you did it more stable and how?" }, { "id": 462076, "postDate": "2019-01-27T16:03:24.540Z", "content": "@brian is the multithread solution the same you would get with no multithreading? I mean from a threoretical point of view. I bet it's not, but I am not 100% sure...
", "rawMarkdown": "@brian is the multithread solution the same you would get with no multithreading? I mean from a threoretical point of view. I bet it's not, but I am not 100% sure...", "votes": 1 }, { "id": 463925, "postDate": "2019-01-30T22:53:56.240Z", "content": "It is not the same, the more threads/split the worse it is. The original matches the entire set against itself, with the threads it is matching only within the subset that each thread processes.
", "rawMarkdown": "It is not the same, the more threads/split the worse it is. The original matches the entire set against itself, with the threads it is matching only within the subset that each thread processes.", "votes": 2 }, { "id": 464060, "postDate": "2019-01-31T06:11:50.933Z", "content": "Thanks!
", "rawMarkdown": "Thanks!" }, { "id": 475782, "postDate": "2019-02-21T07:38:29.100Z", "content": "Hi Brian, regarding on multi-threading that speeds up the training cycle, the code above must be implemented in \"on_epoch_end\" function under TrainingData generator? Am I correct? Thanks again! :-)
", "rawMarkdown": "Hi Brian, regarding on multi-threading that speeds up the training cycle, the code above must be implemented in \"on_epoch_end\" function under TrainingData generator? Am I correct? Thanks again! :-)", "votes": 11 } ] }, { "id": 441738, "postDate": "2018-12-19T01:49:08.093Z", "content": "Thanks for sharing.\nI found that using grayscale(num_output_channels=3) can increase the score, but not sure if it is universal
", "rawMarkdown": "Thanks for sharing.\nI found that using grayscale(num_output_channels=3) can increase the score, but not sure if it is universal", "votes": 1, "replies": [ { "id": 441805, "postDate": "2018-12-19T05:09:15.427Z", "content": "Do you mean you converted RGB to \"grayscale RGB\"?
", "rawMarkdown": "Do you mean you converted RGB to \"grayscale RGB\"?" }, { "id": 442121, "postDate": "2018-12-19T14:05:21.563Z", "content": "yes, just convert image to grayscale(r = g = b)
", "rawMarkdown": "yes, just convert image to grayscale(r = g = b)" }, { "id": 442260, "postDate": "2018-12-19T17:37:43.353Z", "content": "Interesting, I may try that, thanks for sharing.
", "rawMarkdown": "Interesting, I may try that, thanks for sharing." }, { "id": 442383, "postDate": "2018-12-19T22:17:01.097Z", "content": "Probably more like this:\n0.2989 * R + 0.5870 * G + 0.1140 * B, or more like R+G+B / 3 since computers care more about dynamic range than perceptual lightness. You don't want to reduce the blue signal just because it looks better to a human.
", "rawMarkdown": "Probably more like this:\n0.2989 * R + 0.5870 * G + 0.1140 * B, or more like R+G+B / 3 since computers care more about dynamic range than perceptual lightness. You don't want to reduce the blue signal just because it looks better to a human." }, { "id": 443093, "postDate": "2018-12-21T02:16:30.810Z", "content": "I have tried grayscale(numoutputchannels=3) method, and the score decreased 0.116. So maybe it's not a universal method.
", "rawMarkdown": "I have tried grayscale(numoutputchannels=3) method, and the score decreased 0.116. So maybe it's not a universal method.", "votes": 9 }, { "id": 443100, "postDate": "2018-12-21T02:34:51.037Z", "content": "Thanks for sharing @luyang2018, may I ask which method are you pursuing? I guess to break the 0.9 it must be metrics learning.
", "rawMarkdown": "Thanks for sharing @luyang2018, may I ask which method are you pursuing? I guess to break the 0.9 it must be metrics learning.", "votes": 2 }, { "id": 443160, "postDate": "2018-12-21T05:36:57.383Z", "content": "Yes, metrics learning.
", "rawMarkdown": "Yes, metrics learning.", "votes": 5 }, { "id": 457368, "postDate": "2019-01-17T09:38:14.953Z", "content": "RGB-vs-grayscale is an interesting issue. For me, RGB alone or with random grayscale(numoutputchannels=3) doesn`t work. Looking forward to read @luyang2018 solution when competition finish :)
", "rawMarkdown": "RGB-vs-grayscale is an interesting issue. For me, RGB alone or with random grayscale(numoutputchannels=3) doesn`t work. Looking forward to read @luyang2018 solution when competition finish :)", "votes": 1 }, { "id": 467376, "postDate": "2019-02-07T01:30:33.507Z", "content": "Hey, if you don't mind. Could you share you answer on this point: Is Gray better than RGB in you solution?
", "rawMarkdown": "Hey, if you don't mind. Could you share you answer on this point: Is Gray better than RGB in you solution?", "votes": 1 } ] }, { "id": 437663, "postDate": "2018-12-12T09:38:47.267Z", "content": "Hi there, I am new to this competition (okay, yah, not that new, toke me a couple of days to get initial submission done). My hunch feeling is on how to exclude/ insert the new_whale data.\nHave anyone tried test/ identify the new whale data and then re-train with all dataset?
", "rawMarkdown": "Hi there, I am new to this competition (okay, yah, not that new, toke me a couple of days to get initial submission done). My hunch feeling is on how to exclude/ insert the new_whale data.\nHave anyone tried test/ identify the new whale data and then re-train with all dataset?", "votes": 1, "replies": [ { "id": 437829, "postDate": "2018-12-12T15:21:57.107Z", "content": "Errors would quickly introduce bias into the training set. Maybe only for high confidence thresholds?
", "rawMarkdown": "Errors would quickly introduce bias into the training set. Maybe only for high confidence thresholds?" } ] }, { "id": 437506, "postDate": "2018-12-12T03:47:57.320Z", "content": "Thanks for sharing.
\n\nOne more question: do you believe only classification model would perform better than 0.8? I notice that the 1st place at present got above 0.9, and I guess maybe we could try other methods(e.g. detection or others) to get there.
", "rawMarkdown": "Thanks for sharing.\n\nOne more question: do you believe only classification model would perform better than 0.8? I notice that the 1st place at present got above 0.9, and I guess maybe we could try other methods(e.g. detection or others) to get there.", "votes": 1, "replies": [ { "id": 437508, "postDate": "2018-12-12T03:53:34.307Z", "content": "With ensemble: absolutely, i'm not sure a single classification model can do that though.
", "rawMarkdown": "With ensemble: absolutely, i'm not sure a single classification model can do that though.", "votes": 1 }, { "id": 437580, "postDate": "2018-12-12T06:36:04.290Z", "content": "Thanks for your reply.
", "rawMarkdown": "Thanks for your reply." } ] }, { "id": 463713, "postDate": "2019-01-30T13:20:14.740Z", "content": "Hello, how are you get score above 0.9?
", "rawMarkdown": "Hello, how are you get score above 0.9?", "replies": [ { "id": 463763, "postDate": "2019-01-30T15:12:17.023Z", "content": "I dont know too.. I get LB 0.611 without new whales and with them 0.82.... Whats yous score without new whales, could you share it? I think choose the new whales its a important key here
", "rawMarkdown": "I dont know too.. I get LB 0.611 without new whales and with them 0.82.... Whats yous score without new whales, could you share it? I think choose the new whales its a important key here" }, { "id": 463775, "postDate": "2019-01-30T15:47:54.990Z", "content": "0.640 without new whales. Yes, i think so also.
", "rawMarkdown": "0.640 without new whales. Yes, i think so also." } ] }, { "id": 460568, "postDate": "2019-01-24T01:22:13.333Z", "content": "In the martinpiotte's solution, he compute a derangement of its pictures. What is the point of this operation?
", "rawMarkdown": "In the martinpiotte's solution, he compute a derangement of its pictures. What is the point of this operation?", "replies": [ { "id": 460980, "postDate": "2019-01-24T23:17:17.787Z", "content": "Do you mean the data augmentation by image transformation ? It prevents overfitting.
", "rawMarkdown": "Do you mean the data augmentation by image transformation ? It prevents overfitting." }, { "id": 462410, "postDate": "2019-01-28T09:03:10.357Z", "content": "Not really, it is when he talks about his training set creation:\n \"Half the examples used during training are for pair of images. For each whale of the training set, compute a derangement of its pictures. Use the original order as picture A, and the derangment as picture B. This creates a random number of matching image pairs, with each image taken exactly two times.\"\nWhy shuffle pixels from picture A give us a random number of matching image pairs, with each image taken exactly two times.
", "rawMarkdown": "Not really, it is when he talks about his training set creation:\n \"Half the examples used during training are for pair of images. For each whale of the training set, compute a derangement of its pictures. Use the original order as picture A, and the derangment as picture B. This creates a random number of matching image pairs, with each image taken exactly two times.\"\nWhy shuffle pixels from picture A give us a random number of matching image pairs, with each image taken exactly two times. ", "votes": 1 } ] }, { "id": 453575, "postDate": "2019-01-10T12:07:23.590Z", "content": "Hi :\nAre you using martinpiotte's solution as baseline and modifying it or justing implement your own code?
", "rawMarkdown": "Hi :\nAre you using martinpiotte's solution as baseline and modifying it or justing implement your own code?" }, { "id": 450750, "postDate": "2019-01-05T17:02:14.730Z", "content": "Do you think we should remove class 0 when we try metrics learning as well ? Metrics learnings don't tend to overfit compared with classification. I am investing the way to use new_whales effectively.
", "rawMarkdown": "Do you think we should remove class 0 when we try metrics learning as well ? Metrics learnings don't tend to overfit compared with classification. I am investing the way to use new_whales effectively." }, { "id": 446766, "postDate": "2018-12-28T16:57:03.350Z", "content": "Hi Khoi, am I correct that you have been using just the submission csvs for the ensembling?
\n\nI tried ensembling with the full model predictions and that seems to score considerably better (0.727 vs 0.71 for my not-so-finetuned ensemble)?
", "rawMarkdown": "Hi Khoi, am I correct that you have been using just the submission csvs for the ensembling? \n\nI tried ensembling with the full model predictions and that seems to score considerably better (0.727 vs 0.71 for my not-so-finetuned ensemble)?", "replies": [ { "id": 447085, "postDate": "2018-12-29T05:42:51.250Z", "content": "I used both.
\n\n\n", "rawMarkdown": "I used both.\n\n> Ensembling helped, with 3-4 models it can give ~0.1 boost in score, using probabilities worked a bit better than ranking alone, but not by much. Gave diminished returns beyond 5 models in my experience." }, { "id": 452991, "postDate": "2019-01-09T13:05:27.003Z", "content": "Ensembling helped, with 3-4 models it can give ~0.1 boost in score, using probabilities worked a bit better than ranking alone, but not by much. Gave diminished returns beyond 5 models in my experience.
\n
I saw that your LB score is 0.898 so... You have 4 models with around 0.800 score on LB and ensembling them using probabilities or just csv your score increased so much?
", "rawMarkdown": "I saw that your LB score is 0.898 so... You have 4 models with around 0.800 score on LB and ensembling them using probabilities or just csv your score increased so much?" }, { "id": 453001, "postDate": "2019-01-09T13:35:40.177Z", "content": "That score was from a single model, ensembling gives diminished returns as the scores go up so I dont think it will boost mine as much as when I was in the 0.7 range.
", "rawMarkdown": "That score was from a single model, ensembling gives diminished returns as the scores go up so I dont think it will boost mine as much as when I was in the 0.7 range." }, { "id": 453040, "postDate": "2019-01-09T15:13:13.163Z", "content": "Ah okey. I was able to get 0.83 with siamese, black and white and 384x384 images... Is this your approach or could you share something :)
", "rawMarkdown": "Ah okey. I was able to get 0.83 with siamese, black and white and 384x384 images... Is this your approach or could you share something :)" }, { "id": 455051, "postDate": "2019-01-12T20:25:12.193Z", "content": "I am getting 0.86 with black and white, 384x384. From the same model output, choosing where to put new_whale results in a range of scores from 0.73 to 0.86
", "rawMarkdown": "I am getting 0.86 with black and white, 384x384. From the same model output, choosing where to put new_whale results in a range of scores from 0.73 to 0.86", "votes": 4 }, { "id": 455263, "postDate": "2019-01-13T12:31:40.923Z", "content": "Thanks! Could you share your LB without putting on the result 'new_whales'
", "rawMarkdown": "Thanks! Could you share your LB without putting on the result 'new_whales'" }, { "id": 458948, "postDate": "2019-01-20T21:36:18.867Z", "content": "Once the model I'm training now is ready I'll post a result without new whale. I suspect it will be quite low.
", "rawMarkdown": "Once the model I'm training now is ready I'll post a result without new whale. I suspect it will be quite low.", "votes": 1 }, { "id": 460677, "postDate": "2019-01-24T08:24:21.833Z", "content": "And then? :D
", "rawMarkdown": "And then? :D" } ] }, { "id": 442276, "postDate": "2018-12-19T18:14:30.693Z", "content": "Are people holding out a portion of the training dataset for validation when training their models? or are people training the models on the entire training dataset and evaluating it's fit after submitted to the competition for scoring?
", "rawMarkdown": "Are people holding out a portion of the training dataset for validation when training their models? or are people training the models on the entire training dataset and evaluating it's fit after submitted to the competition for scoring?" }, { "id": 439590, "postDate": "2018-12-15T21:13:59.607Z", "content": "Excellent thread. I was wondering how you landed on 384x384 specifically?
", "rawMarkdown": "Excellent thread. I was wondering how you landed on 384x384 specifically?" }, { "id": 438701, "postDate": "2018-12-14T03:33:02.270Z", "content": "and how about the focal loss to solve the unbalance label？
", "rawMarkdown": "and how about the focal loss to solve the unbalance label？", "replies": [ { "id": 438782, "postDate": "2018-12-14T07:19:10.933Z", "content": "I'll try this later and see how it goes
", "rawMarkdown": "I'll try this later and see how it goes" }, { "id": 439284, "postDate": "2018-12-15T04:54:08.127Z", "content": "Apparently no changes in score..
", "rawMarkdown": "Apparently no changes in score..", "votes": 6 } ] }, { "id": 438670, "postDate": "2018-12-14T02:13:54.033Z", "content": "How are you dealing with image size? Are you \"Squashing\" everything so each image is a square, or are you maintaining each specific image's aspect ratio
", "rawMarkdown": "How are you dealing with image size? Are you \"Squashing\" everything so each image is a square, or are you maintaining each specific image's aspect ratio ", "replies": [ { "id": 439285, "postDate": "2018-12-15T04:54:31.143Z", "content": "Yes I'm squashing everything to a square.
", "rawMarkdown": "Yes I'm squashing everything to a square." }, { "id": 439831, "postDate": "2018-12-16T13:46:36.917Z", "content": "I keep the aspect ratio and pad images into square ones
", "rawMarkdown": "I keep the aspect ratio and pad images into square ones" }, { "id": 439861, "postDate": "2018-12-16T14:38:15.163Z", "content": "I am also new to the field of machine learning. But i thought about this too. I would also rather go with keeping the aspect ratio, also not produce new images by mirroring or so
", "rawMarkdown": "I am also new to the field of machine learning. But i thought about this too. I would also rather go with keeping the aspect ratio, also not produce new images by mirroring or so" } ] }, { "id": 440105, "postDate": "2018-12-17T03:38:56.787Z", "rawMarkdown": "", "isDeleted": true }, { "id": 439340, "postDate": "2018-12-15T07:42:57.527Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 453223, "author_name": "Michael Tam", "author_url": "", "post_date": "2019-01-09T21:45:38.893000", "content": "I hope you guys do whale in the competition!
", "votes": 12, "replies": [] }, { "id": 463464, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-01-30T03:12:26.267000", "content": "Update 3 : ）\nSome labelled data are shared by participants:\n- all training images' masks, labelled and predicted images are mixed ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78453 )\n- 1000 annotated fluke key points ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78699 )\n- 450 fluke masks ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78257 )
", "votes": 6, "replies": [ { "id": 464712, "author_name": "Eduardo Rocha de Andrade", "author_url": "", "post_date": "2019-02-01T10:43:22.860000", "content": "Just to be sure, all these three links are annotations on the very same images of this competition, right? Or are these new images?
", "votes": 0, "replies": [] } ] }, { "id": 444476, "author_name": "Brian", "author_url": "", "post_date": "2018-12-24T04:55:53.243000", "content": "I am working with @martinpiotte 's solution. I've modified the lap to go with an imperfect answer instead of an exact answer. I ran 225 epochs at an hour per epoch with the initial implementation. Toady I changed the lap.lapjv to lapjv.lapjv and segmented the data into 6 segments running each in it's own thread. With these changes I am now running at 7-8 minutes per epoch.
", "votes": 3, "replies": [ { "id": 444533, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-24T07:49:50.463000", "content": "Thanks for sharing Brian, I'm looking forward to seeing more of your findings in this competition. How is Atlas going?
", "votes": 0, "replies": [] }, { "id": 444864, "author_name": "Brian", "author_url": "", "post_date": "2018-12-25T02:31:49.020000", "content": "I had to pause on things for a while, work took priority and I had no time left. Then I caught the flu. Finally getting back to things,I've got a model running now that hopefully I can get trained before things end.
\n\nAfter making the changes to the LAP yesterday things are looking good. In about 5 hours or so it will have completed the full 400 epochs in the original solution. Interestingly there is some gap in my local vs public LB. My .784 public score was .993 locally
", "votes": 0, "replies": [] }, { "id": 444951, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-25T07:50:38.683000", "content": "Funny, I caught a flu too, guess that's a global thing.
\n\nWhere did you get that lapjv package? I tried pip install lapjv but seems like its output format is different and not compatible with Martin's lap.
I never used to get sick, now with kids in school I get sick every year!
\n\nI am using 1.3.1 from:\nhttps://pypi.org/project/lapjv/
\n\nImporting as follows:
\n\nfrom lapjv import lapjv\n\n\n
In the generator here is how I do it multithreaded:
\n\nnum_threads = 6\n tmp = num_threads*[None]\n threads = []\n thread_input = num_threads*[None]\n thread_idx = 0\n batch = score.shape[0] // (num_threads-1)\n for start in range(0, score.shape[0], batch):\n end = min(score.shape[0], start + batch)\n thread_input[thread_idx] = self.score[start:end, start:end]\n thread_idx += 1\n\n def worker(data_idx):\n x,_,_ = lapjv(thread_input[data_idx]) \n tmp[data_idx] = x\n\n print(\"Start worker threads\")\n for i in range(num_threads):\n t = threading.Thread(target=worker, args=(i,), daemon=True)\n t.start()\n threads.append(t)\n for t in threads:\n if t is not None:\n t.join()\n x = np.concatenate(tmp)\n print(\"LAP completed\")\n\n\n
Basically lap.lapjv would return ,,x, and lapjv.lapjv returns the x first.
", "votes": 7, "replies": [] }, { "id": 445049, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-25T13:18:53.013000", "content": "Thanks, I'll try it out. Btw what is your hardware specs? My 1080Ti also only needs 7-8 minutes per epoch with the original implementation.
", "votes": 0, "replies": [] }, { "id": 445204, "author_name": "Brian", "author_url": "", "post_date": "2018-12-26T00:01:33.323000", "content": "Running this one on a RTX 2080, at the same time running HPA on a Titan X maxwell give epochs in the 7-10 minute range. If I am only running this one on the RTX it will get down to 6 minutes.
", "votes": 0, "replies": [] }, { "id": 445842, "author_name": "datasaurus", "author_url": "", "post_date": "2018-12-27T05:36:49.083000", "content": "Noob question - what does lapjv do (I can see it's some sort of solver), and how should it be used with this dataset?
From my understanding, It is how the whales and non whales are matched up. It takes the two sets and returns the best matches. I am still trying to fully understand this myself. From my standpoint, I just found a faster way to do it than the original.
", "votes": 2, "replies": [] }, { "id": 447551, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-30T03:50:20.063000", "content": "@Brian Does your approach affect LB score compared to the original? I just switched to what you suggested and the training accuracy suddenly jumped up.
", "votes": 0, "replies": [] }, { "id": 448047, "author_name": "Brian", "author_url": "", "post_date": "2018-12-31T06:05:58.303000", "content": "I switched midway through my training. I didn't notice much of a jump but I do have high local LB than public. 0.985 local resulted in 0.862 on the public.
", "votes": 0, "replies": [] }, { "id": 448049, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-31T06:15:29.307000", "content": "The model is very unstable for me, jumping between 0.76 and 0.84 LB, yes local is always much higher than LB also.
", "votes": 1, "replies": [] }, { "id": 451295, "author_name": "XiaokangWang", "author_url": "", "post_date": "2019-01-06T19:36:12.840000", "content": "The kaggle community is so sweet. Hope you get better, Brian.
", "votes": 0, "replies": [] }, { "id": 451535, "author_name": "Philip Gao", "author_url": "", "post_date": "2019-01-07T08:36:37.103000", "content": "@Brain I am trying to understand 'lapjv'. I got a question here. The lap.lapjv() would return cost,x,y and lapjv() returns x,y,cost which means your code is using 'x' and Martin's code is using 'y'. Is there any difference between using x and y?
", "votes": 0, "replies": [] }, { "id": 455039, "author_name": "Brian", "author_url": "", "post_date": "2019-01-12T20:03:31.457000", "content": "Thanks @XiokangWang. What a rough month December was. January meant back to work and I finally have some time to work on this again.
\n\n@philipgao1, That is a good catch, looks like I made a mistake. My intention was to use X there not Y. The input matrix X and Y are the same in most cases so I assume that is why it still works using Y. My background is in programming and I am using these competitions to learn the math for ML tasks.
\n\nEdit:\nLooking at the example here: https://github.com/src-d/lapjv
\n\nTo me that looks like it is returning y, x, cost as column is typically referred to as X. The documentation with lapjv.lapjv is pretty sparse.
", "votes": 2, "replies": [] }, { "id": 456787, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-16T14:49:12.513000", "content": "For me too its very inestable, could you share if you did it more stable and how?
", "votes": 0, "replies": [] }, { "id": 462076, "author_name": "bluetrain", "author_url": "", "post_date": "2019-01-27T16:03:24.540000", "content": "@brian is the multithread solution the same you would get with no multithreading? I mean from a threoretical point of view. I bet it's not, but I am not 100% sure...
", "votes": 1, "replies": [] }, { "id": 463925, "author_name": "Brian", "author_url": "", "post_date": "2019-01-30T22:53:56.240000", "content": "It is not the same, the more threads/split the worse it is. The original matches the entire set against itself, with the threads it is matching only within the subset that each thread processes.
", "votes": 2, "replies": [] }, { "id": 464060, "author_name": "bluetrain", "author_url": "", "post_date": "2019-01-31T06:11:50.933000", "content": "Thanks!
", "votes": 0, "replies": [] }, { "id": 475782, "author_name": "", "author_url": "", "post_date": "2019-02-21T07:38:29.100000", "content": "Hi Brian, regarding on multi-threading that speeds up the training cycle, the code above must be implemented in \"on_epoch_end\" function under TrainingData generator? Am I correct? Thanks again! :-)
", "votes": 11, "replies": [] } ] }, { "id": 441738, "author_name": "HiYellowC", "author_url": "", "post_date": "2018-12-19T01:49:08.093000", "content": "Thanks for sharing.\nI found that using grayscale(num_output_channels=3) can increase the score, but not sure if it is universal
", "votes": 1, "replies": [ { "id": 441805, "author_name": "Simeon Trieu", "author_url": "", "post_date": "2018-12-19T05:09:15.427000", "content": "Do you mean you converted RGB to \"grayscale RGB\"?
", "votes": 0, "replies": [] }, { "id": 442121, "author_name": "HiYellowC", "author_url": "", "post_date": "2018-12-19T14:05:21.563000", "content": "yes, just convert image to grayscale(r = g = b)
", "votes": 0, "replies": [] }, { "id": 442260, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-19T17:37:43.353000", "content": "Interesting, I may try that, thanks for sharing.
", "votes": 0, "replies": [] }, { "id": 442383, "author_name": "Simeon Trieu", "author_url": "", "post_date": "2018-12-19T22:17:01.097000", "content": "Probably more like this:\n0.2989 * R + 0.5870 * G + 0.1140 * B, or more like R+G+B / 3 since computers care more about dynamic range than perceptual lightness. You don't want to reduce the blue signal just because it looks better to a human.
", "votes": 0, "replies": [] }, { "id": 443093, "author_name": "Lu Yang", "author_url": "", "post_date": "2018-12-21T02:16:30.810000", "content": "I have tried grayscale(numoutputchannels=3) method, and the score decreased 0.116. So maybe it's not a universal method.
", "votes": 9, "replies": [] }, { "id": 443100, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-21T02:34:51.037000", "content": "Thanks for sharing @luyang2018, may I ask which method are you pursuing? I guess to break the 0.9 it must be metrics learning.
", "votes": 2, "replies": [] }, { "id": 443160, "author_name": "Lu Yang", "author_url": "", "post_date": "2018-12-21T05:36:57.383000", "content": "Yes, metrics learning.
", "votes": 5, "replies": [] }, { "id": 457368, "author_name": "old-ufo", "author_url": "", "post_date": "2019-01-17T09:38:14.953000", "content": "RGB-vs-grayscale is an interesting issue. For me, RGB alone or with random grayscale(numoutputchannels=3) doesn`t work. Looking forward to read @luyang2018 solution when competition finish :)
", "votes": 1, "replies": [] }, { "id": 467376, "author_name": "Gary", "author_url": "", "post_date": "2019-02-07T01:30:33.507000", "content": "Hey, if you don't mind. Could you share you answer on this point: Is Gray better than RGB in you solution?
", "votes": 1, "replies": [] } ] }, { "id": 437663, "author_name": "wklin8", "author_url": "", "post_date": "2018-12-12T09:38:47.267000", "content": "Hi there, I am new to this competition (okay, yah, not that new, toke me a couple of days to get initial submission done). My hunch feeling is on how to exclude/ insert the new_whale data.\nHave anyone tried test/ identify the new whale data and then re-train with all dataset?
", "votes": 1, "replies": [ { "id": 437829, "author_name": "Simeon Trieu", "author_url": "", "post_date": "2018-12-12T15:21:57.107000", "content": "Errors would quickly introduce bias into the training set. Maybe only for high confidence thresholds?
", "votes": 0, "replies": [] } ] }, { "id": 437506, "author_name": "Tommy Jiang", "author_url": "", "post_date": "2018-12-12T03:47:57.320000", "content": "Thanks for sharing.
\n\nOne more question: do you believe only classification model would perform better than 0.8? I notice that the 1st place at present got above 0.9, and I guess maybe we could try other methods(e.g. detection or others) to get there.
", "votes": 1, "replies": [ { "id": 437508, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-12T03:53:34.307000", "content": "With ensemble: absolutely, i'm not sure a single classification model can do that though.
", "votes": 1, "replies": [] }, { "id": 437580, "author_name": "Tommy Jiang", "author_url": "", "post_date": "2018-12-12T06:36:04.290000", "content": "Thanks for your reply.
", "votes": 0, "replies": [] } ] }, { "id": 463713, "author_name": "NullPoint", "author_url": "", "post_date": "2019-01-30T13:20:14.740000", "content": "Hello, how are you get score above 0.9?
", "votes": 0, "replies": [ { "id": 463763, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-30T15:12:17.023000", "content": "I dont know too.. I get LB 0.611 without new whales and with them 0.82.... Whats yous score without new whales, could you share it? I think choose the new whales its a important key here
", "votes": 0, "replies": [] }, { "id": 463775, "author_name": "NullPoint", "author_url": "", "post_date": "2019-01-30T15:47:54.990000", "content": "0.640 without new whales. Yes, i think so also.
", "votes": 0, "replies": [] } ] }, { "id": 460568, "author_name": "Enamor", "author_url": "", "post_date": "2019-01-24T01:22:13.333000", "content": "In the martinpiotte's solution, he compute a derangement of its pictures. What is the point of this operation?
", "votes": 0, "replies": [ { "id": 460980, "author_name": "toshi_k", "author_url": "", "post_date": "2019-01-24T23:17:17.787000", "content": "Do you mean the data augmentation by image transformation ? It prevents overfitting.
", "votes": 0, "replies": [] }, { "id": 462410, "author_name": "Enamor", "author_url": "", "post_date": "2019-01-28T09:03:10.357000", "content": "Not really, it is when he talks about his training set creation:\n \"Half the examples used during training are for pair of images. For each whale of the training set, compute a derangement of its pictures. Use the original order as picture A, and the derangment as picture B. This creates a random number of matching image pairs, with each image taken exactly two times.\"\nWhy shuffle pixels from picture A give us a random number of matching image pairs, with each image taken exactly two times.
", "votes": 1, "replies": [] } ] }, { "id": 453575, "author_name": "TsungHan", "author_url": "", "post_date": "2019-01-10T12:07:23.590000", "content": "Hi :\nAre you using martinpiotte's solution as baseline and modifying it or justing implement your own code?
", "votes": 0, "replies": [] }, { "id": 450750, "author_name": "toshi_k", "author_url": "", "post_date": "2019-01-05T17:02:14.730000", "content": "Do you think we should remove class 0 when we try metrics learning as well ? Metrics learnings don't tend to overfit compared with classification. I am investing the way to use new_whales effectively.
", "votes": 0, "replies": [] }, { "id": 446766, "author_name": "PieterBlomme", "author_url": "", "post_date": "2018-12-28T16:57:03.350000", "content": "Hi Khoi, am I correct that you have been using just the submission csvs for the ensembling?
\n\nI tried ensembling with the full model predictions and that seems to score considerably better (0.727 vs 0.71 for my not-so-finetuned ensemble)?
", "votes": 0, "replies": [ { "id": 447085, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-29T05:42:51.250000", "content": "I used both.
\n\n\n", "votes": 0, "replies": [] }, { "id": 452991, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-09T13:05:27.003000", "content": "Ensembling helped, with 3-4 models it can give ~0.1 boost in score, using probabilities worked a bit better than ranking alone, but not by much. Gave diminished returns beyond 5 models in my experience.
\n
I saw that your LB score is 0.898 so... You have 4 models with around 0.800 score on LB and ensembling them using probabilities or just csv your score increased so much?
", "votes": 0, "replies": [] }, { "id": 453001, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2019-01-09T13:35:40.177000", "content": "That score was from a single model, ensembling gives diminished returns as the scores go up so I dont think it will boost mine as much as when I was in the 0.7 range.
", "votes": 0, "replies": [] }, { "id": 453040, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-09T15:13:13.163000", "content": "Ah okey. I was able to get 0.83 with siamese, black and white and 384x384 images... Is this your approach or could you share something :)
", "votes": 0, "replies": [] }, { "id": 455051, "author_name": "Brian", "author_url": "", "post_date": "2019-01-12T20:25:12.193000", "content": "I am getting 0.86 with black and white, 384x384. From the same model output, choosing where to put new_whale results in a range of scores from 0.73 to 0.86
", "votes": 4, "replies": [] }, { "id": 455263, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-13T12:31:40.923000", "content": "Thanks! Could you share your LB without putting on the result 'new_whales'
", "votes": 0, "replies": [] }, { "id": 458948, "author_name": "Brian", "author_url": "", "post_date": "2019-01-20T21:36:18.867000", "content": "Once the model I'm training now is ready I'll post a result without new whale. I suspect it will be quite low.
", "votes": 1, "replies": [] }, { "id": 460677, "author_name": "Mario Parreño Lara", "author_url": "", "post_date": "2019-01-24T08:24:21.833000", "content": "And then? :D
", "votes": 0, "replies": [] } ] }, { "id": 442276, "author_name": "Zak Raicik", "author_url": "", "post_date": "2018-12-19T18:14:30.693000", "content": "Are people holding out a portion of the training dataset for validation when training their models? or are people training the models on the entire training dataset and evaluating it's fit after submitted to the competition for scoring?
", "votes": 0, "replies": [] }, { "id": 439590, "author_name": "Matthew Anderson", "author_url": "", "post_date": "2018-12-15T21:13:59.607000", "content": "Excellent thread. I was wondering how you landed on 384x384 specifically?
", "votes": 0, "replies": [] }, { "id": 438701, "author_name": "ShawnTung", "author_url": "", "post_date": "2018-12-14T03:33:02.270000", "content": "and how about the focal loss to solve the unbalance label？
", "votes": 0, "replies": [ { "id": 438782, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-14T07:19:10.933000", "content": "I'll try this later and see how it goes
", "votes": 0, "replies": [] }, { "id": 439284, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-15T04:54:08.127000", "content": "Apparently no changes in score..
", "votes": 6, "replies": [] } ] }, { "id": 438670, "author_name": "Zak Raicik", "author_url": "", "post_date": "2018-12-14T02:13:54.033000", "content": "How are you dealing with image size? Are you \"Squashing\" everything so each image is a square, or are you maintaining each specific image's aspect ratio
", "votes": 0, "replies": [ { "id": 439285, "author_name": "Khoi Nguyen", "author_url": "", "post_date": "2018-12-15T04:54:31.143000", "content": "Yes I'm squashing everything to a square.
", "votes": 0, "replies": [] }, { "id": 439831, "author_name": "youchaoqin", "author_url": "", "post_date": "2018-12-16T13:46:36.917000", "content": "I keep the aspect ratio and pad images into square ones
", "votes": 0, "replies": [] }, { "id": 439861, "author_name": "Markus Degen", "author_url": "", "post_date": "2018-12-16T14:38:15.163000", "content": "I am also new to the field of machine learning. But i thought about this too. I would also rather go with keeping the aspect ratio, also not produce new images by mirroring or so
", "votes": 0, "replies": [] } ] }, { "id": 440105, "author_name": "", "author_url": "", "post_date": "2018-12-17T03:38:56.787000", "content": "", "votes": 0, "replies": [] }, { "id": 439340, "author_name": "", "author_url": "", "post_date": "2018-12-15T07:42:57.527000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "437446": "I'll sum up what I've tried and am going to try in this thread, please share your findings if you can.\n\nWhat I've tried:\n\n- Classification with 5005 classes: not recommended, too easily overfit to class 0.\n\n- [Removing class 0][1]: Kinda worked, however a new problem arose which is how and where to insert the **new_whale** into your submission. Using a threshold worked ok so far, but it seems like it's just mimicking the ratio of **new_whale** on the public test set which is a bad thing.\n\n- [Using bounding boxes][2] increased the score by a 0.0x so there's no reason not to use it.\n\n- Ensembling helped, with 3-4 models it can give ~0.1 boost in score, [using probabilities][3] worked a bit better than [ranking][4] alone, but not by much. Gave diminished returns beyond 5 models in my experience.\n\n- Increasing image size helped, gave diminished returns beyond 384x384, not really sure about the number.\n\nWhat I'll try:\n\n- Better way to choose the **new_whale** threshold, see how far I can get with that approach.\n\n- Metrics learning, starting with @martinpiotte 's solution.\n\n- A different loss functions: Triplet loss, Magnet loss yadda yadda\n\n**Update 1**\n\nMartin's solution is ported here: https://www.kaggle.com/suicaokhoailang/martin-piotte-s-siamese-baseline . Very promising indeed.\n\n\n**Update 2**\n\nA few clues about high scoring solutions:\n\n- The currently #1 solution used metrics learning: https://www.kaggle.com/c/humpback-whale-identification/discussion/74402#443160\n\n- My 0.898 submission was from a single siamese. \n\n- Classification is much better than I thought it would be, able to get 0.9+ LB: https://www.kaggle.com/c/humpback-whale-identification/discussion/75846#452220\n\nAlso, another approach on whether to include **new_whale** in your predictions using nearest neighbors: https://www.kaggle.com/iafoss/similarity-resnext50-0-602-lb\n\n [1]: https://www.kaggle.com/suicaokhoailang/faster-and-better-resnet50-without-class-0-0-598\n [2]: https://www.kaggle.com/suicaokhoailang/resnet50-bounding-boxes-0-628-lb\n [3]: https://www.kaggle.com/suicaokhoailang/ensembling-with-averaged-probabilities-0-701-lb\n [4]: https://www.kaggle.com/matthewa313/ensembling-algorithm-for-average-precision-metric", "453223": "I hope you guys do *whale* in the competition!", "463464": "Update 3 : ）\nSome labelled data are shared by participants:\n- all training images' masks, labelled and predicted images are mixed ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78453 )\n- 1000 annotated fluke key points ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78699 )\n- 450 fluke masks ( https://www.kaggle.com/c/humpback-whale-identification/discussion/78257 )", "444476": "I am working with @martinpiotte 's solution. I've modified the lap to go with an imperfect answer instead of an exact answer. I ran 225 epochs at an hour per epoch with the initial implementation. Toady I changed the lap.lapjv to lapjv.lapjv and segmented the data into 6 segments running each in it's own thread. With these changes I am now running at 7-8 minutes per epoch.", "441738": "Thanks for sharing.\nI found that using grayscale(num_output_channels=3) can increase the score, but not sure if it is universal", "437663": "Hi there, I am new to this competition (okay, yah, not that new, toke me a couple of days to get initial submission done). My hunch feeling is on how to exclude/ insert the new_whale data.\nHave anyone tried test/ identify the new whale data and then re-train with all dataset?", "437506": "Thanks for sharing.\n\nOne more question: do you believe only classification model would perform better than 0.8? I notice that the 1st place at present got above 0.9, and I guess maybe we could try other methods(e.g. detection or others) to get there.", "463713": "Hello, how are you get score above 0.9?", "460568": "In the martinpiotte's solution, he compute a derangement of its pictures. What is the point of this operation?", "453575": "Hi :\nAre you using martinpiotte's solution as baseline and modifying it or justing implement your own code?", "450750": "Do you think we should remove class 0 when we try metrics learning as well ? Metrics learnings don't tend to overfit compared with classification. I am investing the way to use new_whales effectively.", "446766": "Hi Khoi, am I correct that you have been using just the submission csvs for the ensembling? \n\nI tried ensembling with the full model predictions and that seems to score considerably better (0.727 vs 0.71 for my not-so-finetuned ensemble)?", "442276": "Are people holding out a portion of the training dataset for validation when training their models? or are people training the models on the entire training dataset and evaluating it's fit after submitted to the competition for scoring?", "439590": "Excellent thread. I was wondering how you landed on 384x384 specifically?", "438701": "and how about the focal loss to solve the unbalance label？", "438670": "How are you dealing with image size? Are you \"Squashing\" everything so each image is a square, or are you maintaining each specific image's aspect ratio ", "440105": "", "439340": "" } }