![\"Model](\"http://i.imgur.com/kQFtX6s.jpg\")
I share the code on github here.
\n\nThe approach is very basic but is based on sound methodology. In some sense, this made working on this a pleasure.
\n\nIt is a resnet50 model trained in a way to address specifics of the dataset. Local CV tracks LB score closely.
\n\nNow I realize this places high on the LB at the moment. But in reality this is just a small step above where the blenders are. Also, the competition just launched which also needs to be taken into consideration.
\n\nI chose to share this because I think it can be a very good learning resource and a good way for people to find their bearings. To some extent, this demonstrates a canonical way of solving a deep learning classification problem using modern tools (this does not imply that phrasing this competition as a classification problem is the best way to address it, though it would be interesting to see how far one can take this approach).
", "messages": [ { "id": 438771, "postDate": "2018-12-14T06:44:06.767Z", "content": "I share the code on github here.
\n\nThe approach is very basic but is based on sound methodology. In some sense, this made working on this a pleasure.
\n\nIt is a resnet50 model trained in a way to address specifics of the dataset. Local CV tracks LB score closely.
\n\nNow I realize this places high on the LB at the moment. But in reality this is just a small step above where the blenders are. Also, the competition just launched which also needs to be taken into consideration.
\n\nI chose to share this because I think it can be a very good learning resource and a good way for people to find their bearings. To some extent, this demonstrates a canonical way of solving a deep learning classification problem using modern tools (this does not imply that phrasing this competition as a classification problem is the best way to address it, though it would be interesting to see how far one can take this approach).
", "rawMarkdown": "I share the code on github [here][1].\n\nThe approach is very basic but is based on sound methodology. In some sense, this made working on this a pleasure.\n\nIt is a resnet50 model trained in a way to address specifics of the dataset. Local CV tracks LB score closely.\n\nNow I realize this places high on the LB at the moment. But in reality this is just a small step above where the blenders are. Also, the competition just launched which also needs to be taken into consideration. \n\nI chose to share this because I think it can be a very good learning resource and a good way for people to find their bearings. To some extent, this demonstrates a canonical way of solving a deep learning classification problem using modern tools (this does not imply that phrasing this competition as a classification problem is the best way to address it, though it would be interesting to see how far one can take this approach).\n\n\n [1]: https://github.com/radekosmulski/whale", "votes": 95 }, { "id": 439763, "postDate": "2018-12-16T09:48:24.827Z", "content": "hi Radek, how did you address this issue: \"OSError: [Errno 30] Read-only file system: '../input/models'\", during execution of \"learn = create_cnn(data, models.resnet50, metrics=[accuracy])\"? I tried to load this one before executing your script in kaggle kernel: !ln -s /kaggle/input/pytorch-pretrained-models/ /opt/conda/lib/python3.6/site-packages/fastai/weights but did not have success? Do you have any idea on how to solve this issue? Thanks!
", "rawMarkdown": "hi Radek, how did you address this issue: \"OSError: [Errno 30] Read-only file system: '../input/models'\", during execution of \"learn = create_cnn(data, models.resnet50, metrics=[accuracy])\"? I tried to load this one before executing your script in kaggle kernel: !ln -s /kaggle/input/pytorch-pretrained-models/ /opt/conda/lib/python3.6/site-packages/fastai/weights but did not have success? Do you have any idea on how to solve this issue? Thanks!", "votes": 12, "replies": [ { "id": 439811, "postDate": "2018-12-16T12:24:51.327Z", "content": "Sorry, I don't. I am not familiar with Kaggle kernels.
", "rawMarkdown": "Sorry, I don't. I am not familiar with Kaggle kernels." }, { "id": 441152, "postDate": "2018-12-18T10:32:26.337Z", "content": "(Edited)Hi,\nBased on radek's starter pack, I have created a Kaggle kernel for this competition. Github-https://github.com/bhuvanakundumani/hump_back_whale_Kaggle_Kernel
\n\nI am getting an error while i run learn = create_cnn(data, models.resnet34, metrics=[accuracy, map5])\nThe error :AttributeError: 'NoneType' object has no attribute 'detach'
", "rawMarkdown": "(Edited)Hi,\nBased on radek's starter pack, I have created a Kaggle kernel for this competition. Github-https://github.com/bhuvanakundumani/hump_back_whale_Kaggle_Kernel\n\nI am getting an error while i run learn = create_cnn(data, models.resnet34, metrics=[accuracy, map5])\nThe error :AttributeError: 'NoneType' object has no attribute 'detach'\n", "votes": -1 }, { "id": 441774, "postDate": "2018-12-19T03:49:12.607Z", "content": "you need to pass in model_dir=MODEL_PATH where MODEL_PATH = \"/tmp/model/\"
", "rawMarkdown": "you need to pass in model_dir=MODEL_PATH where MODEL_PATH = \"/tmp/model/\"" }, { "id": 442909, "postDate": "2018-12-20T17:14:15.123Z", "content": "Thanks.
", "rawMarkdown": "Thanks. " } ] }, { "id": 481638, "postDate": "2019-03-01T16:42:24.807Z", "content": "Hi Radek, I just wanted to thank you for the resources! Having just started the fastai course it was a great help to gain some more understanding.
\n\nI was wondering what kind of machine you are running your training on. I was using a P2.xlarge SageMaker Instance and I had significantly longer runtimes than you, but wasn't quite willing to scale up.
", "rawMarkdown": "Hi Radek, I just wanted to thank you for the resources! Having just started the fastai course it was a great help to gain some more understanding.\n \nI was wondering what kind of machine you are running your training on. I was using a P2.xlarge SageMaker Instance and I had significantly longer runtimes than you, but wasn't quite willing to scale up.", "votes": 3, "replies": [ { "id": 548374, "postDate": "2019-06-09T09:40:07.887Z", "content": "Sorry for the late reply - for the training I used a rig that I built with a ryzen 5, 16 gb of RAM and a single 1080ti. p2.xlarge has an older gen GPU so slower train time is expected - you would probably get comparable train times (probably even slightly faster) on a p3.2xlarge
", "rawMarkdown": "Sorry for the late reply - for the training I used a rig that I built with a ryzen 5, 16 gb of RAM and a single 1080ti. p2.xlarge has an older gen GPU so slower train time is expected - you would probably get comparable train times (probably even slightly faster) on a p3.2xlarge" } ] }, { "id": 454951, "postDate": "2019-01-12T15:48:54.360Z", "content": "Hey Radek, this is a amazing resource, thank you for sharing!
\n\nIt also cleared up a big confusion I had about the fastai library and training speed:
\n\nIn my recent notebooks fit_one_cycle and lr_find got stuck for about a second every 8 batches (num_workers=8, high batch_size)
\n\nAfter resizing the Images to the correct size beforehand this behaviour stopped so I'm pretty sure the internal resizing process slowed down my training!
", "rawMarkdown": "Hey Radek, this is a amazing resource, thank you for sharing!\n\n\nIt also cleared up a big confusion I had about the fastai library and training speed:\n\nIn my recent notebooks fit\\_one\\_cycle and lr\\_find got stuck for about a second every 8 batches (num_workers=8, high batch\\_size)\n\nAfter resizing the Images to the correct size beforehand this behaviour stopped so I'm pretty sure the internal resizing process slowed down my training!", "votes": 3, "replies": [ { "id": 455011, "postDate": "2019-01-12T18:41:25.997Z", "content": "Thank you for sharing this! I’ve been seeing the same problem in another competition and hadn’t figured it out yet.
", "rawMarkdown": "Thank you for sharing this! I’ve been seeing the same problem in another competition and hadn’t figured it out yet.", "votes": 1 }, { "id": 455211, "postDate": "2019-01-13T08:52:59.597Z", "content": "I always train with nvidia-smi -l and htop running, this helps me diagnose a lot of problems like this. For resizing for instance, I will see the cores maxed out while at the same time the GPU not being fed quickly enough.
\n\nI use tmux and am making an effort to start using tmuxinator for more projects. It is a really cool little program where with just a single command it brings up as many tmux panes as you'd like and starts the programs that you'd like. My pane #1 is nvidia-smi, #2 is htop and #3 is the fastai directory of the fastai library, likely with VIM open. This allows me to get some consistency regardless what I work on.
", "rawMarkdown": "I always train with nvidia-smi -l and htop running, this helps me diagnose a lot of problems like this. For resizing for instance, I will see the cores maxed out while at the same time the GPU not being fed quickly enough.\n\nI use tmux and am making an effort to start using tmuxinator for more projects. It is a really cool little program where with just a single command it brings up as many tmux panes as you'd like and starts the programs that you'd like. My pane #1 is nvidia-smi, #2 is htop and #3 is the fastai directory of the fastai library, likely with VIM open. This allows me to get some consistency regardless what I work on.", "votes": 2 }, { "id": 461405, "postDate": "2019-01-25T23:32:26.597Z", "content": "I add a 4th pane: sudo iotop\nObserving the disk activity
", "rawMarkdown": "I add a 4th pane: sudo iotop\nObserving the disk activity" } ] }, { "id": 559465, "postDate": "2019-06-24T06:43:17.600Z", "content": "Thanks for sharing! always been looking for fastai-based project like this
", "rawMarkdown": "Thanks for sharing! always been looking for fastai-based project like this", "votes": 1 }, { "id": 464615, "postDate": "2019-02-01T06:53:01.240Z", "content": "Thanks for sharing your code, @radek. I'm trying to run it first.
\n\nI have a feeling the recently added annotations.json hasn't been tested with fluke_detection.ipynb. Not sure why it's expecting the fluke class entry to be at first position, but it's not always the case, so collate fn fails. Here is a dump of one such entry where it's second:
{\"annotations\": [\n\n{\"class\": \"left\",\n \"type\": \"point\",\n \"x\": 0.7092198581560337,\n \"y\": 134.7517730496464},\n\n {\"class\": \"fluke\",\n \"height\": 253.9007092198601,\n \"type\": \"rect\",\n \"width\": 1039.0070921985894,\n \"x\": 0.7092198581560337,\n \"y\": 77.30496453900767},\n\n {\"class\": \"notch\",\n \"type\": \"point\",\n \"x\": 496.878528952789,\n \"y\": 261.35895342188473},\n\n {\"class\": \"right\",\n \"type\": \"point\",\n \"x\": 1036.170212766017,\n \"y\": 81.56028368794794}\n\n],\n \"class\": \"image\",\n \"filename\": \"0bb469e7d.jpg\"}\nIt's probably a bad idea to rely on a fixed position in a json file.
\n\nI applied a quick fix to match the entry that has 'width', there is probably a more elegant solution to choose the class fluke. This is just the beginning of the function:
\n\ndef j2anno(j):\n # bbox coordinates are returned in pascal voc format [x_min, y_min, x_max, y_max]\n im = cv2.imread(f\"data/train-{SZ}/{j['filename']}\", cv2.IMREAD_COLOR) \n im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n im_height, im_width, _ = im.shape\n\n orig_im = cv2.imread(f\"data/train/{j['filename']}\", cv2.IMREAD_COLOR)\n orig_im_height, orig_im_width, _ = orig_im.shape\n\n# bbox_info = j['annotations'][0]\n bbox_info = None \n for i in j['annotations']:\n if 'width' in i:\n bbox_info = i\n break\nThanks.
", "rawMarkdown": "Thanks for sharing your code, @radek. I'm trying to run it first.\n\nI have a feeling the recently added `annotations.json` hasn't been tested with `fluke_detection.ipynb`. Not sure why it's expecting the fluke class entry to be at first position, but it's not always the case, so collate fn fails. Here is a dump of one such entry where it's second:\n\n {\"annotations\": [\n \n {\"class\": \"left\",\n \"type\": \"point\",\n \"x\": 0.7092198581560337,\n \"y\": 134.7517730496464},\n \n {\"class\": \"fluke\",\n \"height\": 253.9007092198601,\n \"type\": \"rect\",\n \"width\": 1039.0070921985894,\n \"x\": 0.7092198581560337,\n \"y\": 77.30496453900767},\n \n {\"class\": \"notch\",\n \"type\": \"point\",\n \"x\": 496.878528952789,\n \"y\": 261.35895342188473},\n \n {\"class\": \"right\",\n \"type\": \"point\",\n \"x\": 1036.170212766017,\n \"y\": 81.56028368794794}\n \n ],\n \"class\": \"image\",\n \"filename\": \"0bb469e7d.jpg\"}\n\nIt's probably a bad idea to rely on a fixed position in a json file.\n\nI applied a quick fix to match the entry that has 'width', there is probably a more elegant solution to choose the class fluke. This is just the beginning of the function:\n\n\n def j2anno(j):\n # bbox coordinates are returned in pascal voc format [x_min, y_min, x_max, y_max]\n im = cv2.imread(f\"data/train-{SZ}/{j['filename']}\", cv2.IMREAD_COLOR) \n im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n im_height, im_width, _ = im.shape\n \n orig_im = cv2.imread(f\"data/train/{j['filename']}\", cv2.IMREAD_COLOR)\n orig_im_height, orig_im_width, _ = orig_im.shape\n \n # bbox_info = j['annotations'][0]\n bbox_info = None \n for i in j['annotations']:\n if 'width' in i:\n bbox_info = i\n break\n\nThanks.", "votes": 1, "replies": [ { "id": 465519, "postDate": "2019-02-03T10:43:02.403Z", "content": "Thank you for the heads up on this Stas, good points. I now updated the notebook and pushed the fixed version.
", "rawMarkdown": "Thank you for the heads up on this Stas, good points. I now updated the notebook and pushed the fixed version.", "votes": 1 }, { "id": 465701, "postDate": "2019-02-03T20:07:28.607Z", "content": "just don't forget to push!
", "rawMarkdown": "just don't forget to push!", "votes": 1 }, { "id": 465924, "postDate": "2019-02-04T10:35:30.240Z", "content": "thank you Stas :) I pushed but to Bitbucket by accident - now everything should be in place on github as well
", "rawMarkdown": "thank you Stas :) I pushed but to Bitbucket by accident - now everything should be in place on github as well" } ] }, { "id": 459560, "postDate": "2019-01-22T01:54:15.383Z", "content": "Thank you very much for sharing, but I can't find the boxes.json file in your github. Can you re-send it?
", "rawMarkdown": "\nThank you very much for sharing, but I can't find the boxes.json file in your github. Can you re-send it?\n", "votes": 1, "replies": [ { "id": 459700, "postDate": "2019-01-22T07:52:27.477Z", "content": "Sorry about this - I renamed the file to annotations.json and must have not renamed it in code everywhere. Will correct shortly.
", "rawMarkdown": "Sorry about this - I renamed the file to annotations.json and must have not renamed it in code everywhere. Will correct shortly." }, { "id": 459796, "postDate": "2019-01-22T11:37:51.427Z", "content": "ohoh,thanks,and Is there any change from boxes.json to annotations.json?
", "rawMarkdown": "\nohoh,thanks,and Is there any change from boxes.json to annotations.json?" }, { "id": 459803, "postDate": "2019-01-22T11:48:34.373Z", "content": "this should be fixed now - let me know if you have any issues
", "rawMarkdown": "this should be fixed now - let me know if you have any issues" } ] }, { "id": 532431, "postDate": "2019-05-17T00:05:34.183Z", "content": "Hi I am new to deep learning and have question specific to a file. I was analyzing the output of @radek1 for file only_known_train.ipynb which seems strange to me as per theoretical concept. I am talking about the training loss is higher than the validation loss. I understand the imbalance in the data set but after oversampling the scenario is same. I am sorry for the silly question.
", "rawMarkdown": "Hi I am new to deep learning and have question specific to a file. I was analyzing the output of @radek1 for file only_known_train.ipynb which seems strange to me as per theoretical concept. I am talking about the training loss is higher than the validation loss. I understand the imbalance in the data set but after oversampling the scenario is same. I am sorry for the silly question.", "votes": 2, "replies": [ { "id": 535784, "postDate": "2019-05-23T12:46:02.127Z", "content": "I'm also no expert but I will try to explain:
\n\nYou can see @radek1 intentionally used a validation set with only one image:\nval_fns = {'69823499d.jpg'}
He explains his reasoning for this in the beginning:\n``` \nI take a curriculum approach to training here. I first expose the model to as many different images of whales as quickly as possible (no oversampling) and train on images resized to 224x224.
\n\nI would like the conv layers to start picking up on features useful for identifying whales. For that, I want to show the model as rich of a dataset as possible. \n```
\n\nMy explanation
\n\nThis is quite an odd thing to do but it makes more sense in the context of this competition.\nWe had 3,000+ classes with only a few images per class. \nPutting some images of each class from the training set in the validation set would lead to classes having almost no images to learn from.
\n\nUsing only 1 image in the validation set makes the validation loss an useless indicator because it only shows if this 1 image is predicted correctly.
\n\nSo the payoff you get is you train on all images and don't loose valuable data for validation, but you don't know how good the model predicts on images that are not used to change weights (it does not know yet).\nThis makes it very hard (impossible?) to know when the model is overfitting.
", "rawMarkdown": "I'm also no expert but I will try to explain:\n\nYou can see @radek1 intentionally used a validation set with only one image:\n`val_fns = {'69823499d.jpg'}`\n\nHe explains his reasoning for this in the beginning:\n``` \nI take a curriculum approach to training here. I first expose the model to as many different images of whales as quickly as possible (no oversampling) and train on images resized to 224x224.\n\nI would like the conv layers to start picking up on features useful for identifying whales. For that, I want to show the model as rich of a dataset as possible. \n```\n\n**My explanation** \n\nThis is quite an odd thing to do but it makes more sense in the context of this competition.\nWe had 3,000+ classes with only a few images per class. \nPutting some images of each class from the training set in the validation set would lead to classes having almost no images to learn from.\n\nUsing only 1 image in the validation set makes the validation loss an useless indicator because it only shows if this 1 image is predicted correctly. \n\nSo the payoff you get is you train on all images and don't loose valuable data for validation, but you don't know how good the model predicts on images that are not used to change weights (it does not know yet).\nThis makes it very hard (impossible?) to know when the model is overfitting.", "votes": 2 }, { "id": 536035, "postDate": "2019-05-23T21:09:38.897Z", "content": "Thank You. I got the point now.
", "rawMarkdown": "Thank You. I got the point now." } ] }, { "id": 446590, "postDate": "2018-12-28T10:27:17.217Z", "content": "Jeremy mentioned that their Dawnbench implementation of image size is better than the current one. It can use rectangular images without changing the aspect ratio (no squash and not even crop). Unfortunately it is not implemented yet, and will be implemented in fastai 2nd course in March.
\n\nI suspect this would be important for a dataset like this, where image sizes are not consistent resulting in different amount of squashing the image.
\n\nMore details here and Jeremy's replies :\nhttps://forums.fast.ai/t/best-way-to-resize-pictures-for-model-training/28307/27?u=hwasiti
", "rawMarkdown": "Jeremy mentioned that their Dawnbench implementation of image size is better than the current one. It can use rectangular images without changing the aspect ratio (no squash and not even crop). Unfortunately it is not implemented yet, and will be implemented in fastai 2nd course in March.\n\nI suspect this would be important for a dataset like this, where image sizes are not consistent resulting in different amount of squashing the image. \n\nMore details here and Jeremy's replies :\nhttps://forums.fast.ai/t/best-way-to-resize-pictures-for-model-training/28307/27?u=hwasiti", "votes": 1, "replies": [ { "id": 448483, "postDate": "2019-01-01T10:06:37.263Z", "content": "I think this approach sounds very interesting. I'm rather new in the field and would like to check out the details on fast.ai. I just registered, but apparently I do not have access to the topic. Is it supposed to be hidden?
", "rawMarkdown": "I think this approach sounds very interesting. I'm rather new in the field and would like to check out the details on fast.ai. I just registered, but apparently I do not have access to the topic. Is it supposed to be hidden?" }, { "id": 461406, "postDate": "2019-01-25T23:33:42.197Z", "content": "Today it has been released for public
", "rawMarkdown": "Today it has been released for public", "votes": 1 } ] }, { "id": 441246, "postDate": "2018-12-18T13:20:28.247Z", "content": "Thanks for the kernel! You can try using bbox of the image. Details can be found at https://www.kaggle.com/suicaokhoailang/generating-whale-bounding-boxes.
", "rawMarkdown": "Thanks for the kernel! You can try using bbox of the image. Details can be found at https://www.kaggle.com/suicaokhoailang/generating-whale-bounding-boxes.", "votes": 1 }, { "id": 439869, "postDate": "2018-12-16T14:45:57.820Z", "content": "Sorry, how to solve this problem that \n'Exception: Your validation data contains a label that isn't present in the training set, please fix your data.'? \nAnd only appear in when I use this\ndata = (ImageItemList. ... .random_split_by_pct(seed=0)) \nand it tell me about \nKeyError: 'w_d8a08f8'\nSorry, I am not familiar with fastai.\nThis problem occurred when I was researching this code and I wanted to know why. Any help would be useful. Thank you!
", "rawMarkdown": "Sorry, how to solve this problem that \n'Exception: Your validation data contains a label that isn't present in the training set, please fix your data.'? \nAnd only appear in when I use this\ndata = (ImageItemList. ... .random_split_by_pct(seed=0)) \nand it tell me about \nKeyError: 'w_d8a08f8'\nSorry, I am not familiar with fastai.\nThis problem occurred when I was researching this code and I wanted to know why. Any help would be useful. Thank you!", "votes": 1, "replies": [ { "id": 440016, "postDate": "2018-12-16T21:42:25.440Z", "content": "I am not sure why people are getting this error. I will try to find time tomorrow to change the code so that this no longer will be an issue.
", "rawMarkdown": "I am not sure why people are getting this error. I will try to find time tomorrow to change the code so that this no longer will be an issue.", "votes": 1 }, { "id": 440624, "postDate": "2018-12-17T19:55:25.123Z", "content": "@gakki Hello, I have not gone through the code, but I would run into this issue when doing a train/valid split at random. You end up having labels in your validation set that do not exist in the training set and the dataloader would fail. To combat this, I do my train/valid splits at the label level. This ensures the error does not occur and also keeps my validation set balanced with the training data. Below is a quick example:\nhttps://gist.github.com/sariabod/56aeea96abe58671375eb52a883e356e
", "rawMarkdown": "@gakki Hello, I have not gone through the code, but I would run into this issue when doing a train/valid split at random. You end up having labels in your validation set that do not exist in the training set and the dataloader would fail. To combat this, I do my train/valid splits at the label level. This ensures the error does not occur and also keeps my validation set balanced with the training data. Below is a quick example:\nhttps://gist.github.com/sariabod/56aeea96abe58671375eb52a883e356e", "votes": 5 }, { "id": 441018, "postDate": "2018-12-18T07:27:32.080Z", "content": "Thank you, I think I almost know why, That's very useful.
", "rawMarkdown": "Thank you, I think I almost know why, That's very useful." }, { "id": 441049, "postDate": "2018-12-18T08:04:17.413Z", "content": "I am experiencing some hw issues, hence making the changes is taking longer than I anticipated. One solution to this would be to move how the validation set is constructed in later notebooks over to the first_submission one.
", "rawMarkdown": "I am experiencing some hw issues, hence making the changes is taking longer than I anticipated. One solution to this would be to move how the validation set is constructed in later notebooks over to the first_submission one." }, { "id": 441957, "postDate": "2018-12-19T09:40:00.197Z", "content": "I decided to not change the code so that in first_submission.ipynb we would sample the validation set like we do in subsequent notebooks.
\n\nI do not want to make the code of the first_submission notebook unnecessarily complex. In general, I go by the philosophy that the first submission should be as simple as it can be. In fact, my first submission to this competition was predicting only the new_whale class. This way I was able to tell quite a few of the things I had in place worked correctly. It also allowed me to learn something about the LB test set that was quite useful for the construction of the validation set.
If you encounter an error while running the first_submission notebook, fixing it could be a fun exercise (moving the code for creating the validation set over from one of the later notebooks). In either case, you should be okay to skip this notebook and still run any of the later notebooks just fine, including the one that achieves 0.760 on the LB (only_known_train.ipynb)
@gakki Hello! What did you do to get rid of that error?
", "rawMarkdown": "@gakki Hello! What did you do to get rid of that error?" }, { "id": 442826, "postDate": "2018-12-20T15:22:15.993Z", "content": "@Ankesh Sorry, I don't think I solved this problem very well, I just use some of the more categorical categories (like w_23a388d) as the validation set.
", "rawMarkdown": "@Ankesh Sorry, I don't think I solved this problem very well, I just use some of the more categorical categories (like w_23a388d) as the validation set." } ] }, { "id": 438920, "postDate": "2018-12-14T12:18:01.603Z", "content": "Thank you for sharing this! The Kernel notebooks are somehow confusing to me, but your github repository is far easier to navigate.
", "rawMarkdown": "Thank you for sharing this! The Kernel notebooks are somehow confusing to me, but your github repository is far easier to navigate.", "votes": 1 }, { "id": 480144, "postDate": "2019-02-27T20:36:27.977Z", "content": "Great kernel Radek ! The map5 implementation in the fastai learner class I found particularly useful
", "rawMarkdown": "Great kernel Radek ! The map5 implementation in the fastai learner class I found particularly useful", "votes": 2 }, { "id": 439266, "postDate": "2018-12-15T03:50:14.190Z", "content": "Good work ! How about sharing your code on Kaggle-Kernel to ensure reproducibility ?
", "rawMarkdown": "Good work ! How about sharing your code on Kaggle-Kernel to ensure reproducibility ?", "votes": 2, "replies": [ { "id": 439578, "postDate": "2018-12-15T20:50:25.183Z", "content": "This should be possible now because Kernels upgraded to fastai v1.
", "rawMarkdown": "This should be possible now because Kernels upgraded to fastai v1.", "votes": 2 }, { "id": 440408, "postDate": "2018-12-17T14:11:54.487Z", "content": "I tried running the code on Kaggla Kernels, but got some errors. Did anyone try it?
", "rawMarkdown": "I tried running the code on Kaggla Kernels, but got some errors. Did anyone try it?", "votes": 2 }, { "id": 444939, "postDate": "2018-12-25T07:24:59.580Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 468284, "postDate": "2019-02-08T16:10:05.397Z", "content": "Have people tried using these classification notebooks with different models bar Resnet 50, if so what LR's did you use and which model?
", "rawMarkdown": "Have people tried using these classification notebooks with different models bar Resnet 50, if so what LR's did you use and which model?" }, { "id": 453824, "postDate": "2019-01-10T21:01:12.083Z", "content": "If I mimic the code in 'only_known_train', I get much worse performance. Does anyone else have this issue?
", "rawMarkdown": "If I mimic the code in 'only_known_train', I get much worse performance. Does anyone else have this issue?" }, { "id": 448821, "postDate": "2019-01-02T07:00:18.543Z", "content": "Hi Radek,
\n\nThanks a lot for the starter pack. I am a starter into Computer Vision, had some queries from your code. Request you to please help me with it.
\n\n1) preds[:, 5004] = 0.06 \nWhy did you use the above code ? Didn't really get it
\n\n2) classes = learn.data.classes + ['new_whale']
\n\nHow exactly it updates the new_whale for the top5 predictions, even when the model doesn't has \"new_whale\".
\n\nSorry if the questions are stupid.
", "rawMarkdown": "Hi Radek,\n\nThanks a lot for the starter pack. I am a starter into Computer Vision, had some queries from your code. Request you to please help me with it.\n\n1) preds[:, 5004] = 0.06 \nWhy did you use the above code ? Didn't really get it\n\n2) classes = learn.data.classes + ['new_whale']\n\nHow exactly it updates the new_whale for the top5 predictions, even when the model doesn't has \"new_whale\".\n\nSorry if the questions are stupid. ", "replies": [ { "id": 448842, "postDate": "2019-01-02T07:59:17.280Z", "content": "Thanks Raghava!!\nCan you please elaborate how to get the threshold for i.e. 0.06. Didn't really get what is \"tested using ps\"
", "rawMarkdown": "Thanks Raghava!!\nCan you please elaborate how to get the threshold for i.e. 0.06. Didn't really get what is \"tested using ps\"" }, { "id": 448861, "postDate": "2019-01-02T09:01:19.477Z", "content": "Refer to https://github.com/radekosmulski/whale/blob/master/only_known_research.ipynb notebook. Particularly this block of code
\n\nps = np.linspace(0, 1, 51)\nfor p in ps:\n preds[:, 5004] = p\n res.append(map5(preds, targs).item())\nbest_p = ps[np.argmax(res)]; best_p
", "rawMarkdown": "Refer to https://github.com/radekosmulski/whale/blob/master/only_known_research.ipynb notebook. Particularly this block of code\n\nps = np.linspace(0, 1, 51)\nfor p in ps:\n preds[:, 5004] = p\n res.append(map5(preds, targs).item())\nbest_p = ps[np.argmax(res)]; best_p", "votes": 1 }, { "id": 448868, "postDate": "2019-01-02T09:21:45.823Z", "content": "Thanks a ton Raghava !!!!
", "rawMarkdown": "Thanks a ton Raghava !!!! " }, { "id": 453832, "postDate": "2019-01-10T21:30:05.497Z", "content": "I would recommend to use \nps = np.linspace(0, 1, 201)\nYes, it is slower, but gives a bit more finegrained threshold
", "rawMarkdown": "I would recommend to use \nps = np.linspace(0, 1, 201)\nYes, it is slower, but gives a bit more finegrained threshold" } ] }, { "id": 446664, "postDate": "2018-12-28T13:01:33.240Z", "content": "@radek Thanks for the kernel. That was very helpful. I see that it was taking about more than an hour to run each experiment(training both head and base). Do u think the idea of carrying experiments on a much small subset of train data to do multiple quick iterations and picking the good ones to run on the whole data is a good idea in case of DL(this was much talked by jeremy in ML class)? And r u currently runnin ur experiments on the entire data?
", "rawMarkdown": "@radek Thanks for the kernel. That was very helpful. I see that it was taking about more than an hour to run each experiment(training both head and base). Do u think the idea of carrying experiments on a much small subset of train data to do multiple quick iterations and picking the good ones to run on the whole data is a good idea in case of DL(this was much talked by jeremy in ML class)? And r u currently runnin ur experiments on the entire data?", "replies": [ { "id": 447623, "postDate": "2018-12-30T07:28:24.577Z", "content": "I have not done a whole lot of running small experiments on this dataset. Sometimes training for a few epochs will suffice to say if a change is in the right direction or not, but for most hyperparameters one would need to run the full training (and even that might not give a conclusive answer).
\n\nIt is not easy to figure out what to train on and how to figure out if one is moving in the right direction or not. The lr finder gives some indication, the shape of the loss plot is also telling. This workflow of what works is something one needs to work out for themselves with time I am afraid (probably a big impact will also come from what resources one has available and how much time one has to give to this - since I can work on this only sporadically, this leaves large amount of time in between where I can run a longer training if I wanted to, though have not gotten to that point in this competition just yet).
", "rawMarkdown": "I have not done a whole lot of running small experiments on this dataset. Sometimes training for a few epochs will suffice to say if a change is in the right direction or not, but for most hyperparameters one would need to run the full training (and even that might not give a conclusive answer).\n\nIt is not easy to figure out what to train on and how to figure out if one is moving in the right direction or not. The lr finder gives some indication, the shape of the loss plot is also telling. This workflow of what works is something one needs to work out for themselves with time I am afraid (probably a big impact will also come from what resources one has available and how much time one has to give to this - since I can work on this only sporadically, this leaves large amount of time in between where I can run a longer training if I wanted to, though have not gotten to that point in this competition just yet). " } ] }, { "id": 443829, "postDate": "2018-12-22T14:14:52.660Z", "content": "Very good kernel. It looks a very standard way to implement cnn using fastai 1.0. I am trying to figure out why it performed about 0.1 better than other previous implementation in the forum. What is the secret sauce here? Is it the oversample method? Or the map5 metric? Or anything hidden in the details?
", "rawMarkdown": "Very good kernel. It looks a very standard way to implement cnn using fastai 1.0. I am trying to figure out why it performed about 0.1 better than other previous implementation in the forum. What is the secret sauce here? Is it the oversample method? Or the map5 metric? Or anything hidden in the details? ", "replies": [ { "id": 443929, "postDate": "2018-12-22T18:44:11.070Z", "content": "Map 5 is just being reported here -- it's telemetry, not feedback going back in to the model.
", "rawMarkdown": "Map 5 is just being reported here -- it's telemetry, not feedback going back in to the model." }, { "id": 443981, "postDate": "2018-12-22T20:37:26.513Z", "content": "My bet is one cycle policy + good default augmentation set
", "rawMarkdown": "My bet is one cycle policy + good default augmentation set", "votes": 2 }, { "id": 463709, "postDate": "2019-01-30T13:16:30.713Z", "content": "Hello, could i ask you how are you get above 0.9 ?
", "rawMarkdown": "Hello, could i ask you how are you get above 0.9 ?" } ] }, { "id": 443786, "postDate": "2018-12-22T12:26:37.163Z", "content": "It used fastai 1.0. Does 1.0 perform better than 0.7?
", "rawMarkdown": "It used fastai 1.0. Does 1.0 perform better than 0.7? ", "replies": [ { "id": 443801, "postDate": "2018-12-22T13:11:46.757Z", "content": "1.0
", "rawMarkdown": "1.0" } ] }, { "id": 442058, "postDate": "2018-12-19T12:25:37.520Z", "content": "Hi Radek, thank you again for the Kernel. I've trained the first two stages using your methodology for validation set creation a couple different ways (with upsampling, without, at 224, at 392, etc) and gotten decent results.
\n\nHowever, whenever I start progressive resizing, my model seems to fall apart, resulting in things like this (image size 448, 16 batch size):
\n\n
Here is the training on the second stage of training the model (224 image size, 64 batch size):
\n\n![\"Prior](\"https://i.imgur.com/gMlTKoS.jpg\")
Do you have any suggestions? Thank you for the time.
", "rawMarkdown": "Hi Radek, thank you again for the Kernel. I've trained the first two stages using your methodology for validation set creation a couple different ways (with upsampling, without, at 224, at 392, etc) and gotten decent results.\n\nHowever, whenever I start progressive resizing, my model seems to fall apart, resulting in things like this (image size 448, 16 batch size):\n\n![Model death][1]\n\nHere is the training on the second stage of training the model (224 image size, 64 batch size):\n\n![Prior to death][2]\n\nDo you have any suggestions? Thank you for the time.\n\n\n [1]: http://i.imgur.com/kQFtX6s.jpg\n [2]: https://i.imgur.com/gMlTKoS.jpg", "replies": [ { "id": 442108, "postDate": "2018-12-19T13:49:56.253Z", "content": "If I am reading this right (at the top you are continuing to train the model you trained at the bottom but with a larger image size) you might be using too high of an LR.
\n\nMight want to give the following lrs a try and see what happens:
\n\nmax_lr = 1e-3 / 4\nlrs = [max_lr/100, max_lr/10, max_lr]\nRadek, thanks for the reply. You were reading it right.
\n\nThat's what I was attempting with the 1e-06, 1e-05, 1e-04 vs. 2e-03.
\n\nI'll try cranking them down even further. Appreciate the insight!
", "rawMarkdown": "Radek, thanks for the reply. You were reading it right.\n\nThat's what I was attempting with the 1e-06, 1e-05, 1e-04 vs. 2e-03.\n\nI'll try cranking them down even further. Appreciate the insight!\n", "votes": 1 }, { "id": 442314, "postDate": "2018-12-19T19:34:02.227Z", "content": "The push on LR seems to have gotten me past that -- I was trying to use the learning rate vs. loss functions in fastai. I replaced with yours and that seems to have me improving from larger images.
\n\nDo you find your initial rates experimentally, or do you always divide your prior LRs by the scaling factor of the BS necessitated by the larger images?
\n\nIf you had a Titan with 24 gig of ram or whatever, would you still be training at these divided-by-four-rates?
\n\nThanks again for the time -- trying to use this competition to learn.
", "rawMarkdown": "The push on LR seems to have gotten me past that -- I was trying to use the learning rate vs. loss functions in fastai. I replaced with yours and that seems to have me improving from larger images.\n\nDo you find your initial rates experimentally, or do you always divide your prior LRs by the scaling factor of the BS necessitated by the larger images?\n\nIf you had a Titan with 24 gig of ram or whatever, would you still be training at these divided-by-four-rates?\n\nThanks again for the time -- trying to use this competition to learn.", "votes": 1 }, { "id": 442351, "postDate": "2018-12-19T21:11:44.523Z", "content": "There are no answers set in stone here. A lot of this comes through experience and having spent a lot of time reading / implementing papers. The experience does not necessarily come through doing complex or fancy things - mostly coming up with a small project I could complete in a couple of days. Fastai lesson notebooks (and applying the techniques there to other datasets) is what would constitute the core of how I learn.
\n\nAnyhow, yes, one can use this rule of a thumb that if you decrease the batch size by 4, you might want to decrease your LR by a factor of 4 as well. Adam, the optimizer we are using here, seems to be quite sensitive to parameter values, but for instance using SGD there have been instances where I would cut down the BS by half and not reduce the LR at all. That's where gradient clipping comes in handy.
\n\nA large dataset like this one makes it a real pain to experiment and start getting an intuition on what could be going on (an learning to read the signs whether you are overfitting / underfitting, etc). It's much easier to pick this up using something like cifar10. Here is a repo I put together for training on cifar10 using docker and the previous version of fastai and here are some of the small projects I mentioned above.
\n\nBut anyhow, I bet nothing beats working through the course notebooks. And for nearly anything you can throw at the GPU, the defaults fastai v1 comes with are really, really good.
", "rawMarkdown": "There are no answers set in stone here. A lot of this comes through experience and having spent a lot of time reading / implementing papers. The experience does not necessarily come through doing complex or fancy things - mostly coming up with a small project I could complete in a couple of days. Fastai lesson notebooks (and applying the techniques there to other datasets) is what would constitute the core of how I learn.\n\nAnyhow, yes, one can use this rule of a thumb that if you decrease the batch size by 4, you might want to decrease your LR by a factor of 4 as well. Adam, the optimizer we are using here, seems to be quite sensitive to parameter values, but for instance using SGD there have been instances where I would cut down the BS by half and not reduce the LR at all. That's where gradient clipping comes in handy.\n\nA large dataset like this one makes it a real pain to experiment and start getting an intuition on what could be going on (an learning to read the signs whether you are overfitting / underfitting, etc). It's much easier to pick this up using something like cifar10. Here is a [repo](https://github.com/radekosmulski/cifar10_docker) I put together for training on cifar10 using docker and the previous version of fastai and here are some of the [small projects](https://github.com/radekosmulski/machine_learning_notebooks) I mentioned above.\n\nBut anyhow, I bet nothing beats working through the course notebooks. And for nearly anything you can throw at the GPU, the defaults fastai v1 comes with are really, really good.", "votes": 3 }, { "id": 443329, "postDate": "2018-12-21T12:51:52.973Z", "content": "Hi @radek1,
\n\nThere's a paper from 2018 that directly supports your approach to tweaking LRs when decreasing/increasing batchsize:
\n\nhttps://arxiv.org/abs/1706.02677
\n\nThanks again for sharing your approaches in this challenge.
\n\n-L
", "rawMarkdown": "Hi @radek1,\n\nThere's a paper from 2018 that directly supports your approach to tweaking LRs when decreasing/increasing batchsize:\n\nhttps://arxiv.org/abs/1706.02677\n\nThanks again for sharing your approaches in this challenge.\n\n-L\n", "votes": 2 } ] }, { "id": 439281, "postDate": "2018-12-15T04:47:57.063Z", "content": "Hey, handful of questions if you've got the time:
\n\n1) Impact of lin_ftrs -- Can you talk about that?\n2) Impact of clip_grad().\n3) Totally niave but can you explain what's happening in:\nfn2label = {row[1].Image: row[1].Id for row in df.iterrows()}\nWhen doing the labeling? I can't for the life of me figure out how this works.\nThanks, and sorry for the silly questions.
fn2label is a dict mapping image name to label. In a fastai data block you can label your data from a function .label_from_func(lambda path: fn2label[path2fn(path)]) given a path. He uses the data block to create the DataBunch object. Read more about the data block here: https://docs.fast.ai/data_block.html
1) Increasing lin_ftrs makes the head bigger, increases the capacity of the model. Given the way I chose to train, this particular value gave me a better result on my validation set than the default value of 512.\n2) I assume it to be helpful when training with high learning rates.
", "rawMarkdown": "1) Increasing lin_ftrs makes the head bigger, increases the capacity of the model. Given the way I chose to train, this particular value gave me a better result on my validation set than the default value of 512.\n2) I assume it to be helpful when training with high learning rates. " }, { "id": 442106, "postDate": "2018-12-19T13:48:15.063Z", "content": "Thanks for the explanation on lin_ftrs -- Read up a bit in various places and I think it makes more sense. This is most due to image size and number of classes I believe?
", "rawMarkdown": "Thanks for the explanation on lin_ftrs -- Read up a bit in various places and I think it makes more sense. This is most due to image size and number of classes I believe?" }, { "id": 442143, "postDate": "2018-12-19T14:30:54.740Z", "content": "Yes, that could be the case but I am not sure one can get a lot of mileage out of trying to figure out why a more complex model is required, or guess the complexity of model required up front. What if we halved the number of classes? Would this make much of a difference? I am not really sure one is able to answer this question without running experiments.
\n\nHere, given the specific architecture and this particular dataset, seemed we were underfitting with the default value of 512 and a bigger number of trainable weights in the classifier seemed like something worth exploring.
\n\nBased on experiments, this seemed like an okay value when training without including the validation set in the train set (in the only_known_research NB) but there is an overwhelming chance (supported by the train loss) that this might again be too low of a value for training on the entire train set of known whales.
", "rawMarkdown": "Yes, that could be the case but I am not sure one can get a lot of mileage out of trying to figure out why a more complex model is required, or guess the complexity of model required up front. What if we halved the number of classes? Would this make much of a difference? I am not really sure one is able to answer this question without running experiments.\n\nHere, given the specific architecture and this particular dataset, seemed we were underfitting with the default value of 512 and a bigger number of trainable weights in the classifier seemed like something worth exploring.\n\nBased on experiments, this seemed like an okay value when training without including the validation set in the train set (in the only_known_research NB) but there is an overwhelming chance (supported by the train loss) that this might again be too low of a value for training on the entire train set of known whales.", "votes": 1 } ] }, { "id": 439220, "postDate": "2018-12-14T23:23:21.163Z", "content": "Thank you for posting your map5 code :)\nYeah, thanks again. This is the first \"kernel\" I've seen that follows my basic thinking, but implements it in a much more elegant way. My method for over sampling was really hamfisted, and I'm going to convert over to yours. Much more clever.
\n\nAs an aside, I came to the same conclusions that you did about augmentation methods -- i.e, flip, etc. I had been having shower thoughts about squish -- padding with zeroes doesn't work and reflection, etc obviously has some problems given the nature of the images. Thanks again, I've got some stuff to play with now this afternoon.
\n\nYou made me realize one big thing -- The over sampling methods I was using that resulted in more images being run through training were the ones that were doing the best for me (beating the no information, but not doing \"well\".) Your oversampling schema pushes even harder on that, and I'm a little embarrassed now that doubling down on that strategy didn't occur to me. It's almost like another hyper parameter for this comp I think.
", "rawMarkdown": "*Thank you* for posting your map5 code :)\nYeah, thanks again. This is the first \"kernel\" I've seen that follows my basic thinking, but implements it in a much more elegant way. My method for over sampling was *really* hamfisted, and I'm going to convert over to yours. Much more clever.\n\nAs an aside, I came to the same conclusions that you did about augmentation methods -- i.e, flip, etc. I had been having shower thoughts about squish -- padding with zeroes doesn't work and reflection, etc obviously has some problems given the nature of the images. Thanks again, I've got some stuff to play with now this afternoon.\n\nYou made me realize one big thing -- The over sampling methods I was using that resulted in *more* images being run through training were the ones that were doing the best for me (beating the no information, but not doing \"well\".) Your oversampling schema pushes even harder on that, and I'm a little embarrassed now that doubling down on that strategy didn't occur to me. It's almost like another hyper parameter for this comp I think.\n" }, { "id": 439195, "postDate": "2018-12-14T21:46:01.520Z", "content": "Really excited to read this!
\n\nBased on my experimentation so far, sampling method makes a huge difference, and binary for new/not new was my big idea to do this weekend.
\n\nThanks for sharing!
", "rawMarkdown": "Really excited to read this!\n\nBased on my experimentation so far, sampling method makes a huge difference, and binary for new/not new was my big idea to do this weekend.\n\nThanks for sharing!", "replies": [ { "id": 446997, "postDate": "2018-12-29T01:09:27.157Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 469102, "postDate": "2019-02-10T13:50:47.137Z", "rawMarkdown": "", "isDeleted": true }, { "id": 463283, "postDate": "2019-01-29T18:29:44.773Z", "content": "Is this based on V3 of the course?
", "rawMarkdown": "Is this based on V3 of the course?", "isDeleted": true, "replies": [ { "id": 463312, "postDate": "2019-01-29T19:37:22.843Z", "content": "It uses v1 of the library, which was introduced in v3 of the course
", "rawMarkdown": "It uses v1 of the library, which was introduced in v3 of the course" } ] }, { "id": 447630, "postDate": "2018-12-30T07:42:41.587Z", "rawMarkdown": "", "isDeleted": true }, { "id": 446857, "postDate": "2018-12-28T19:37:46.767Z", "rawMarkdown": "", "isDeleted": true }, { "id": 442716, "postDate": "2018-12-20T11:27:44.867Z", "content": "Thank you for your beautiful kernel!
", "rawMarkdown": "Thank you for your beautiful kernel!", "votes": 1 }, { "id": 438800, "postDate": "2018-12-14T07:54:48.663Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!", "votes": 1 } ], "comments": [ { "id": 439763, "author_name": "", "author_url": "", "post_date": "2018-12-16T09:48:24.827000", "content": "hi Radek, how did you address this issue: \"OSError: [Errno 30] Read-only file system: '../input/models'\", during execution of \"learn = create_cnn(data, models.resnet50, metrics=[accuracy])\"? I tried to load this one before executing your script in kaggle kernel: !ln -s /kaggle/input/pytorch-pretrained-models/ /opt/conda/lib/python3.6/site-packages/fastai/weights but did not have success? Do you have any idea on how to solve this issue? Thanks!
", "votes": 12, "replies": [ { "id": 439811, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-16T12:24:51.327000", "content": "Sorry, I don't. I am not familiar with Kaggle kernels.
", "votes": 0, "replies": [] }, { "id": 441152, "author_name": "Bhuvana Kundumani", "author_url": "", "post_date": "2018-12-18T10:32:26.337000", "content": "(Edited)Hi,\nBased on radek's starter pack, I have created a Kaggle kernel for this competition. Github-https://github.com/bhuvanakundumani/hump_back_whale_Kaggle_Kernel
\n\nI am getting an error while i run learn = create_cnn(data, models.resnet34, metrics=[accuracy, map5])\nThe error :AttributeError: 'NoneType' object has no attribute 'detach'
", "votes": -1, "replies": [] }, { "id": 441774, "author_name": "dromosys", "author_url": "", "post_date": "2018-12-19T03:49:12.607000", "content": "you need to pass in model_dir=MODEL_PATH where MODEL_PATH = \"/tmp/model/\"
", "votes": 0, "replies": [] }, { "id": 442909, "author_name": "Bhuvana Kundumani", "author_url": "", "post_date": "2018-12-20T17:14:15.123000", "content": "Thanks.
", "votes": 0, "replies": [] } ] }, { "id": 481638, "author_name": "Carl Philipp Hoppe", "author_url": "", "post_date": "2019-03-01T16:42:24.807000", "content": "Hi Radek, I just wanted to thank you for the resources! Having just started the fastai course it was a great help to gain some more understanding.
\n\nI was wondering what kind of machine you are running your training on. I was using a P2.xlarge SageMaker Instance and I had significantly longer runtimes than you, but wasn't quite willing to scale up.
", "votes": 3, "replies": [ { "id": 548374, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-06-09T09:40:07.887000", "content": "Sorry for the late reply - for the training I used a rig that I built with a ryzen 5, 16 gb of RAM and a single 1080ti. p2.xlarge has an older gen GPU so slower train time is expected - you would probably get comparable train times (probably even slightly faster) on a p3.2xlarge
", "votes": 0, "replies": [] } ] }, { "id": 454951, "author_name": "Cozy Doomer", "author_url": "", "post_date": "2019-01-12T15:48:54.360000", "content": "Hey Radek, this is a amazing resource, thank you for sharing!
\n\nIt also cleared up a big confusion I had about the fastai library and training speed:
\n\nIn my recent notebooks fit_one_cycle and lr_find got stuck for about a second every 8 batches (num_workers=8, high batch_size)
\n\nAfter resizing the Images to the correct size beforehand this behaviour stopped so I'm pretty sure the internal resizing process slowed down my training!
", "votes": 3, "replies": [ { "id": 455011, "author_name": "William Horton", "author_url": "", "post_date": "2019-01-12T18:41:25.997000", "content": "Thank you for sharing this! I’ve been seeing the same problem in another competition and hadn’t figured it out yet.
", "votes": 1, "replies": [] }, { "id": 455211, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-01-13T08:52:59.597000", "content": "I always train with nvidia-smi -l and htop running, this helps me diagnose a lot of problems like this. For resizing for instance, I will see the cores maxed out while at the same time the GPU not being fed quickly enough.
\n\nI use tmux and am making an effort to start using tmuxinator for more projects. It is a really cool little program where with just a single command it brings up as many tmux panes as you'd like and starts the programs that you'd like. My pane #1 is nvidia-smi, #2 is htop and #3 is the fastai directory of the fastai library, likely with VIM open. This allows me to get some consistency regardless what I work on.
", "votes": 2, "replies": [] }, { "id": 461405, "author_name": "Haider Alwasiti", "author_url": "", "post_date": "2019-01-25T23:32:26.597000", "content": "I add a 4th pane: sudo iotop\nObserving the disk activity
", "votes": 0, "replies": [] } ] }, { "id": 559465, "author_name": "AaronZhangTD", "author_url": "", "post_date": "2019-06-24T06:43:17.600000", "content": "Thanks for sharing! always been looking for fastai-based project like this
", "votes": 1, "replies": [] }, { "id": 464615, "author_name": "Stas Bekman", "author_url": "", "post_date": "2019-02-01T06:53:01.240000", "content": "Thanks for sharing your code, @radek. I'm trying to run it first.
\n\nI have a feeling the recently added annotations.json hasn't been tested with fluke_detection.ipynb. Not sure why it's expecting the fluke class entry to be at first position, but it's not always the case, so collate fn fails. Here is a dump of one such entry where it's second:
{\"annotations\": [\n\n{\"class\": \"left\",\n \"type\": \"point\",\n \"x\": 0.7092198581560337,\n \"y\": 134.7517730496464},\n\n {\"class\": \"fluke\",\n \"height\": 253.9007092198601,\n \"type\": \"rect\",\n \"width\": 1039.0070921985894,\n \"x\": 0.7092198581560337,\n \"y\": 77.30496453900767},\n\n {\"class\": \"notch\",\n \"type\": \"point\",\n \"x\": 496.878528952789,\n \"y\": 261.35895342188473},\n\n {\"class\": \"right\",\n \"type\": \"point\",\n \"x\": 1036.170212766017,\n \"y\": 81.56028368794794}\n\n],\n \"class\": \"image\",\n \"filename\": \"0bb469e7d.jpg\"}\nIt's probably a bad idea to rely on a fixed position in a json file.
\n\nI applied a quick fix to match the entry that has 'width', there is probably a more elegant solution to choose the class fluke. This is just the beginning of the function:
\n\ndef j2anno(j):\n # bbox coordinates are returned in pascal voc format [x_min, y_min, x_max, y_max]\n im = cv2.imread(f\"data/train-{SZ}/{j['filename']}\", cv2.IMREAD_COLOR) \n im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n im_height, im_width, _ = im.shape\n\n orig_im = cv2.imread(f\"data/train/{j['filename']}\", cv2.IMREAD_COLOR)\n orig_im_height, orig_im_width, _ = orig_im.shape\n\n# bbox_info = j['annotations'][0]\n bbox_info = None \n for i in j['annotations']:\n if 'width' in i:\n bbox_info = i\n break\nThanks.
", "votes": 1, "replies": [ { "id": 465519, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-02-03T10:43:02.403000", "content": "Thank you for the heads up on this Stas, good points. I now updated the notebook and pushed the fixed version.
", "votes": 1, "replies": [] }, { "id": 465701, "author_name": "Stas Bekman", "author_url": "", "post_date": "2019-02-03T20:07:28.607000", "content": "just don't forget to push!
", "votes": 1, "replies": [] }, { "id": 465924, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-02-04T10:35:30.240000", "content": "thank you Stas :) I pushed but to Bitbucket by accident - now everything should be in place on github as well
", "votes": 0, "replies": [] } ] }, { "id": 459560, "author_name": "liuchang", "author_url": "", "post_date": "2019-01-22T01:54:15.383000", "content": "Thank you very much for sharing, but I can't find the boxes.json file in your github. Can you re-send it?
", "votes": 1, "replies": [ { "id": 459700, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-01-22T07:52:27.477000", "content": "Sorry about this - I renamed the file to annotations.json and must have not renamed it in code everywhere. Will correct shortly.
", "votes": 0, "replies": [] }, { "id": 459796, "author_name": "liuchang", "author_url": "", "post_date": "2019-01-22T11:37:51.427000", "content": "ohoh,thanks,and Is there any change from boxes.json to annotations.json?
", "votes": 0, "replies": [] }, { "id": 459803, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2019-01-22T11:48:34.373000", "content": "this should be fixed now - let me know if you have any issues
", "votes": 0, "replies": [] } ] }, { "id": 532431, "author_name": "Amit Vadnere", "author_url": "", "post_date": "2019-05-17T00:05:34.183000", "content": "Hi I am new to deep learning and have question specific to a file. I was analyzing the output of @radek1 for file only_known_train.ipynb which seems strange to me as per theoretical concept. I am talking about the training loss is higher than the validation loss. I understand the imbalance in the data set but after oversampling the scenario is same. I am sorry for the silly question.
", "votes": 2, "replies": [ { "id": 535784, "author_name": "Cozy Doomer", "author_url": "", "post_date": "2019-05-23T12:46:02.127000", "content": "I'm also no expert but I will try to explain:
\n\nYou can see @radek1 intentionally used a validation set with only one image:\nval_fns = {'69823499d.jpg'}
He explains his reasoning for this in the beginning:\n``` \nI take a curriculum approach to training here. I first expose the model to as many different images of whales as quickly as possible (no oversampling) and train on images resized to 224x224.
\n\nI would like the conv layers to start picking up on features useful for identifying whales. For that, I want to show the model as rich of a dataset as possible. \n```
\n\nMy explanation
\n\nThis is quite an odd thing to do but it makes more sense in the context of this competition.\nWe had 3,000+ classes with only a few images per class. \nPutting some images of each class from the training set in the validation set would lead to classes having almost no images to learn from.
\n\nUsing only 1 image in the validation set makes the validation loss an useless indicator because it only shows if this 1 image is predicted correctly.
\n\nSo the payoff you get is you train on all images and don't loose valuable data for validation, but you don't know how good the model predicts on images that are not used to change weights (it does not know yet).\nThis makes it very hard (impossible?) to know when the model is overfitting.
", "votes": 2, "replies": [] }, { "id": 536035, "author_name": "Amit Vadnere", "author_url": "", "post_date": "2019-05-23T21:09:38.897000", "content": "Thank You. I got the point now.
", "votes": 0, "replies": [] } ] }, { "id": 446590, "author_name": "Haider Alwasiti", "author_url": "", "post_date": "2018-12-28T10:27:17.217000", "content": "Jeremy mentioned that their Dawnbench implementation of image size is better than the current one. It can use rectangular images without changing the aspect ratio (no squash and not even crop). Unfortunately it is not implemented yet, and will be implemented in fastai 2nd course in March.
\n\nI suspect this would be important for a dataset like this, where image sizes are not consistent resulting in different amount of squashing the image.
\n\nMore details here and Jeremy's replies :\nhttps://forums.fast.ai/t/best-way-to-resize-pictures-for-model-training/28307/27?u=hwasiti
", "votes": 1, "replies": [ { "id": 448483, "author_name": "Max Metz", "author_url": "", "post_date": "2019-01-01T10:06:37.263000", "content": "I think this approach sounds very interesting. I'm rather new in the field and would like to check out the details on fast.ai. I just registered, but apparently I do not have access to the topic. Is it supposed to be hidden?
", "votes": 0, "replies": [] }, { "id": 461406, "author_name": "Haider Alwasiti", "author_url": "", "post_date": "2019-01-25T23:33:42.197000", "content": "Today it has been released for public
", "votes": 1, "replies": [] } ] }, { "id": 441246, "author_name": "Tommy Jiang", "author_url": "", "post_date": "2018-12-18T13:20:28.247000", "content": "Thanks for the kernel! You can try using bbox of the image. Details can be found at https://www.kaggle.com/suicaokhoailang/generating-whale-bounding-boxes.
", "votes": 1, "replies": [] }, { "id": 439869, "author_name": "gakki", "author_url": "", "post_date": "2018-12-16T14:45:57.820000", "content": "Sorry, how to solve this problem that \n'Exception: Your validation data contains a label that isn't present in the training set, please fix your data.'? \nAnd only appear in when I use this\ndata = (ImageItemList. ... .random_split_by_pct(seed=0)) \nand it tell me about \nKeyError: 'w_d8a08f8'\nSorry, I am not familiar with fastai.\nThis problem occurred when I was researching this code and I wanted to know why. Any help would be useful. Thank you!
", "votes": 1, "replies": [ { "id": 440016, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-16T21:42:25.440000", "content": "I am not sure why people are getting this error. I will try to find time tomorrow to change the code so that this no longer will be an issue.
", "votes": 1, "replies": [] }, { "id": 440624, "author_name": "Sam Ariabod", "author_url": "", "post_date": "2018-12-17T19:55:25.123000", "content": "@gakki Hello, I have not gone through the code, but I would run into this issue when doing a train/valid split at random. You end up having labels in your validation set that do not exist in the training set and the dataloader would fail. To combat this, I do my train/valid splits at the label level. This ensures the error does not occur and also keeps my validation set balanced with the training data. Below is a quick example:\nhttps://gist.github.com/sariabod/56aeea96abe58671375eb52a883e356e
", "votes": 5, "replies": [] }, { "id": 441018, "author_name": "gakki", "author_url": "", "post_date": "2018-12-18T07:27:32.080000", "content": "Thank you, I think I almost know why, That's very useful.
", "votes": 0, "replies": [] }, { "id": 441049, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-18T08:04:17.413000", "content": "I am experiencing some hw issues, hence making the changes is taking longer than I anticipated. One solution to this would be to move how the validation set is constructed in later notebooks over to the first_submission one.
", "votes": 0, "replies": [] }, { "id": 441957, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-19T09:40:00.197000", "content": "I decided to not change the code so that in first_submission.ipynb we would sample the validation set like we do in subsequent notebooks.
\n\nI do not want to make the code of the first_submission notebook unnecessarily complex. In general, I go by the philosophy that the first submission should be as simple as it can be. In fact, my first submission to this competition was predicting only the new_whale class. This way I was able to tell quite a few of the things I had in place worked correctly. It also allowed me to learn something about the LB test set that was quite useful for the construction of the validation set.
If you encounter an error while running the first_submission notebook, fixing it could be a fun exercise (moving the code for creating the validation set over from one of the later notebooks). In either case, you should be okay to skip this notebook and still run any of the later notebooks just fine, including the one that achieves 0.760 on the LB (only_known_train.ipynb)
@gakki Hello! What did you do to get rid of that error?
", "votes": 0, "replies": [] }, { "id": 442826, "author_name": "gakki", "author_url": "", "post_date": "2018-12-20T15:22:15.993000", "content": "@Ankesh Sorry, I don't think I solved this problem very well, I just use some of the more categorical categories (like w_23a388d) as the validation set.
", "votes": 0, "replies": [] } ] }, { "id": 438920, "author_name": "DevilEars", "author_url": "", "post_date": "2018-12-14T12:18:01.603000", "content": "Thank you for sharing this! The Kernel notebooks are somehow confusing to me, but your github repository is far easier to navigate.
", "votes": 1, "replies": [] }, { "id": 480144, "author_name": "Stéphane Couvreur", "author_url": "", "post_date": "2019-02-27T20:36:27.977000", "content": "Great kernel Radek ! The map5 implementation in the fastai learner class I found particularly useful
", "votes": 2, "replies": [] }, { "id": 439266, "author_name": "toshi_k", "author_url": "", "post_date": "2018-12-15T03:50:14.190000", "content": "Good work ! How about sharing your code on Kaggle-Kernel to ensure reproducibility ?
", "votes": 2, "replies": [ { "id": 439578, "author_name": "William Horton", "author_url": "", "post_date": "2018-12-15T20:50:25.183000", "content": "This should be possible now because Kernels upgraded to fastai v1.
", "votes": 2, "replies": [] }, { "id": 440408, "author_name": "Nazim Girach", "author_url": "", "post_date": "2018-12-17T14:11:54.487000", "content": "I tried running the code on Kaggla Kernels, but got some errors. Did anyone try it?
", "votes": 2, "replies": [] }, { "id": 444939, "author_name": "", "author_url": "", "post_date": "2018-12-25T07:24:59.580000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 468284, "author_name": "IsaacKamlish", "author_url": "", "post_date": "2019-02-08T16:10:05.397000", "content": "Have people tried using these classification notebooks with different models bar Resnet 50, if so what LR's did you use and which model?
", "votes": 0, "replies": [] }, { "id": 453824, "author_name": "Zak Raicik", "author_url": "", "post_date": "2019-01-10T21:01:12.083000", "content": "If I mimic the code in 'only_known_train', I get much worse performance. Does anyone else have this issue?
", "votes": 0, "replies": [] }, { "id": 448821, "author_name": "Sandeep Gupta", "author_url": "", "post_date": "2019-01-02T07:00:18.543000", "content": "Hi Radek,
\n\nThanks a lot for the starter pack. I am a starter into Computer Vision, had some queries from your code. Request you to please help me with it.
\n\n1) preds[:, 5004] = 0.06 \nWhy did you use the above code ? Didn't really get it
\n\n2) classes = learn.data.classes + ['new_whale']
\n\nHow exactly it updates the new_whale for the top5 predictions, even when the model doesn't has \"new_whale\".
\n\nSorry if the questions are stupid.
", "votes": 0, "replies": [ { "id": 448842, "author_name": "Hector", "author_url": "", "post_date": "2019-01-02T07:59:17.280000", "content": "Thanks Raghava!!\nCan you please elaborate how to get the threshold for i.e. 0.06. Didn't really get what is \"tested using ps\"
", "votes": 0, "replies": [] }, { "id": 448861, "author_name": "Hector", "author_url": "", "post_date": "2019-01-02T09:01:19.477000", "content": "Refer to https://github.com/radekosmulski/whale/blob/master/only_known_research.ipynb notebook. Particularly this block of code
\n\nps = np.linspace(0, 1, 51)\nfor p in ps:\n preds[:, 5004] = p\n res.append(map5(preds, targs).item())\nbest_p = ps[np.argmax(res)]; best_p
", "votes": 1, "replies": [] }, { "id": 448868, "author_name": "Sandeep Gupta", "author_url": "", "post_date": "2019-01-02T09:21:45.823000", "content": "Thanks a ton Raghava !!!!
", "votes": 0, "replies": [] }, { "id": 453832, "author_name": "old-ufo", "author_url": "", "post_date": "2019-01-10T21:30:05.497000", "content": "I would recommend to use \nps = np.linspace(0, 1, 201)\nYes, it is slower, but gives a bit more finegrained threshold
", "votes": 0, "replies": [] } ] }, { "id": 446664, "author_name": "Hector", "author_url": "", "post_date": "2018-12-28T13:01:33.240000", "content": "@radek Thanks for the kernel. That was very helpful. I see that it was taking about more than an hour to run each experiment(training both head and base). Do u think the idea of carrying experiments on a much small subset of train data to do multiple quick iterations and picking the good ones to run on the whole data is a good idea in case of DL(this was much talked by jeremy in ML class)? And r u currently runnin ur experiments on the entire data?
", "votes": 0, "replies": [ { "id": 447623, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-30T07:28:24.577000", "content": "I have not done a whole lot of running small experiments on this dataset. Sometimes training for a few epochs will suffice to say if a change is in the right direction or not, but for most hyperparameters one would need to run the full training (and even that might not give a conclusive answer).
\n\nIt is not easy to figure out what to train on and how to figure out if one is moving in the right direction or not. The lr finder gives some indication, the shape of the loss plot is also telling. This workflow of what works is something one needs to work out for themselves with time I am afraid (probably a big impact will also come from what resources one has available and how much time one has to give to this - since I can work on this only sporadically, this leaves large amount of time in between where I can run a longer training if I wanted to, though have not gotten to that point in this competition just yet).
", "votes": 0, "replies": [] } ] }, { "id": 443829, "author_name": "JayNP", "author_url": "", "post_date": "2018-12-22T14:14:52.660000", "content": "Very good kernel. It looks a very standard way to implement cnn using fastai 1.0. I am trying to figure out why it performed about 0.1 better than other previous implementation in the forum. What is the secret sauce here? Is it the oversample method? Or the map5 metric? Or anything hidden in the details?
", "votes": 0, "replies": [ { "id": 443929, "author_name": "larcat", "author_url": "", "post_date": "2018-12-22T18:44:11.070000", "content": "Map 5 is just being reported here -- it's telemetry, not feedback going back in to the model.
", "votes": 0, "replies": [] }, { "id": 443981, "author_name": "old-ufo", "author_url": "", "post_date": "2018-12-22T20:37:26.513000", "content": "My bet is one cycle policy + good default augmentation set
", "votes": 2, "replies": [] }, { "id": 463709, "author_name": "NullPoint", "author_url": "", "post_date": "2019-01-30T13:16:30.713000", "content": "Hello, could i ask you how are you get above 0.9 ?
", "votes": 0, "replies": [] } ] }, { "id": 443786, "author_name": "JayNP", "author_url": "", "post_date": "2018-12-22T12:26:37.163000", "content": "It used fastai 1.0. Does 1.0 perform better than 0.7?
", "votes": 0, "replies": [ { "id": 443801, "author_name": "William Horton", "author_url": "", "post_date": "2018-12-22T13:11:46.757000", "content": "1.0
", "votes": 0, "replies": [] } ] }, { "id": 442058, "author_name": "larcat", "author_url": "", "post_date": "2018-12-19T12:25:37.520000", "content": "Hi Radek, thank you again for the Kernel. I've trained the first two stages using your methodology for validation set creation a couple different ways (with upsampling, without, at 224, at 392, etc) and gotten decent results.
\n\nHowever, whenever I start progressive resizing, my model seems to fall apart, resulting in things like this (image size 448, 16 batch size):
\n\n
Here is the training on the second stage of training the model (224 image size, 64 batch size):
\n\n
Do you have any suggestions? Thank you for the time.
", "votes": 0, "replies": [ { "id": 442108, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-19T13:49:56.253000", "content": "If I am reading this right (at the top you are continuing to train the model you trained at the bottom but with a larger image size) you might be using too high of an LR.
\n\nMight want to give the following lrs a try and see what happens:
\n\nmax_lr = 1e-3 / 4\nlrs = [max_lr/100, max_lr/10, max_lr]\nRadek, thanks for the reply. You were reading it right.
\n\nThat's what I was attempting with the 1e-06, 1e-05, 1e-04 vs. 2e-03.
\n\nI'll try cranking them down even further. Appreciate the insight!
", "votes": 1, "replies": [] }, { "id": 442314, "author_name": "larcat", "author_url": "", "post_date": "2018-12-19T19:34:02.227000", "content": "The push on LR seems to have gotten me past that -- I was trying to use the learning rate vs. loss functions in fastai. I replaced with yours and that seems to have me improving from larger images.
\n\nDo you find your initial rates experimentally, or do you always divide your prior LRs by the scaling factor of the BS necessitated by the larger images?
\n\nIf you had a Titan with 24 gig of ram or whatever, would you still be training at these divided-by-four-rates?
\n\nThanks again for the time -- trying to use this competition to learn.
", "votes": 1, "replies": [] }, { "id": 442351, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-19T21:11:44.523000", "content": "There are no answers set in stone here. A lot of this comes through experience and having spent a lot of time reading / implementing papers. The experience does not necessarily come through doing complex or fancy things - mostly coming up with a small project I could complete in a couple of days. Fastai lesson notebooks (and applying the techniques there to other datasets) is what would constitute the core of how I learn.
\n\nAnyhow, yes, one can use this rule of a thumb that if you decrease the batch size by 4, you might want to decrease your LR by a factor of 4 as well. Adam, the optimizer we are using here, seems to be quite sensitive to parameter values, but for instance using SGD there have been instances where I would cut down the BS by half and not reduce the LR at all. That's where gradient clipping comes in handy.
\n\nA large dataset like this one makes it a real pain to experiment and start getting an intuition on what could be going on (an learning to read the signs whether you are overfitting / underfitting, etc). It's much easier to pick this up using something like cifar10. Here is a repo I put together for training on cifar10 using docker and the previous version of fastai and here are some of the small projects I mentioned above.
\n\nBut anyhow, I bet nothing beats working through the course notebooks. And for nearly anything you can throw at the GPU, the defaults fastai v1 comes with are really, really good.
", "votes": 3, "replies": [] }, { "id": 443329, "author_name": "larcat", "author_url": "", "post_date": "2018-12-21T12:51:52.973000", "content": "Hi @radek1,
\n\nThere's a paper from 2018 that directly supports your approach to tweaking LRs when decreasing/increasing batchsize:
\n\nhttps://arxiv.org/abs/1706.02677
\n\nThanks again for sharing your approaches in this challenge.
\n\n-L
", "votes": 2, "replies": [] } ] }, { "id": 439281, "author_name": "larcat", "author_url": "", "post_date": "2018-12-15T04:47:57.063000", "content": "Hey, handful of questions if you've got the time:
\n\n1) Impact of lin_ftrs -- Can you talk about that?\n2) Impact of clip_grad().\n3) Totally niave but can you explain what's happening in:\nfn2label = {row[1].Image: row[1].Id for row in df.iterrows()}\nWhen doing the labeling? I can't for the life of me figure out how this works.\nThanks, and sorry for the silly questions.
fn2label is a dict mapping image name to label. In a fastai data block you can label your data from a function .label_from_func(lambda path: fn2label[path2fn(path)]) given a path. He uses the data block to create the DataBunch object. Read more about the data block here: https://docs.fast.ai/data_block.html
1) Increasing lin_ftrs makes the head bigger, increases the capacity of the model. Given the way I chose to train, this particular value gave me a better result on my validation set than the default value of 512.\n2) I assume it to be helpful when training with high learning rates.
", "votes": 0, "replies": [] }, { "id": 442106, "author_name": "larcat", "author_url": "", "post_date": "2018-12-19T13:48:15.063000", "content": "Thanks for the explanation on lin_ftrs -- Read up a bit in various places and I think it makes more sense. This is most due to image size and number of classes I believe?
", "votes": 0, "replies": [] }, { "id": 442143, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2018-12-19T14:30:54.740000", "content": "Yes, that could be the case but I am not sure one can get a lot of mileage out of trying to figure out why a more complex model is required, or guess the complexity of model required up front. What if we halved the number of classes? Would this make much of a difference? I am not really sure one is able to answer this question without running experiments.
\n\nHere, given the specific architecture and this particular dataset, seemed we were underfitting with the default value of 512 and a bigger number of trainable weights in the classifier seemed like something worth exploring.
\n\nBased on experiments, this seemed like an okay value when training without including the validation set in the train set (in the only_known_research NB) but there is an overwhelming chance (supported by the train loss) that this might again be too low of a value for training on the entire train set of known whales.
", "votes": 1, "replies": [] } ] }, { "id": 439220, "author_name": "larcat", "author_url": "", "post_date": "2018-12-14T23:23:21.163000", "content": "Thank you for posting your map5 code :)\nYeah, thanks again. This is the first \"kernel\" I've seen that follows my basic thinking, but implements it in a much more elegant way. My method for over sampling was really hamfisted, and I'm going to convert over to yours. Much more clever.
\n\nAs an aside, I came to the same conclusions that you did about augmentation methods -- i.e, flip, etc. I had been having shower thoughts about squish -- padding with zeroes doesn't work and reflection, etc obviously has some problems given the nature of the images. Thanks again, I've got some stuff to play with now this afternoon.
\n\nYou made me realize one big thing -- The over sampling methods I was using that resulted in more images being run through training were the ones that were doing the best for me (beating the no information, but not doing \"well\".) Your oversampling schema pushes even harder on that, and I'm a little embarrassed now that doubling down on that strategy didn't occur to me. It's almost like another hyper parameter for this comp I think.
", "votes": 0, "replies": [] }, { "id": 439195, "author_name": "larcat", "author_url": "", "post_date": "2018-12-14T21:46:01.520000", "content": "Really excited to read this!
\n\nBased on my experimentation so far, sampling method makes a huge difference, and binary for new/not new was my big idea to do this weekend.
\n\nThanks for sharing!
", "votes": 0, "replies": [ { "id": 446997, "author_name": "", "author_url": "", "post_date": "2018-12-29T01:09:27.157000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 469102, "author_name": "", "author_url": "", "post_date": "2019-02-10T13:50:47.137000", "content": "", "votes": 0, "replies": [] }, { "id": 463283, "author_name": "", "author_url": "", "post_date": "2019-01-29T18:29:44.773000", "content": "Is this based on V3 of the course?
", "votes": 0, "replies": [ { "id": 463312, "author_name": "William Horton", "author_url": "", "post_date": "2019-01-29T19:37:22.843000", "content": "It uses v1 of the library, which was introduced in v3 of the course
", "votes": 0, "replies": [] } ] }, { "id": 447630, "author_name": "", "author_url": "", "post_date": "2018-12-30T07:42:41.587000", "content": "", "votes": 0, "replies": [] }, { "id": 446857, "author_name": "", "author_url": "", "post_date": "2018-12-28T19:37:46.767000", "content": "", "votes": 0, "replies": [] }, { "id": 442716, "author_name": "XII", "author_url": "", "post_date": "2018-12-20T11:27:44.867000", "content": "Thank you for your beautiful kernel!
", "votes": 1, "replies": [] }, { "id": 438800, "author_name": "Tommy Jiang", "author_url": "", "post_date": "2018-12-14T07:54:48.663000", "content": "Thanks for sharing!
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "438771": "I share the code on github [here][1].\n\nThe approach is very basic but is based on sound methodology. In some sense, this made working on this a pleasure.\n\nIt is a resnet50 model trained in a way to address specifics of the dataset. Local CV tracks LB score closely.\n\nNow I realize this places high on the LB at the moment. But in reality this is just a small step above where the blenders are. Also, the competition just launched which also needs to be taken into consideration. \n\nI chose to share this because I think it can be a very good learning resource and a good way for people to find their bearings. To some extent, this demonstrates a canonical way of solving a deep learning classification problem using modern tools (this does not imply that phrasing this competition as a classification problem is the best way to address it, though it would be interesting to see how far one can take this approach).\n\n\n [1]: https://github.com/radekosmulski/whale", "439763": "hi Radek, how did you address this issue: \"OSError: [Errno 30] Read-only file system: '../input/models'\", during execution of \"learn = create_cnn(data, models.resnet50, metrics=[accuracy])\"? I tried to load this one before executing your script in kaggle kernel: !ln -s /kaggle/input/pytorch-pretrained-models/ /opt/conda/lib/python3.6/site-packages/fastai/weights but did not have success? Do you have any idea on how to solve this issue? Thanks!", "481638": "Hi Radek, I just wanted to thank you for the resources! Having just started the fastai course it was a great help to gain some more understanding.\n \nI was wondering what kind of machine you are running your training on. I was using a P2.xlarge SageMaker Instance and I had significantly longer runtimes than you, but wasn't quite willing to scale up.", "454951": "Hey Radek, this is a amazing resource, thank you for sharing!\n\n\nIt also cleared up a big confusion I had about the fastai library and training speed:\n\nIn my recent notebooks fit\\_one\\_cycle and lr\\_find got stuck for about a second every 8 batches (num_workers=8, high batch\\_size)\n\nAfter resizing the Images to the correct size beforehand this behaviour stopped so I'm pretty sure the internal resizing process slowed down my training!", "559465": "Thanks for sharing! always been looking for fastai-based project like this", "464615": "Thanks for sharing your code, @radek. I'm trying to run it first.\n\nI have a feeling the recently added `annotations.json` hasn't been tested with `fluke_detection.ipynb`. Not sure why it's expecting the fluke class entry to be at first position, but it's not always the case, so collate fn fails. Here is a dump of one such entry where it's second:\n\n {\"annotations\": [\n \n {\"class\": \"left\",\n \"type\": \"point\",\n \"x\": 0.7092198581560337,\n \"y\": 134.7517730496464},\n \n {\"class\": \"fluke\",\n \"height\": 253.9007092198601,\n \"type\": \"rect\",\n \"width\": 1039.0070921985894,\n \"x\": 0.7092198581560337,\n \"y\": 77.30496453900767},\n \n {\"class\": \"notch\",\n \"type\": \"point\",\n \"x\": 496.878528952789,\n \"y\": 261.35895342188473},\n \n {\"class\": \"right\",\n \"type\": \"point\",\n \"x\": 1036.170212766017,\n \"y\": 81.56028368794794}\n \n ],\n \"class\": \"image\",\n \"filename\": \"0bb469e7d.jpg\"}\n\nIt's probably a bad idea to rely on a fixed position in a json file.\n\nI applied a quick fix to match the entry that has 'width', there is probably a more elegant solution to choose the class fluke. This is just the beginning of the function:\n\n\n def j2anno(j):\n # bbox coordinates are returned in pascal voc format [x_min, y_min, x_max, y_max]\n im = cv2.imread(f\"data/train-{SZ}/{j['filename']}\", cv2.IMREAD_COLOR) \n im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)\n im_height, im_width, _ = im.shape\n \n orig_im = cv2.imread(f\"data/train/{j['filename']}\", cv2.IMREAD_COLOR)\n orig_im_height, orig_im_width, _ = orig_im.shape\n \n # bbox_info = j['annotations'][0]\n bbox_info = None \n for i in j['annotations']:\n if 'width' in i:\n bbox_info = i\n break\n\nThanks.", "459560": "\nThank you very much for sharing, but I can't find the boxes.json file in your github. Can you re-send it?\n", "532431": "Hi I am new to deep learning and have question specific to a file. I was analyzing the output of @radek1 for file only_known_train.ipynb which seems strange to me as per theoretical concept. I am talking about the training loss is higher than the validation loss. I understand the imbalance in the data set but after oversampling the scenario is same. I am sorry for the silly question.", "446590": "Jeremy mentioned that their Dawnbench implementation of image size is better than the current one. It can use rectangular images without changing the aspect ratio (no squash and not even crop). Unfortunately it is not implemented yet, and will be implemented in fastai 2nd course in March.\n\nI suspect this would be important for a dataset like this, where image sizes are not consistent resulting in different amount of squashing the image. \n\nMore details here and Jeremy's replies :\nhttps://forums.fast.ai/t/best-way-to-resize-pictures-for-model-training/28307/27?u=hwasiti", "441246": "Thanks for the kernel! You can try using bbox of the image. Details can be found at https://www.kaggle.com/suicaokhoailang/generating-whale-bounding-boxes.", "439869": "Sorry, how to solve this problem that \n'Exception: Your validation data contains a label that isn't present in the training set, please fix your data.'? \nAnd only appear in when I use this\ndata = (ImageItemList. ... .random_split_by_pct(seed=0)) \nand it tell me about \nKeyError: 'w_d8a08f8'\nSorry, I am not familiar with fastai.\nThis problem occurred when I was researching this code and I wanted to know why. Any help would be useful. Thank you!", "438920": "Thank you for sharing this! The Kernel notebooks are somehow confusing to me, but your github repository is far easier to navigate.", "480144": "Great kernel Radek ! The map5 implementation in the fastai learner class I found particularly useful", "439266": "Good work ! How about sharing your code on Kaggle-Kernel to ensure reproducibility ?", "468284": "Have people tried using these classification notebooks with different models bar Resnet 50, if so what LR's did you use and which model?", "453824": "If I mimic the code in 'only_known_train', I get much worse performance. Does anyone else have this issue?", "448821": "Hi Radek,\n\nThanks a lot for the starter pack. I am a starter into Computer Vision, had some queries from your code. Request you to please help me with it.\n\n1) preds[:, 5004] = 0.06 \nWhy did you use the above code ? Didn't really get it\n\n2) classes = learn.data.classes + ['new_whale']\n\nHow exactly it updates the new_whale for the top5 predictions, even when the model doesn't has \"new_whale\".\n\nSorry if the questions are stupid. ", "446664": "@radek Thanks for the kernel. That was very helpful. I see that it was taking about more than an hour to run each experiment(training both head and base). Do u think the idea of carrying experiments on a much small subset of train data to do multiple quick iterations and picking the good ones to run on the whole data is a good idea in case of DL(this was much talked by jeremy in ML class)? And r u currently runnin ur experiments on the entire data?", "443829": "Very good kernel. It looks a very standard way to implement cnn using fastai 1.0. I am trying to figure out why it performed about 0.1 better than other previous implementation in the forum. What is the secret sauce here? Is it the oversample method? Or the map5 metric? Or anything hidden in the details? ", "443786": "It used fastai 1.0. Does 1.0 perform better than 0.7? ", "442058": "Hi Radek, thank you again for the Kernel. I've trained the first two stages using your methodology for validation set creation a couple different ways (with upsampling, without, at 224, at 392, etc) and gotten decent results.\n\nHowever, whenever I start progressive resizing, my model seems to fall apart, resulting in things like this (image size 448, 16 batch size):\n\n![Model death][1]\n\nHere is the training on the second stage of training the model (224 image size, 64 batch size):\n\n![Prior to death][2]\n\nDo you have any suggestions? Thank you for the time.\n\n\n [1]: http://i.imgur.com/kQFtX6s.jpg\n [2]: https://i.imgur.com/gMlTKoS.jpg", "439281": "Hey, handful of questions if you've got the time:\n\n1) Impact of lin_ftrs -- Can you talk about that?\n2) Impact of clip_grad().\n3) Totally niave but can you explain what's happening in:\n`fn2label = {row[1].Image: row[1].Id for row in df.iterrows()}`\nWhen doing the labeling? I can't for the life of me figure out how this works.\nThanks, and sorry for the silly questions.", "439220": "*Thank you* for posting your map5 code :)\nYeah, thanks again. This is the first \"kernel\" I've seen that follows my basic thinking, but implements it in a much more elegant way. My method for over sampling was *really* hamfisted, and I'm going to convert over to yours. Much more clever.\n\nAs an aside, I came to the same conclusions that you did about augmentation methods -- i.e, flip, etc. I had been having shower thoughts about squish -- padding with zeroes doesn't work and reflection, etc obviously has some problems given the nature of the images. Thanks again, I've got some stuff to play with now this afternoon.\n\nYou made me realize one big thing -- The over sampling methods I was using that resulted in *more* images being run through training were the ones that were doing the best for me (beating the no information, but not doing \"well\".) Your oversampling schema pushes even harder on that, and I'm a little embarrassed now that doubling down on that strategy didn't occur to me. It's almost like another hyper parameter for this comp I think.\n", "439195": "Really excited to read this!\n\nBased on my experimentation so far, sampling method makes a huge difference, and binary for new/not new was my big idea to do this weekend.\n\nThanks for sharing!", "469102": "", "463283": "Is this based on V3 of the course?", "447630": "", "446857": "", "442716": "Thank you for your beautiful kernel!", "438800": "Thanks for sharing!" } }