This was our very first project in computer vision and in machine learning in general. We didn't manage to reach the top 10% (as we hoped), but we learned a lot from this contest. Now we would like to share our experience with everyone. Hopefully this will be helpful for the beginners.
\n\nAlso we would be very grateful if you point out more mistakes we have made :)
\n\nhttp://yerevann.github.io/2015/08/17/diabetic-retinopathy-detection-contest-what-we-did-wrong/
\n\nThank you all for the great contest.
", "messages": [ { "id": "89534", "postDate": "08/16/2015 22:16:48", "content": "This was our very first project in computer vision and in machine learning in general. We didn't manage to reach the top 10% (as we hoped), but we learned a lot from this contest. Now we would like to share our experience with everyone. Hopefully this will be helpful for the beginners.
\n\nAlso we would be very grateful if you point out more mistakes we have made :)
\n\nhttp://yerevann.github.io/2015/08/17/diabetic-retinopathy-detection-contest-what-we-did-wrong/
\n\nThank you all for the great contest.
", "rawMarkdown": "This was our very first project in computer vision and in machine learning in general. We didn't manage to reach the top 10% (as we hoped), but we learned a lot from this contest. Now we would like to share our experience with everyone. Hopefully this will be helpful for the beginners.\r\n\r\nAlso we would be very grateful if you point out more mistakes we have made :)\r\n\r\nhttp://yerevann.github.io/2015/08/17/diabetic-retinopathy-detection-contest-what-we-did-wrong/\r\n\r\nThank you all for the great contest.", "votes": null }, { "id": "89987", "postDate": "08/21/2015 07:55:48", "content": "Thanks for sharing :-), can you please also share the code you used to prepare lmdb file from raw image dataset
", "rawMarkdown": "Thanks for sharing :-), can you please also share the code you used to prepare lmdb file from raw image dataset", "votes": null }, { "id": "89988", "postDate": "08/21/2015 08:04:43", "content": "We used LevelDB. Please check the last paragraphs of the training and validation sets section. The following command created the DB:
\n\n./build/tools/convert_imageset -backend=leveldb -gray=true -shuffle=true data/train.g/ train.g.01v234.txt leveldb/train.g.01v234\nThanks
", "rawMarkdown": "Thanks", "votes": null }, { "id": "90502", "postDate": "08/27/2015 05:38:16", "content": "Your code is really helpful thanks for posting, m still in process of understanding deep learning and caffe both. I have some doubts please help,\nIn your network architecture did you designed it in way so that it can handle class imbalance. From your document you mentioned that you tried to do it with oversmapling and undersampling of the classes. But did you try changing the architecture also?\nAlso can you please tell how much minimum iteration one should go for while training the network.\nThanks :-)
", "rawMarkdown": "Your code is really helpful thanks for posting, m still in process of understanding deep learning and caffe both. I have some doubts please help,\r\nIn your network architecture did you designed it in way so that it can handle class imbalance. From your document you mentioned that you tried to do it with oversmapling and undersampling of the classes. But did you try changing the architecture also?\r\nAlso can you please tell how much minimum iteration one should go for while training the network.\r\nThanks :-)", "votes": null }, { "id": "90550", "postDate": "08/27/2015 18:32:25", "content": "No․․ I don't know any way to compensate for the class imbalance by modifying the architecture.
", "rawMarkdown": "No․․ I don't know any way to compensate for the class imbalance by modifying the architecture.", "votes": null }, { "id": "90551", "postDate": "08/27/2015 18:33:04", "content": "Usually overfitting started between 40000-60000 iterations
", "rawMarkdown": "Usually overfitting started between 40000-60000 iterations", "votes": null }, { "id": "103112", "postDate": "12/29/2015 01:02:14", "content": "Hi, could I ask when you split the train data to train and val sets, how do you generate the label files from trainlabels.csv please?
", "rawMarkdown": "Hi, could I ask when you split the train data to train and val sets, how do you generate the label files from trainlabels.csv please?", "votes": null }, { "id": "103148", "postDate": "12/29/2015 11:40:56", "content": "Hi,
\n\nI think we did it manually: we copied some of the lines to train.csv and the remaining ones to val.csv. Caffe's convert_imageset tool (which creates a LevelDB or LMDB database) requires a space separated file which contains lines like this:
image1.jpg 0\nimage2.jpg 3\nwhere 0 and 3 are the class labels (they must be integers!). So technically it's not a CSV (comma seperated) anymore.
Thanks for sharing :-), can you please also share the code you used to prepare lmdb file from raw image dataset
", "votes": null, "replies": [] }, { "id": 89988, "author_name": "hrantkhachatrian", "author_url": "", "post_date": "08/21/2015 08:04:43", "content": "We used LevelDB. Please check the last paragraphs of the training and validation sets section. The following command created the DB:
\n\n./build/tools/convert_imageset -backend=leveldb -gray=true -shuffle=true data/train.g/ train.g.01v234.txt leveldb/train.g.01v234\nThanks
", "votes": null, "replies": [] }, { "id": 90502, "author_name": "shivang27", "author_url": "", "post_date": "08/27/2015 05:38:16", "content": "Your code is really helpful thanks for posting, m still in process of understanding deep learning and caffe both. I have some doubts please help,\nIn your network architecture did you designed it in way so that it can handle class imbalance. From your document you mentioned that you tried to do it with oversmapling and undersampling of the classes. But did you try changing the architecture also?\nAlso can you please tell how much minimum iteration one should go for while training the network.\nThanks :-)
", "votes": null, "replies": [] }, { "id": 90550, "author_name": "hrantkhachatrian", "author_url": "", "post_date": "08/27/2015 18:32:25", "content": "No․․ I don't know any way to compensate for the class imbalance by modifying the architecture.
", "votes": null, "replies": [] }, { "id": 90551, "author_name": "hrantkhachatrian", "author_url": "", "post_date": "08/27/2015 18:33:04", "content": "Usually overfitting started between 40000-60000 iterations
", "votes": null, "replies": [] }, { "id": 103112, "author_name": "peterxu1986", "author_url": "", "post_date": "12/29/2015 01:02:14", "content": "Hi, could I ask when you split the train data to train and val sets, how do you generate the label files from trainlabels.csv please?
", "votes": null, "replies": [] }, { "id": 103148, "author_name": "hrantkhachatrian", "author_url": "", "post_date": "12/29/2015 11:40:56", "content": "Hi,
\n\nI think we did it manually: we copied some of the lines to train.csv and the remaining ones to val.csv. Caffe's convert_imageset tool (which creates a LevelDB or LMDB database) requires a space separated file which contains lines like this:
image1.jpg 0\nimage2.jpg 3\nwhere 0 and 3 are the class labels (they must be integers!). So technically it's not a CSV (comma seperated) anymore.