{ "id": 17579, "title": "MATLAB (Jump)Starter Codes for latecomers", "url": "/competitions/noaa-right-whale-recognition/discussion/17579", "author_name": "", "post_date": "2015-11-26T14:01:16.490Z", "votes": 4, "comment_count": 18, "views": 3997, "content": "

Sorry for posting this late,

\n\n

For all the latecomers (an all other kagglers of course) whom intend or are using MATLAB for the challenge, here are some starter codes that were presented (more than a month ago) at the Paris Kaggle meetup group.

\n\n

If you have questions about the codes please feel free to use this thread.

\n\n

Cheers,

\n\n

Amine

", "messages": [ { "id": "99562", "postDate": "11/26/2015 14:01:16", "content": "

Sorry for posting this late,

\n\n

For all the latecomers (an all other kagglers of course) whom intend or are using MATLAB for the challenge, here are some starter codes that were presented (more than a month ago) at the Paris Kaggle meetup group.

\n\n

If you have questions about the codes please feel free to use this thread.

\n\n

Cheers,

\n\n

Amine

", "rawMarkdown": "Sorry for posting this late, \r\n\r\nFor all the latecomers (an all other kagglers of course) whom intend or are using MATLAB for the challenge, here are some starter codes that were presented (more than a month ago) at the Paris Kaggle meetup group.\r\n\r\nIf you have questions about the codes please feel free to use this thread.\r\n\r\nCheers,\r\n\r\nAmine", "votes": null }, { "id": "99572", "postDate": "11/26/2015 18:04:29", "content": "

please share more code

", "rawMarkdown": "please share more code", "votes": null }, { "id": "99620", "postDate": "11/27/2015 14:46:40", "content": "

Thanks a lot!

", "rawMarkdown": "Thanks a lot!", "votes": null }, { "id": "99783", "postDate": "11/30/2015 09:40:37", "content": "

@irumshafique

\n\n

Sure, could you be more specific on the code examples you may need?

\n\n

Glad you're enjoying those codes!

", "rawMarkdown": "irumshafique\r\n\r\nSure, could you be more specific on the code examples you may need?\r\n\r\nGlad you're enjoying those codes!", "votes": null }, { "id": "100105", "postDate": "12/02/2015 19:22:59", "content": "

can you send me your email address please

", "rawMarkdown": "can you send me your email address please", "votes": null }, { "id": "100145", "postDate": "12/03/2015 03:01:28", "content": "

in Step0, I receive:

\n\n

Cannot CD to imgs_rand_subset (Name is nonexistent or not a\ndirectory).

\n\n

Error in Step0_ImageFiles_reorganization (line 80)\ncd([subsetFolder_out]);

", "rawMarkdown": "in Step0, I receive:\r\n\r\nCannot CD to imgs_rand_subset (Name is nonexistent or not a\r\ndirectory).\r\n\r\nError in Step0_ImageFiles_reorganization (line 80)\r\ncd([subsetFolder_out]);", "votes": null }, { "id": "100166", "postDate": "12/03/2015 08:57:14", "content": "

@ sonoporation:

\n\n

You have to create the folder where you want to save your pics into (all or just a subset) .\nIn the example, I had called the folder : 'imgs_rand_subset ' but you can call it as you like and if you want to create it dircetly from MATLAB, you can use the command:\nmkdir('imgs_rand_subset');

\n\n

Hope this helps

", "rawMarkdown": "sonoporation:\r\n\r\nYou have to create the folder where you want to save your pics into (all or just a subset) .\r\nIn the example, I had called the folder : 'imgs_rand_subset ' but you can call it as you like and if you want to create it dircetly from MATLAB, you can use the command:\r\nmkdir('imgs_rand_subset');\r\n\r\nHope this helps", "votes": null }, { "id": "100214", "postDate": "12/03/2015 19:10:18", "content": "

find this error in step 0 \n??? Undefined function or method 'readtable' for input arguments of\ntype 'char'.

\n\n

Error in ==> Step0_ImageFiles_reorganization at 7\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');

", "rawMarkdown": "find this error in step 0 \r\n??? Undefined function or method 'readtable' for input arguments of\r\ntype 'char'.\r\n\r\nError in ==> Step0_ImageFiles_reorganization at 7\r\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');", "votes": null }, { "id": "100220", "postDate": "12/03/2015 19:41:35", "content": "

find this error in step 0 \n??? Undefined function or method 'readtable' for input arguments of\ntype 'char'.

\n\n

Error in ==> Step0_ImageFiles_reorganization at 7\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');

", "rawMarkdown": "find this error in step 0 \r\n??? Undefined function or method 'readtable' for input arguments of\r\ntype 'char'.\r\n\r\nError in ==> Step0_ImageFiles_reorganization at 7\r\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');", "votes": null }, { "id": "100242", "postDate": "12/04/2015 05:18:25", "content": "

@AmineHelou can you please tell us which matlab version you are used to write this code ?

", "rawMarkdown": "AmineHelou can you please tell us which matlab version you are used to write this code ?", "votes": null }, { "id": "100261", "postDate": "12/04/2015 09:00:22", "content": "

Yes indeed, I am using R2015b and I think the 'readtable' function was introduced in 14a.

\n\n

Cheers

", "rawMarkdown": "Yes indeed, I am using R2015b and I think the 'readtable' function was introduced in 14a.\r\n\r\nCheers", "votes": null }, { "id": "100351", "postDate": "12/06/2015 09:19:13", "content": "

@AmineHelou please share the code for step 4 and so on :)

", "rawMarkdown": "AmineHelou please share the code for step 4 and so on :)", "votes": null }, { "id": "100352", "postDate": "12/06/2015 09:21:19", "content": "

@AmineHelou \nplease share the code for step3 step 4 and so on :)

", "rawMarkdown": "AmineHelou \r\nplease share the code for step3 step 4 and so on :)", "votes": null }, { "id": "100422", "postDate": "12/07/2015 09:03:39", "content": "

How many ROIs needed to have the detector work? I used ~ 70 to train but the result was always incorrect or none.

", "rawMarkdown": "How many ROIs needed to have the detector work? I used ~ 70 to train but the result was always incorrect or none.", "votes": null }, { "id": "100424", "postDate": "12/07/2015 09:48:28", "content": "

@Stanly: Very good point, so there's no rule, the more you have the better. However, as you may have read in other threads, the detection of the whale (so does the cascade object detector) is very sensitive to its orientation on the image. So a preliminary step I would recommend doing before training the detector is to detect the orientation of the whale (using a sobel edge detection for example: http://fr.mathworks.com/help/images/examples/detecting-a-cell-using-image-segmentation.html) on the image and correct this offset (to a recommended 0° or vertical orientation) by applying the correct rotation to the image OR train different object detectors for different orientations (but I don't think the orientations of whales are overall well balanced on the training set).

\n\n

At the end you could get to a more robust 'hybrid' approach using image segmentation techniques (check the documentation for 'edge', 'entropyfilt' & 'regionprops' functions) & the cascade object detector before starting the feature extraction part.

\n\n

@irumshafique: step 3 & 4 are more the core of the challenge in which you have to figure out which features & machine-learning models work best. I will however try to post some more codes on the whale detection part (discussed in this answer) by the end of next week hopefully.

\n\n

Thank you for your patience

", "rawMarkdown": "Stanly: Very good point, so there's no rule, the more you have the better. However, as you may have read in other threads, the detection of the whale (so does the cascade object detector) is very sensitive to its orientation on the image. So a preliminary step I would recommend doing before training the detector is to detect the orientation of the whale (using a sobel edge detection for example: http://fr.mathworks.com/help/images/examples/detecting-a-cell-using-image-segmentation.html) on the image and correct this offset (to a recommended 0° or vertical orientation) by applying the correct rotation to the image OR train different object detectors for different orientations (but I don't think the orientations of whales are overall well balanced on the training set).\r\n\r\nAt the end you could get to a more robust 'hybrid' approach using image segmentation techniques (check the documentation for ['edge'][1], ['entropyfilt'][2] & ['regionprops'][3] functions) & the cascade object detector before starting the feature extraction part.\r\n\r\n@irumshafique: step 3 & 4 are more the core of the challenge in which you have to figure out which features & machine-learning models work best. I will however try to post some more codes on the whale detection part (discussed in this answer) by the end of next week hopefully.\r\n\r\nThank you for your patience\r\n\r\n\r\n [1]: http://fr.mathworks.com/help/images/ref/edge.html\r\n [2]: http://fr.mathworks.com/help/images/ref/entropyfilt.html\r\n [3]: http://fr.mathworks.com/help/images/ref/regionprops.html", "votes": null }, { "id": "100465", "postDate": "12/07/2015 18:18:46", "content": "

@AmineHelou: Thanks for your quick reply! I initially tried to rotate and crop with "regionprops" and "rgb2ycbcr". The success rate was around 60% but I still need more efforts to align the heads (as they could be under two cases: 0 or 180 degree). I thought that the ROI detector could do a better job.

", "rawMarkdown": "AmineHelou: Thanks for your quick reply! I initially tried to rotate and crop with \"regionprops\" and \"rgb2ycbcr\". The success rate was around 60% but I still need more efforts to align the heads (as they could be under two cases: 0 or 180 degree). I thought that the ROI detector could do a better job.", "votes": null }, { "id": "100549", "postDate": "12/08/2015 10:52:43", "content": "

@Stanley: Well that's indeed what makes it a 'challenge' right? :)\nThe generic Cascade Object Detector is originally dedicated to detect basic shapes (persons standing, faces, eyes, nose...) the goal is to tweak it &/or add some extra image processing steps to take advantage from it in order to detect the whale on the image. Then focus on identifying which whale in another independent step.\nOne other advice I could give is to maximize the number of 'negative images' (the accuracy is not linearly related to the quantity), but it always help gain some % at no extra cost of coding and some extra disk usage (just increase the N value for the 'create_negatives_auto' utility function) and adjusting the alignement to a 'vertical' position of the whale (face up ideally) would also increase accuracy & whale detection.

", "rawMarkdown": "Stanley: Well that's indeed what makes it a 'challenge' right? :)\r\nThe generic Cascade Object Detector is originally dedicated to detect basic shapes (persons standing, faces, eyes, nose...) the goal is to tweak it &/or add some extra image processing steps to take advantage from it in order to detect the whale on the image. Then focus on identifying which whale in another independent step.\r\nOne other advice I could give is to maximize the number of 'negative images' (the accuracy is not linearly related to the quantity), but it always help gain some % at no extra cost of coding and some extra disk usage (just increase the N value for the 'create_negatives_auto' utility function) and adjusting the alignement to a 'vertical' position of the whale (face up ideally) would also increase accuracy & whale detection.", "votes": null }, { "id": "101847", "postDate": "12/17/2015 13:19:24", "content": "

error in step0:\nError using histcounts\nExpected input number 1, x, to be one of these types:

\n\n

numeric

\n\n

Instead its type was categorical.

\n\n

Error in histcounts (line 95)\nvalidateattributes(x,{'numeric'},{'real'}, mfilename, 'x', 1)

\n\n

Error in Step0_ImageFiles_reorganization (line 13)\n[c,h] = histcounts(train.whaleID);

", "rawMarkdown": "error in step0:\r\nError using histcounts\r\nExpected input number 1, x, to be one of these types:\r\n\r\nnumeric\r\n\r\nInstead its type was categorical.\r\n\r\nError in histcounts (line 95)\r\nvalidateattributes(x,{'numeric'},{'real'}, mfilename, 'x', 1)\r\n\r\nError in Step0_ImageFiles_reorganization (line 13)\r\n[c,h] = histcounts(train.whaleID);", "votes": null }, { "id": "103560", "postDate": "01/04/2016 11:00:44", "content": "

@haiyuansun: Sorry for posting late but which version of MATLAB are you using?

\n\n

If necessary I could show you some workaround, but it would be easier to get the rest of the starter codes running to ask for the latest version here

", "rawMarkdown": "haiyuansun: Sorry for posting late but which version of MATLAB are you using?\r\n\r\nIf necessary I could show you some workaround, but it would be easier to get the rest of the starter codes running to ask for the latest version [here][1]\r\n\r\n\r\n [1]: https://fr.mathworks.com/academia/student-competitions/software-request-registration-kaggle.html", "votes": null } ], "comments": [ { "id": 99572, "author_name": "irumshafique07", "author_url": "", "post_date": "11/26/2015 18:04:29", "content": "

please share more code

", "votes": null, "replies": [] }, { "id": 99620, "author_name": "poizona", "author_url": "", "post_date": "11/27/2015 14:46:40", "content": "

Thanks a lot!

", "votes": null, "replies": [] }, { "id": 99783, "author_name": "mooune", "author_url": "", "post_date": "11/30/2015 09:40:37", "content": "

@irumshafique

\n\n

Sure, could you be more specific on the code examples you may need?

\n\n

Glad you're enjoying those codes!

", "votes": null, "replies": [] }, { "id": 100105, "author_name": "irumshafique07", "author_url": "", "post_date": "12/02/2015 19:22:59", "content": "

can you send me your email address please

", "votes": null, "replies": [] }, { "id": 100145, "author_name": "yjcheng", "author_url": "", "post_date": "12/03/2015 03:01:28", "content": "

in Step0, I receive:

\n\n

Cannot CD to imgs_rand_subset (Name is nonexistent or not a\ndirectory).

\n\n

Error in Step0_ImageFiles_reorganization (line 80)\ncd([subsetFolder_out]);

", "votes": null, "replies": [] }, { "id": 100166, "author_name": "mooune", "author_url": "", "post_date": "12/03/2015 08:57:14", "content": "

@ sonoporation:

\n\n

You have to create the folder where you want to save your pics into (all or just a subset) .\nIn the example, I had called the folder : 'imgs_rand_subset ' but you can call it as you like and if you want to create it dircetly from MATLAB, you can use the command:\nmkdir('imgs_rand_subset');

\n\n

Hope this helps

", "votes": null, "replies": [] }, { "id": 100214, "author_name": "irumshafique07", "author_url": "", "post_date": "12/03/2015 19:10:18", "content": "

find this error in step 0 \n??? Undefined function or method 'readtable' for input arguments of\ntype 'char'.

\n\n

Error in ==> Step0_ImageFiles_reorganization at 7\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');

", "votes": null, "replies": [] }, { "id": 100220, "author_name": "irumshafique07", "author_url": "", "post_date": "12/03/2015 19:41:35", "content": "

find this error in step 0 \n??? Undefined function or method 'readtable' for input arguments of\ntype 'char'.

\n\n

Error in ==> Step0_ImageFiles_reorganization at 7\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');

", "votes": null, "replies": [] }, { "id": 100242, "author_name": "irumshafique07", "author_url": "", "post_date": "12/04/2015 05:18:25", "content": "

@AmineHelou can you please tell us which matlab version you are used to write this code ?

", "votes": null, "replies": [] }, { "id": 100261, "author_name": "mooune", "author_url": "", "post_date": "12/04/2015 09:00:22", "content": "

Yes indeed, I am using R2015b and I think the 'readtable' function was introduced in 14a.

\n\n

Cheers

", "votes": null, "replies": [] }, { "id": 100351, "author_name": "irumshafique07", "author_url": "", "post_date": "12/06/2015 09:19:13", "content": "

@AmineHelou please share the code for step 4 and so on :)

", "votes": null, "replies": [] }, { "id": 100352, "author_name": "irumshafique07", "author_url": "", "post_date": "12/06/2015 09:21:19", "content": "

@AmineHelou \nplease share the code for step3 step 4 and so on :)

", "votes": null, "replies": [] }, { "id": 100422, "author_name": "shujin", "author_url": "", "post_date": "12/07/2015 09:03:39", "content": "

How many ROIs needed to have the detector work? I used ~ 70 to train but the result was always incorrect or none.

", "votes": null, "replies": [] }, { "id": 100424, "author_name": "mooune", "author_url": "", "post_date": "12/07/2015 09:48:28", "content": "

@Stanly: Very good point, so there's no rule, the more you have the better. However, as you may have read in other threads, the detection of the whale (so does the cascade object detector) is very sensitive to its orientation on the image. So a preliminary step I would recommend doing before training the detector is to detect the orientation of the whale (using a sobel edge detection for example: http://fr.mathworks.com/help/images/examples/detecting-a-cell-using-image-segmentation.html) on the image and correct this offset (to a recommended 0° or vertical orientation) by applying the correct rotation to the image OR train different object detectors for different orientations (but I don't think the orientations of whales are overall well balanced on the training set).

\n\n

At the end you could get to a more robust 'hybrid' approach using image segmentation techniques (check the documentation for 'edge', 'entropyfilt' & 'regionprops' functions) & the cascade object detector before starting the feature extraction part.

\n\n

@irumshafique: step 3 & 4 are more the core of the challenge in which you have to figure out which features & machine-learning models work best. I will however try to post some more codes on the whale detection part (discussed in this answer) by the end of next week hopefully.

\n\n

Thank you for your patience

", "votes": null, "replies": [] }, { "id": 100465, "author_name": "shujin", "author_url": "", "post_date": "12/07/2015 18:18:46", "content": "

@AmineHelou: Thanks for your quick reply! I initially tried to rotate and crop with "regionprops" and "rgb2ycbcr". The success rate was around 60% but I still need more efforts to align the heads (as they could be under two cases: 0 or 180 degree). I thought that the ROI detector could do a better job.

", "votes": null, "replies": [] }, { "id": 100549, "author_name": "mooune", "author_url": "", "post_date": "12/08/2015 10:52:43", "content": "

@Stanley: Well that's indeed what makes it a 'challenge' right? :)\nThe generic Cascade Object Detector is originally dedicated to detect basic shapes (persons standing, faces, eyes, nose...) the goal is to tweak it &/or add some extra image processing steps to take advantage from it in order to detect the whale on the image. Then focus on identifying which whale in another independent step.\nOne other advice I could give is to maximize the number of 'negative images' (the accuracy is not linearly related to the quantity), but it always help gain some % at no extra cost of coding and some extra disk usage (just increase the N value for the 'create_negatives_auto' utility function) and adjusting the alignement to a 'vertical' position of the whale (face up ideally) would also increase accuracy & whale detection.

", "votes": null, "replies": [] }, { "id": 101847, "author_name": "haiyuansun", "author_url": "", "post_date": "12/17/2015 13:19:24", "content": "

error in step0:\nError using histcounts\nExpected input number 1, x, to be one of these types:

\n\n

numeric

\n\n

Instead its type was categorical.

\n\n

Error in histcounts (line 95)\nvalidateattributes(x,{'numeric'},{'real'}, mfilename, 'x', 1)

\n\n

Error in Step0_ImageFiles_reorganization (line 13)\n[c,h] = histcounts(train.whaleID);

", "votes": null, "replies": [] }, { "id": 103560, "author_name": "mooune", "author_url": "", "post_date": "01/04/2016 11:00:44", "content": "

@haiyuansun: Sorry for posting late but which version of MATLAB are you using?

\n\n

If necessary I could show you some workaround, but it would be easier to get the rest of the starter codes running to ask for the latest version here

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "99562": "Sorry for posting this late, \r\n\r\nFor all the latecomers (an all other kagglers of course) whom intend or are using MATLAB for the challenge, here are some starter codes that were presented (more than a month ago) at the Paris Kaggle meetup group.\r\n\r\nIf you have questions about the codes please feel free to use this thread.\r\n\r\nCheers,\r\n\r\nAmine", "99572": "please share more code", "99620": "Thanks a lot!", "99783": "irumshafique\r\n\r\nSure, could you be more specific on the code examples you may need?\r\n\r\nGlad you're enjoying those codes!", "100105": "can you send me your email address please", "100145": "in Step0, I receive:\r\n\r\nCannot CD to imgs_rand_subset (Name is nonexistent or not a\r\ndirectory).\r\n\r\nError in Step0_ImageFiles_reorganization (line 80)\r\ncd([subsetFolder_out]);", "100166": "sonoporation:\r\n\r\nYou have to create the folder where you want to save your pics into (all or just a subset) .\r\nIn the example, I had called the folder : 'imgs_rand_subset ' but you can call it as you like and if you want to create it dircetly from MATLAB, you can use the command:\r\nmkdir('imgs_rand_subset');\r\n\r\nHope this helps", "100214": "find this error in step 0 \r\n??? Undefined function or method 'readtable' for input arguments of\r\ntype 'char'.\r\n\r\nError in ==> Step0_ImageFiles_reorganization at 7\r\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');", "100220": "find this error in step 0 \r\n??? Undefined function or method 'readtable' for input arguments of\r\ntype 'char'.\r\n\r\nError in ==> Step0_ImageFiles_reorganization at 7\r\n train = readtable([dataFolder filesep 'train.csv'],'Format','%s%C');", "100242": "AmineHelou can you please tell us which matlab version you are used to write this code ?", "100261": "Yes indeed, I am using R2015b and I think the 'readtable' function was introduced in 14a.\r\n\r\nCheers", "100351": "AmineHelou please share the code for step 4 and so on :)", "100352": "AmineHelou \r\nplease share the code for step3 step 4 and so on :)", "100422": "How many ROIs needed to have the detector work? I used ~ 70 to train but the result was always incorrect or none.", "100424": "Stanly: Very good point, so there's no rule, the more you have the better. However, as you may have read in other threads, the detection of the whale (so does the cascade object detector) is very sensitive to its orientation on the image. So a preliminary step I would recommend doing before training the detector is to detect the orientation of the whale (using a sobel edge detection for example: http://fr.mathworks.com/help/images/examples/detecting-a-cell-using-image-segmentation.html) on the image and correct this offset (to a recommended 0° or vertical orientation) by applying the correct rotation to the image OR train different object detectors for different orientations (but I don't think the orientations of whales are overall well balanced on the training set).\r\n\r\nAt the end you could get to a more robust 'hybrid' approach using image segmentation techniques (check the documentation for ['edge'][1], ['entropyfilt'][2] & ['regionprops'][3] functions) & the cascade object detector before starting the feature extraction part.\r\n\r\n@irumshafique: step 3 & 4 are more the core of the challenge in which you have to figure out which features & machine-learning models work best. I will however try to post some more codes on the whale detection part (discussed in this answer) by the end of next week hopefully.\r\n\r\nThank you for your patience\r\n\r\n\r\n [1]: http://fr.mathworks.com/help/images/ref/edge.html\r\n [2]: http://fr.mathworks.com/help/images/ref/entropyfilt.html\r\n [3]: http://fr.mathworks.com/help/images/ref/regionprops.html", "100465": "AmineHelou: Thanks for your quick reply! I initially tried to rotate and crop with \"regionprops\" and \"rgb2ycbcr\". The success rate was around 60% but I still need more efforts to align the heads (as they could be under two cases: 0 or 180 degree). I thought that the ROI detector could do a better job.", "100549": "Stanley: Well that's indeed what makes it a 'challenge' right? :)\r\nThe generic Cascade Object Detector is originally dedicated to detect basic shapes (persons standing, faces, eyes, nose...) the goal is to tweak it &/or add some extra image processing steps to take advantage from it in order to detect the whale on the image. Then focus on identifying which whale in another independent step.\r\nOne other advice I could give is to maximize the number of 'negative images' (the accuracy is not linearly related to the quantity), but it always help gain some % at no extra cost of coding and some extra disk usage (just increase the N value for the 'create_negatives_auto' utility function) and adjusting the alignement to a 'vertical' position of the whale (face up ideally) would also increase accuracy & whale detection.", "101847": "error in step0:\r\nError using histcounts\r\nExpected input number 1, x, to be one of these types:\r\n\r\nnumeric\r\n\r\nInstead its type was categorical.\r\n\r\nError in histcounts (line 95)\r\nvalidateattributes(x,{'numeric'},{'real'}, mfilename, 'x', 1)\r\n\r\nError in Step0_ImageFiles_reorganization (line 13)\r\n[c,h] = histcounts(train.whaleID);", "103560": "haiyuansun: Sorry for posting late but which version of MATLAB are you using?\r\n\r\nIf necessary I could show you some workaround, but it would be easier to get the rest of the starter codes running to ask for the latest version [here][1]\r\n\r\n\r\n [1]: https://fr.mathworks.com/academia/student-competitions/software-request-registration-kaggle.html" }, "source": "meta" }