This script creates folders for each of the 447 whales, and moves the training images into the corresponding folder. The test images stay put.
\n\nThis makes it easier if you want to manually crop images (and avoid touching the test images), check out images of the same whale, etc.
\n\n...
\n\nimport os\nimport pandas as pd\n\ntrain = pd.read_csv('train.csv', index_col='Image')\n\nwhaleIDs = list(train['whaleID'].unique())\n\nfor w in whaleIDs:\n os.makedirs('./imgs/'+w)\n\nfor image in train.index:\n folder = train.loc[image, 'whaleID']\n old = './imgs/{}'.format(image)\n new = './imgs/{}/{}'.format(folder, image)\n try:\n os.rename(old, new)\n except:\n print('{} - {}'.format(image,folder))\nThis script creates folders for each of the 447 whales, and moves the training images into the corresponding folder. The test images stay put.
\n\nThis makes it easier if you want to manually crop images (and avoid touching the test images), check out images of the same whale, etc.
\n\n...
\n\nimport os\nimport pandas as pd\n\ntrain = pd.read_csv('train.csv', index_col='Image')\n\nwhaleIDs = list(train['whaleID'].unique())\n\nfor w in whaleIDs:\n os.makedirs('./imgs/'+w)\n\nfor image in train.index:\n folder = train.loc[image, 'whaleID']\n old = './imgs/{}'.format(image)\n new = './imgs/{}/{}'.format(folder, image)\n try:\n os.rename(old, new)\n except:\n print('{} - {}'.format(image,folder))\nAnd the same for those who like R - Only difference ( it doesnt move the image - creates a copy in another folder called train)
\n\nsetwd ("E:\\\\Kaggle\\\\whale\\\\data")\n\ninputFile <- read.csv("train.csv",header=T)\nfor(i in 1:nrow(inputFile)) { \n filePathFrom <- paste(".\\\\imgs\\\\",inputFile[i,1],sep="")\n filePathTo <- paste(".\\\\train\\\\",inputFile[i,1],sep="") \n cat(" file exist= ",file.exists(filePathFrom)," ",filePathFrom)\n success<-file.copy(from = filePathFrom, to = filePathTo)\n cat (" Copied = ", success, "\\n")\n}
\n\nOr to create individual directories for each whale -
\n\nsetwd ("E:\\\\Kaggle\\\\whale\\\\data")\ninputFile <- read.csv("train.csv",header=T)\n for(i in 1:nrow(inputFile)){\n filePathFrom <- paste(".\\\\imgs\\\\",inputFile[i,1],sep="") \n DirPathTo <- paste(".\\\\train\\\\",inputFile[i,2],sep="") \n if(!file.exists(DirPathTo)){\n a<-dir.create(DirPathTo)\n } \n filePathTo <- paste(DirPathTo,"\\\\",inputFile[i,1],sep="") \n success<-file.copy(from = filePathFrom, to = filePathTo)\n cat ("Copied ",filePathTo," = ", success, "\\n")\n}
", "rawMarkdown": "And the same for those who like R - Only difference ( it doesnt move the image - creates a copy in another folder called train)\r\n\r\n------------------------------\r\n setwd (\"E:\\\\Kaggle\\\\whale\\\\data\")\r\n\r\n inputFile <- read.csv(\"train.csv\",header=T)\r\n for(i in 1:nrow(inputFile)) { \r\n filePathFrom <- paste(\".\\\\imgs\\\\\",inputFile[i,1],sep=\"\")\r\n filePathTo <- paste(\".\\\\train\\\\\",inputFile[i,1],sep=\"\") \r\n cat(\" file exist= \",file.exists(filePathFrom),\" \",filePathFrom)\r\n success<-file.copy(from = filePathFrom, to = filePathTo)\r\n cat (\" Copied = \", success, \"\\n\")\r\n}\r\n\r\n------------------------------\r\n\r\nOr to create individual directories for each whale - \r\n\r\n setwd (\"E:\\\\Kaggle\\\\whale\\\\data\")\r\n inputFile <- read.csv(\"train.csv\",header=T)\r\n for(i in 1:nrow(inputFile)){\r\n filePathFrom <- paste(\".\\\\imgs\\\\\",inputFile[i,1],sep=\"\") \r\n DirPathTo <- paste(\".\\\\train\\\\\",inputFile[i,2],sep=\"\") \r\n if(!file.exists(DirPathTo)){\r\n a<-dir.create(DirPathTo)\r\n } \r\n filePathTo <- paste(DirPathTo,\"\\\\\",inputFile[i,1],sep=\"\") \r\n success<-file.copy(from = filePathFrom, to = filePathTo)\r\n cat (\"Copied \",filePathTo,\" = \", success, \"\\n\")\r\n}", "votes": null }, { "id": "91584", "postDate": "09/04/2015 15:43:42", "content": "I don't use Matlab, so I wrote a script to make it easy to generate positive and negative images for a Cascade classifier. This uses OpenCV (cv2) and Tkinter.
\n\nFor each image loaded click on any two diagonal corners of the rectangle you'd like to form to crop an image around the desired 'positive' portion of the photo. This will be saved in your 'positive' directory. Four 'negative' images will be saved in your 'negative' directory, corresponding to the 4 rectangular areas that can be cut out of the original photo without including the 'positive' region. This script pairs with user Inversion's image-sorting script above.
", "rawMarkdown": "I don't use Matlab, so I wrote a script to make it easy to generate positive and negative images for a Cascade classifier. This uses OpenCV (cv2) and Tkinter. \r\n\r\nFor each image loaded click on any two diagonal corners of the rectangle you'd like to form to crop an image around the desired 'positive' portion of the photo. This will be saved in your 'positive' directory. Four 'negative' images will be saved in your 'negative' directory, corresponding to the 4 rectangular areas that can be cut out of the original photo without including the 'positive' region. This script pairs with user Inversion's image-sorting script above.", "votes": null }, { "id": "91592", "postDate": "09/04/2015 16:41:26", "content": "Does there exists good algorithms for automatically cropping of whales ?
", "rawMarkdown": "Does there exists good algorithms for automatically cropping of whales ?", "votes": null }, { "id": "92674", "postDate": "09/15/2015 11:33:32", "content": "@CauchyKun: Really great script, thanks for sharing!
\n\nI've never made an object detector before. Does it matter when forming positive and negative training regions that they are similar in shape and size? I haven't thought yet how my classifier is going to work, but I could imagine it matters...
", "rawMarkdown": "CauchyKun: Really great script, thanks for sharing!\r\n\r\nI've never made an object detector before. Does it matter when forming positive and negative training regions that they are similar in shape and size? I haven't thought yet how my classifier is going to work, but I could imagine it matters...", "votes": null }, { "id": "92930", "postDate": "09/18/2015 21:00:06", "content": "Python is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
", "rawMarkdown": "Python is overkill ;-)\r\nHere is two lines in Bash for the same thing:\r\n\r\n # create directories structure\r\n tail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\r\n# copy files\r\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\r\n\r\nRun in directory with ***train.csv*** and ***imgs***.", "votes": null }, { "id": "92933", "postDate": "09/18/2015 21:12:13", "content": "[quote=Rasim Akhunzyanov;92930]
\n\nPython is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
\n\n[/quote]
\n\nShowoff. :-)
\n\nAnyway, I could've done it in one line of python had I, too, used semi-colons. ;-)
", "rawMarkdown": "[quote=Rasim Akhunzyanov;92930]\r\n\r\nPython is overkill ;-)\r\nHere is two lines in Bash for the same thing:\r\n\r\n # create directories structure\r\n tail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\r\n# copy files\r\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\r\n\r\nRun in directory with ***train.csv*** and ***imgs***.\r\n\r\n[/quote]\r\n\r\nShowoff. :-)\r\n\r\nAnyway, I could've done it in one line of python had I, too, used semi-colons. ;-)", "votes": null }, { "id": "93130", "postDate": "09/21/2015 21:37:32", "content": "[quote=Rasim Akhunzyanov;92930]
\n\nPython is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
\n\n[/quote]
\n\nTo make it OSX compatible, replace the second command with:
\n\n# copy files\ntail -n +2 train.csv | tr ',' ' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nAnd the same for those who like R - Only difference ( it doesnt move the image - creates a copy in another folder called train)
\n\nsetwd ("E:\\\\Kaggle\\\\whale\\\\data")\n\ninputFile <- read.csv("train.csv",header=T)\nfor(i in 1:nrow(inputFile)) { \n filePathFrom <- paste(".\\\\imgs\\\\",inputFile[i,1],sep="")\n filePathTo <- paste(".\\\\train\\\\",inputFile[i,1],sep="") \n cat(" file exist= ",file.exists(filePathFrom)," ",filePathFrom)\n success<-file.copy(from = filePathFrom, to = filePathTo)\n cat (" Copied = ", success, "\\n")\n}
\n\nOr to create individual directories for each whale -
\n\nsetwd ("E:\\\\Kaggle\\\\whale\\\\data")\ninputFile <- read.csv("train.csv",header=T)\n for(i in 1:nrow(inputFile)){\n filePathFrom <- paste(".\\\\imgs\\\\",inputFile[i,1],sep="") \n DirPathTo <- paste(".\\\\train\\\\",inputFile[i,2],sep="") \n if(!file.exists(DirPathTo)){\n a<-dir.create(DirPathTo)\n } \n filePathTo <- paste(DirPathTo,"\\\\",inputFile[i,1],sep="") \n success<-file.copy(from = filePathFrom, to = filePathTo)\n cat ("Copied ",filePathTo," = ", success, "\\n")\n}
", "votes": null, "replies": [] }, { "id": 91584, "author_name": "cauchykun", "author_url": "", "post_date": "09/04/2015 15:43:42", "content": "I don't use Matlab, so I wrote a script to make it easy to generate positive and negative images for a Cascade classifier. This uses OpenCV (cv2) and Tkinter.
\n\nFor each image loaded click on any two diagonal corners of the rectangle you'd like to form to crop an image around the desired 'positive' portion of the photo. This will be saved in your 'positive' directory. Four 'negative' images will be saved in your 'negative' directory, corresponding to the 4 rectangular areas that can be cut out of the original photo without including the 'positive' region. This script pairs with user Inversion's image-sorting script above.
", "votes": null, "replies": [] }, { "id": 91592, "author_name": "usixuz", "author_url": "", "post_date": "09/04/2015 16:41:26", "content": "Does there exists good algorithms for automatically cropping of whales ?
", "votes": null, "replies": [] }, { "id": 92674, "author_name": "devonbrackbill", "author_url": "", "post_date": "09/15/2015 11:33:32", "content": "@CauchyKun: Really great script, thanks for sharing!
\n\nI've never made an object detector before. Does it matter when forming positive and negative training regions that they are similar in shape and size? I haven't thought yet how my classifier is going to work, but I could imagine it matters...
", "votes": null, "replies": [] }, { "id": 92930, "author_name": "rasimakhunzyanov", "author_url": "", "post_date": "09/18/2015 21:00:06", "content": "Python is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
", "votes": null, "replies": [] }, { "id": 92933, "author_name": "inversion", "author_url": "", "post_date": "09/18/2015 21:12:13", "content": "[quote=Rasim Akhunzyanov;92930]
\n\nPython is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
\n\n[/quote]
\n\nShowoff. :-)
\n\nAnyway, I could've done it in one line of python had I, too, used semi-colons. ;-)
", "votes": null, "replies": [] }, { "id": 93130, "author_name": "isabelmoreno", "author_url": "", "post_date": "09/21/2015 21:37:32", "content": "[quote=Rasim Akhunzyanov;92930]
\n\nPython is overkill ;-)\nHere is two lines in Bash for the same thing:
\n\n# create directories structure\ntail -n +2 train.csv | cut -d, -f2 | uniq | while read whale; do mkdir -p train/$whale; done\n# copy files\ntail -n +2 train.csv | cut -d, --fields=1-2 --output-delimiter=' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\nRun in directory with train.csv and imgs.
\n\n[/quote]
\n\nTo make it OSX compatible, replace the second command with:
\n\n# copy files\ntail -n +2 train.csv | tr ',' ' ' | while read image whale; do cp imgs/$image train/$whale/$image; done\n