I have scrapped flickr for pictures taken with the phones from the dataset, their urls are in flickr_images.tar.gz. There are ten folders, in each folder there is a urls file (contains links to flickr pages with images) and urls_final (contains direct links for downloading jpg’s), all photos are under creative commons license. There is also a good_jpgs file — that’s a list of all photos with resolution matching the ones in the provided training dataset. There is one problem, though — photos taken with moto x that flickr provides are made with an old (2013) model, but for our task a 2014 version is used, so these photos should be discarded. I haven’t yet been able to find any way to download large set of photos taken with moto x 2014. In total there are 5377 photos, after leaving only the ones that are in good_jpgs file.
I have also scrapped phone review sites for validation photos, their urls are in val_images.tar.gz archive. Each phone has its own folder with urls_smth file inside, where smth is the name of a website I have downloaded these images from, this file contains urls of the photos.
I have scrapped flickr for pictures taken with the phones from the dataset, their urls are in flickr_images.tar.gz. There are ten folders, in each folder there is a urls file (contains links to flickr pages with images) and urls_final (contains direct links for downloading jpg’s), all photos are under creative commons license. There is also a good_jpgs file — that’s a list of all photos with resolution matching the ones in the provided training dataset. There is one problem, though — photos taken with moto x that flickr provides are made with an old (2013) model, but for our task a 2014 version is used, so these photos should be discarded. I haven’t yet been able to find any way to download large set of photos taken with moto x 2014. In total there are 5377 photos, after leaving only the ones that are in good_jpgs file.
I have also scrapped phone review sites for validation photos, their urls are in val_images.tar.gz archive. Each phone has its own folder with urls_smth file inside, where smth is the name of a website I have downloaded these images from, this file contains urls of the photos.
That is awesome finding.
", "rawMarkdown": "That is awesome finding.", "votes": null }, { "id": "267315", "postDate": "01/11/2018 01:47:32", "content": "In good_jpgs we have images per camera:htc_m7\n749,\niphone_4s\n504,\niphone_6\n550,\nmoto_maxx\n550,\nmoto_x\n0,\nnexus_5x\n405,\nnexus_6\n651,\nsamsung_note3\n274,\nsamsung_s4\n1137,\nsony_nex7\n557\n
Did you use moto droid ultra for moto droid maxx? \nI think both camera are the same.
", "rawMarkdown": "Did you use moto droid ultra for moto droid maxx? \nI think both camera are the same.", "votes": null }, { "id": "272878", "postDate": "01/23/2018 22:28:11", "content": "This might be interesting too https://www.esato.com/phonephotos/
", "rawMarkdown": "This might be interesting too https://www.esato.com/phonephotos/", "votes": null }, { "id": "273881", "postDate": "01/25/2018 12:16:07", "content": "Collected 344 images for Moto X (camera XT1096) from flickr / wiki commons. Most of the images are in resolution 4160 x 2340, while Moto X images from the competition data section is in 4160 x 3120. This is because Moto X has wide and standard camera mode, so I suppose it is ok to use these images. Big part of images were taken by only 3-4 cameras.
", "rawMarkdown": "Collected 344 images for Moto X (camera XT1096) from flickr / wiki commons. Most of the images are in resolution 4160 x 2340, while Moto X images from the competition data section is in 4160 x 3120. This is because Moto X has wide and standard camera mode, so I suppose it is ok to use these images. Big part of images were taken by only 3-4 cameras.", "votes": null }, { "id": "274869", "postDate": "01/27/2018 15:19:56", "content": "Also consider that the quality of many of those jpgs is <93 hence for the NN it's almost as it's been already manipulated (using jpg compression quality 80 or 90).
\n\nI've decided to discard those.
", "rawMarkdown": "Also consider that the quality of many of those jpgs is <93 hence for the NN it's almost as it's been already manipulated (using jpg compression quality 80 or 90). \n\nI've decided to discard those.", "votes": null }, { "id": "274873", "postDate": "01/27/2018 15:26:12", "content": "hello, may I ask u that what is the meaning of 93? the quality?
", "rawMarkdown": "hello, may I ask u that what is the meaning of 93? the quality?", "votes": null }, { "id": "274874", "postDate": "01/27/2018 15:28:46", "content": "Original train set and one cherrypicked offending image from Gleb's train set:
\n\n$ identify -verbose HTC-1-M7/\\(HTC-1-M7\\)1.jpg | grep Quality\nQuality: 95\n$ identify -verbose ./samsung_s4/24750425488_00b9cac713_o.jpg | grep Quality\nQuality: 85\nthanks for your reply, I have thought about this question, but I have no way to get the quality. thanks again!
", "rawMarkdown": "thanks for your reply, I have thought about this question, but I have no way to get the quality. thanks again!", "votes": null }, { "id": "276816", "postDate": "02/01/2018 15:20:07", "content": "Collected 1162 images for iPhones, Samsungs, Moto X and Nex7 from https://www.esato.com/phonephotos/
", "rawMarkdown": "Collected 1162 images for iPhones, Samsungs, Moto X and Nex7 from https://www.esato.com/phonephotos/", "votes": null }, { "id": "277254", "postDate": "02/02/2018 16:42:18", "content": "Do these pictures have CC license?
", "rawMarkdown": "Do these pictures have CC license?", "votes": null }, { "id": "311796", "postDate": "04/10/2018 17:25:26", "content": "Thank you! It was very useful
", "rawMarkdown": "Thank you! It was very useful", "votes": null } ], "comments": [ { "id": 267150, "author_name": "serhiy", "author_url": "", "post_date": "01/10/2018 17:27:18", "content": "That is awesome finding.
", "votes": null, "replies": [] }, { "id": 267315, "author_name": "serhiy", "author_url": "", "post_date": "01/11/2018 01:47:32", "content": "In good_jpgs we have images per camera:htc_m7\n749,\niphone_4s\n504,\niphone_6\n550,\nmoto_maxx\n550,\nmoto_x\n0,\nnexus_5x\n405,\nnexus_6\n651,\nsamsung_note3\n274,\nsamsung_s4\n1137,\nsony_nex7\n557\n
Did you use moto droid ultra for moto droid maxx? \nI think both camera are the same.
", "votes": null, "replies": [] }, { "id": 272878, "author_name": "CVxTz", "author_url": "", "post_date": "01/23/2018 22:28:11", "content": "This might be interesting too https://www.esato.com/phonephotos/
", "votes": null, "replies": [] }, { "id": 273881, "author_name": "alexfir", "author_url": "", "post_date": "01/25/2018 12:16:07", "content": "Collected 344 images for Moto X (camera XT1096) from flickr / wiki commons. Most of the images are in resolution 4160 x 2340, while Moto X images from the competition data section is in 4160 x 3120. This is because Moto X has wide and standard camera mode, so I suppose it is ok to use these images. Big part of images were taken by only 3-4 cameras.
", "votes": null, "replies": [] }, { "id": 274869, "author_name": "antorsae", "author_url": "", "post_date": "01/27/2018 15:19:56", "content": "Also consider that the quality of many of those jpgs is <93 hence for the NN it's almost as it's been already manipulated (using jpg compression quality 80 or 90).
\n\nI've decided to discard those.
", "votes": null, "replies": [ { "id": 274873, "author_name": "oujiayu", "author_url": "", "post_date": "01/27/2018 15:26:12", "content": "hello, may I ask u that what is the meaning of 93? the quality?
", "votes": null, "replies": [] }, { "id": 274874, "author_name": "antorsae", "author_url": "", "post_date": "01/27/2018 15:28:46", "content": "Original train set and one cherrypicked offending image from Gleb's train set:
\n\n$ identify -verbose HTC-1-M7/\\(HTC-1-M7\\)1.jpg | grep Quality\nQuality: 95\n$ identify -verbose ./samsung_s4/24750425488_00b9cac713_o.jpg | grep Quality\nQuality: 85\nthanks for your reply, I have thought about this question, but I have no way to get the quality. thanks again!
", "votes": null, "replies": [] } ] }, { "id": 276816, "author_name": "monsta", "author_url": "", "post_date": "02/01/2018 15:20:07", "content": "Collected 1162 images for iPhones, Samsungs, Moto X and Nex7 from https://www.esato.com/phonephotos/
", "votes": null, "replies": [ { "id": 277254, "author_name": "bobutis", "author_url": "", "post_date": "02/02/2018 16:42:18", "content": "Do these pictures have CC license?
", "votes": null, "replies": [] } ] }, { "id": 311796, "author_name": "kostyaatarik", "author_url": "", "post_date": "04/10/2018 17:25:26", "content": "Thank you! It was very useful
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "267128": "I have scrapped flickr for pictures taken with the phones from the dataset, their urls are in `flickr_images.tar.gz`. There are ten folders, in each folder there is a `urls` file (contains links to flickr pages with images) and `urls_final` (contains direct links for downloading jpg’s), all photos are under creative commons license. There is also a `good_jpgs` file — that’s a list of all photos with resolution matching the ones in the provided training dataset. There is one problem, though — photos taken with moto x that flickr provides are made with an old (2013) model, but for our task a 2014 version is used, so these photos should be discarded. I haven’t yet been able to find any way to download large set of photos taken with moto x 2014. In total there are 5377 photos, after leaving only the ones that are in `good_jpgs` file.\n\nI have also scrapped phone review sites for validation photos, their urls are in `val_images.tar.gz` archive. Each phone has its own folder with `urls_smth` file inside, where `smth` is the name of a website I have downloaded these images from, this file contains urls of the photos.", "267150": "That is awesome finding.", "267315": "In good_jpgs we have images per camera:`htc_m7\n749,\niphone_4s\n504,\niphone_6\n550,\nmoto_maxx\n550,\nmoto_x\n0,\nnexus_5x\n405,\nnexus_6\n651,\nsamsung_note3\n274,\nsamsung_s4\n1137,\nsony_nex7\n557\n`", "267915": "Did you use moto droid ultra for moto droid maxx? \nI think both camera are the same.", "272878": "This might be interesting too https://www.esato.com/phonephotos/", "273881": "Collected 344 images for Moto X (camera XT1096) from flickr / wiki commons. Most of the images are in resolution 4160 x 2340, while Moto X images from the competition data section is in 4160 x 3120. This is because Moto X has wide and standard camera mode, so I suppose it is ok to use these images. Big part of images were taken by only 3-4 cameras.", "274869": "Also consider that the quality of many of those jpgs is <93 hence for the NN it's almost as it's been already manipulated (using jpg compression quality 80 or 90). \n\nI've decided to discard those.", "274873": "hello, may I ask u that what is the meaning of 93? the quality?", "274874": "Original train set and one cherrypicked offending image from Gleb's train set:\n\n $ identify -verbose HTC-1-M7/\\(HTC-1-M7\\)1.jpg | grep Quality\n Quality: 95\n $ identify -verbose ./samsung_s4/24750425488_00b9cac713_o.jpg | grep Quality\n Quality: 85", "274878": "thanks for your reply, I have thought about this question, but I have no way to get the quality. thanks again!", "276816": "Collected 1162 images for iPhones, Samsungs, Moto X and Nex7 from https://www.esato.com/phonephotos/", "277254": "Do these pictures have CC license?", "311796": "Thank you! It was very useful" }, "source": "meta" }