As far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.
", "messages": [ { "id": "91752", "postDate": "09/07/2015 05:42:33", "content": "As far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.
", "rawMarkdown": "As far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.", "votes": null }, { "id": "91754", "postDate": "09/07/2015 06:59:00", "content": "[quote=SalvadorDali;91752]
\n\nAs far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.
\n\n[/quote]
\n\nIt would not be possible to publish that work and win the prize, since it would be clear that part of the pipeline was missing and the result could not be replicated without it.
\n\nSo the only purpose would be to get a higher leaderboard position.
\n\nIn addition to costing money, posts on crowd-sourcing sites are usually public, so there would be evidence of cheating which if discovered will get even the few kaggle rank/tier improvements possible with this exploit rendered null and void. Result: lost cash for the outsourcing, wasted time, nothing useful about ML learned by competing.
\n\nYou could I suppose outsource privately. In which case yes you might get away with it. It still does seem a lot of effort to go to for no financial reward.
", "rawMarkdown": "[quote=SalvadorDali;91752]\r\n\r\nAs far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.\r\n\r\n[/quote]\r\n\r\nIt would not be possible to publish that work and win the prize, since it would be clear that part of the pipeline was missing and the result could not be replicated without it.\r\n\r\nSo the only purpose would be to get a higher leaderboard position.\r\n\r\nIn addition to costing money, posts on crowd-sourcing sites are usually public, so there would be evidence of cheating which if discovered will get even the few kaggle rank/tier improvements possible with this exploit rendered null and void. Result: lost cash for the outsourcing, wasted time, nothing useful about ML learned by competing.\r\n\r\nYou could I suppose outsource privately. In which case yes you might get away with it. It still does seem a lot of effort to go to for no financial reward.", "votes": null }, { "id": "99678", "postDate": "11/28/2015 04:38:39", "content": "That is a valid approach but seems there are hard limits on performance as well as high cost.
", "rawMarkdown": "That is a valid approach but seems there are hard limits on performance as well as high cost.", "votes": null }, { "id": "100453", "postDate": "12/07/2015 17:10:11", "content": "Salvador
\n\nI think the answer is cost to scale.
\n\nThe task at hand is to take an image and match it against a reference database of say 5k known whales. So for each image an expert would need to spend 2 weeks to identify a particular whale. Expert would not be someone working on lower wages but some scientist who gets paid $1k per week to do accurately. Hence an accurate identification using humans would be prohibitively expensive.
\n\nWith an algorithm however you could do classify 1k target images in a matter of seconds for $1 variable cost.
\n\nLet me provide an example. A relative of mine automated the fingerprint identification process for FBI using Machine Learning. Prior to that system FBI used to use a manual review for matching a target fingerprint against a reference data base of 2 million fingerprints. So the fingerprint analyst used to take an image and sift through tens of thousands of fingerprints to compare against. If you are the analyst you would not be able to do a good job. If you see fingerprints all day you can go pretty crazy. In any case it much quicker for you to spend a month doing nothing but reading magazines and sip coffee and come back and report a 'no match' without actually doing any efforts.
\n\nInstead they a used Fourier transforms based fingerprint matching algorithm to automate this. Now the algo can generate a short list of handful of prints - ten or so for manual review. This is done in a matter of seconds. What a human could not do accurately in a month a computer can do in seconds.
\n\nAgain this is at a very small variable cost. The main cost is the fixed cost of man, data, and machine. They are trying to do a similar thing here for identifying whales instead of human fingerprints.
\n\nHope that helps.
", "rawMarkdown": "Salvador\r\n\r\nI think the answer is cost to scale.\r\n\r\nThe task at hand is to take an image and match it against a reference database of say 5k known whales. So for each image an expert would need to spend 2 weeks to identify a particular whale. Expert would not be someone working on lower wages but some scientist who gets paid $1k per week to do accurately. Hence an accurate identification using humans would be prohibitively expensive.\r\n\r\nWith an algorithm however you could do classify 1k target images in a matter of seconds for $1 variable cost. \r\n\r\nLet me provide an example. A relative of mine automated the fingerprint identification process for FBI using Machine Learning. Prior to that system FBI used to use a manual review for matching a target fingerprint against a reference data base of 2 million fingerprints. So the fingerprint analyst used to take an image and sift through tens of thousands of fingerprints to compare against. If you are the analyst you would not be able to do a good job. If you see fingerprints all day you can go pretty crazy. In any case it much quicker for you to spend a month doing nothing but reading magazines and sip coffee and come back and report a 'no match' without actually doing any efforts. \r\n\r\nInstead they a used Fourier transforms based fingerprint matching algorithm to automate this. Now the algo can generate a short list of handful of prints - ten or so for manual review. This is done in a matter of seconds. What a human could not do accurately in a month a computer can do in seconds. \r\n\r\nAgain this is at a very small variable cost. The main cost is the fixed cost of man, data, and machine. They are trying to do a similar thing here for identifying whales instead of human fingerprints. \r\n\r\nHope that helps.", "votes": null } ], "comments": [ { "id": 91754, "author_name": "slobo777", "author_url": "", "post_date": "09/07/2015 06:59:00", "content": "[quote=SalvadorDali;91752]
\n\nAs far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.
\n\n[/quote]
\n\nIt would not be possible to publish that work and win the prize, since it would be clear that part of the pipeline was missing and the result could not be replicated without it.
\n\nSo the only purpose would be to get a higher leaderboard position.
\n\nIn addition to costing money, posts on crowd-sourcing sites are usually public, so there would be evidence of cheating which if discovered will get even the few kaggle rank/tier improvements possible with this exploit rendered null and void. Result: lost cash for the outsourcing, wasted time, nothing useful about ML learned by competing.
\n\nYou could I suppose outsource privately. In which case yes you might get away with it. It still does seem a lot of effort to go to for no financial reward.
", "votes": null, "replies": [] }, { "id": 99678, "author_name": "michaeltconway", "author_url": "", "post_date": "11/28/2015 04:38:39", "content": "That is a valid approach but seems there are hard limits on performance as well as high cost.
", "votes": null, "replies": [] }, { "id": 100453, "author_name": "veerkar", "author_url": "", "post_date": "12/07/2015 17:10:11", "content": "Salvador
\n\nI think the answer is cost to scale.
\n\nThe task at hand is to take an image and match it against a reference database of say 5k known whales. So for each image an expert would need to spend 2 weeks to identify a particular whale. Expert would not be someone working on lower wages but some scientist who gets paid $1k per week to do accurately. Hence an accurate identification using humans would be prohibitively expensive.
\n\nWith an algorithm however you could do classify 1k target images in a matter of seconds for $1 variable cost.
\n\nLet me provide an example. A relative of mine automated the fingerprint identification process for FBI using Machine Learning. Prior to that system FBI used to use a manual review for matching a target fingerprint against a reference data base of 2 million fingerprints. So the fingerprint analyst used to take an image and sift through tens of thousands of fingerprints to compare against. If you are the analyst you would not be able to do a good job. If you see fingerprints all day you can go pretty crazy. In any case it much quicker for you to spend a month doing nothing but reading magazines and sip coffee and come back and report a 'no match' without actually doing any efforts.
\n\nInstead they a used Fourier transforms based fingerprint matching algorithm to automate this. Now the algo can generate a short list of handful of prints - ten or so for manual review. This is done in a matter of seconds. What a human could not do accurately in a month a computer can do in seconds.
\n\nAgain this is at a very small variable cost. The main cost is the fixed cost of man, data, and machine. They are trying to do a similar thing here for identifying whales instead of human fingerprints.
\n\nHope that helps.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "91752": "As far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.", "91754": "[quote=SalvadorDali;91752]\r\n\r\nAs far as I understood, a huge part of the competition is to build a system which will automatically detect the face of the whale on the picture. So what stops people from detecting the faces by hand (takes like 30 seconds per picture) and then feeding these nicely selected faces to an algorithm? It will not take a lot of time to do this, by outsourcing the work.\r\n\r\n[/quote]\r\n\r\nIt would not be possible to publish that work and win the prize, since it would be clear that part of the pipeline was missing and the result could not be replicated without it.\r\n\r\nSo the only purpose would be to get a higher leaderboard position.\r\n\r\nIn addition to costing money, posts on crowd-sourcing sites are usually public, so there would be evidence of cheating which if discovered will get even the few kaggle rank/tier improvements possible with this exploit rendered null and void. Result: lost cash for the outsourcing, wasted time, nothing useful about ML learned by competing.\r\n\r\nYou could I suppose outsource privately. In which case yes you might get away with it. It still does seem a lot of effort to go to for no financial reward.", "99678": "That is a valid approach but seems there are hard limits on performance as well as high cost.", "100453": "Salvador\r\n\r\nI think the answer is cost to scale.\r\n\r\nThe task at hand is to take an image and match it against a reference database of say 5k known whales. So for each image an expert would need to spend 2 weeks to identify a particular whale. Expert would not be someone working on lower wages but some scientist who gets paid $1k per week to do accurately. Hence an accurate identification using humans would be prohibitively expensive.\r\n\r\nWith an algorithm however you could do classify 1k target images in a matter of seconds for $1 variable cost. \r\n\r\nLet me provide an example. A relative of mine automated the fingerprint identification process for FBI using Machine Learning. Prior to that system FBI used to use a manual review for matching a target fingerprint against a reference data base of 2 million fingerprints. So the fingerprint analyst used to take an image and sift through tens of thousands of fingerprints to compare against. If you are the analyst you would not be able to do a good job. If you see fingerprints all day you can go pretty crazy. In any case it much quicker for you to spend a month doing nothing but reading magazines and sip coffee and come back and report a 'no match' without actually doing any efforts. \r\n\r\nInstead they a used Fourier transforms based fingerprint matching algorithm to automate this. Now the algo can generate a short list of handful of prints - ten or so for manual review. This is done in a matter of seconds. What a human could not do accurately in a month a computer can do in seconds. \r\n\r\nAgain this is at a very small variable cost. The main cost is the fixed cost of man, data, and machine. They are trying to do a similar thing here for identifying whales instead of human fingerprints. \r\n\r\nHope that helps." }, "source": "meta" }