Just wondering so we can make our own "unofficial" public/private LB there for pure ML solutions (filtering out hand labeled submissions) and competitors who are here to learn (and prefer understanding where they are at against other pure ML competitors).
\n\nNot that hand labeling is bad, but it is easy to get >0.80 using hand labeling (on the train set, to train yourself). Especially with the Latitude/Longitude given for about 80% of the test data (and nearly 100% of train data), it is feasible to get much better than that score in the public/private LB (and it skews the top LB). But how ML will go would be interesting to see.
\n\nHand labelers who have issues ordering the images: use an ImageJ stack, register the images, and start panning fast enough so you can get hands on the differences and start sorting the images out. And/Or use online services (ex: MapBox, GIS free services, etc.).
", "messages": [ { "id": "120611", "postDate": "05/19/2016 12:59:41", "content": "Just wondering so we can make our own "unofficial" public/private LB there for pure ML solutions (filtering out hand labeled submissions) and competitors who are here to learn (and prefer understanding where they are at against other pure ML competitors).
\n\nNot that hand labeling is bad, but it is easy to get >0.80 using hand labeling (on the train set, to train yourself). Especially with the Latitude/Longitude given for about 80% of the test data (and nearly 100% of train data), it is feasible to get much better than that score in the public/private LB (and it skews the top LB). But how ML will go would be interesting to see.
\n\nHand labelers who have issues ordering the images: use an ImageJ stack, register the images, and start panning fast enough so you can get hands on the differences and start sorting the images out. And/Or use online services (ex: MapBox, GIS free services, etc.).
", "rawMarkdown": "Just wondering so we can make our own *\"unofficial\"* public/private LB there for pure ML solutions (filtering out hand labeled submissions) and competitors who are here to learn (and prefer understanding where they are at against other pure ML competitors).\r\n\r\nNot that hand labeling is bad, but it is easy to get >0.80 using hand labeling (on the train set, to train yourself). Especially with the Latitude/Longitude given for about 80% of the test data (and nearly 100% of train data), it is feasible to get much better than that score in the public/private LB (and it skews the top LB). But how ML will go would be interesting to see.\r\n\r\nHand labelers who have issues ordering the images: use an ImageJ stack, register the images, and start panning fast enough so you can get hands on the differences and start sorting the images out. And/Or use online services (ex: MapBox, GIS free services, etc.).", "votes": null }, { "id": "120716", "postDate": "05/20/2016 03:06:46", "content": "I am using a pure ML solution.
", "rawMarkdown": "I am using a pure ML solution.", "votes": null }, { "id": "120724", "postDate": "05/20/2016 04:51:42", "content": "Pure ML too. Not using any external data at this point either.
", "rawMarkdown": "Pure ML too. Not using any external data at this point either.", "votes": null }, { "id": "120755", "postDate": "05/20/2016 10:11:28", "content": "I am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
", "rawMarkdown": "I am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.", "votes": null }, { "id": "120791", "postDate": "05/20/2016 15:05:55", "content": "[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
", "rawMarkdown": "[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?", "votes": null }, { "id": "120796", "postDate": "05/20/2016 15:32:22", "content": "[quote=thakadu;120791]
\n\n[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
\n\n[/quote]
\n\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.
", "rawMarkdown": "[quote=thakadu;120791]\r\n\r\n[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?\r\n\r\n\r\n[/quote]\r\n\r\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.", "votes": null }, { "id": "120797", "postDate": "05/20/2016 15:33:52", "content": "[quote=Tyler Vigen;120796]
\n\n[quote=thakadu;120791]
\n\n[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
\n\n[/quote]
\n\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.
\n\n[/quote]
\n\nI think so.\nAnd just by noticing that changing the order of these pictures in submission does not change the PL score.
", "rawMarkdown": "[quote=Tyler Vigen;120796]\r\n\r\n[quote=thakadu;120791]\r\n\r\n[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?\r\n\r\n\r\n[/quote]\r\n\r\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.\r\n\r\n[/quote]\r\n\r\nI think so.\r\nAnd just by noticing that changing the order of these pictures in submission does not change the PL score.", "votes": null }, { "id": "120829", "postDate": "05/20/2016 20:09:02", "content": "I'm doing ML only. I should have something up in the next couple of days.
", "rawMarkdown": "I'm doing ML only. I should have something up in the next couple of days.", "votes": null }, { "id": "120847", "postDate": "05/20/2016 22:56:05", "content": "I was using ML-only solution and failed miserably. At least, I learned a lot along the process.
", "rawMarkdown": "I was using ML-only solution and failed miserably. At least, I learned a lot along the process.", "votes": null }, { "id": "120979", "postDate": "05/22/2016 11:27:10", "content": "Pure ML solution end to end
", "rawMarkdown": "Pure ML solution end to end", "votes": null }, { "id": "120980", "postDate": "05/22/2016 11:39:30", "content": "Your score is amazing @Vicens Gaitan. In my opinion it's much better than the hand labeling approach ;) Congrats!
", "rawMarkdown": "Your score is amazing @Vicens Gaitan. In my opinion it's much better than the hand labeling approach ;) Congrats!", "votes": null }, { "id": "120987", "postDate": "05/22/2016 12:56:20", "content": "Thanks Ole, in my opinion there is a lot of room for improvement for machine learning solutions. I expect to see ML models in the .9x by the end of the competition.
", "rawMarkdown": "Thanks Ole, in my opinion there is a lot of room for improvement for machine learning solutions. I expect to see ML models in the .9x by the end of the competition.", "votes": null }, { "id": "121032", "postDate": "05/23/2016 04:30:18", "content": "@Gaitan, do you need to manually extract some image features or just automatically extract features from the image?
", "rawMarkdown": "Gaitan, do you need to manually extract some image features or just automatically extract features from the image?", "votes": null }, { "id": "121043", "postDate": "05/23/2016 07:29:00", "content": "Until now, all steps are fully automated: no hand labeling, no image annotation, no external data (no need to georeference the images in Google Earth). The relevant image features are automagicaly selected by the method.
", "rawMarkdown": "Until now, all steps are fully automated: no hand labeling, no image annotation, no external data (no need to georeference the images in Google Earth). The relevant image features are automagicaly selected by the method.", "votes": null } ], "comments": [ { "id": 120716, "author_name": "wikunia", "author_url": "", "post_date": "05/20/2016 03:06:46", "content": "I am using a pure ML solution.
", "votes": null, "replies": [] }, { "id": 120724, "author_name": "thakadu", "author_url": "", "post_date": "05/20/2016 04:51:42", "content": "Pure ML too. Not using any external data at this point either.
", "votes": null, "replies": [] }, { "id": 120755, "author_name": "olegpolivin", "author_url": "", "post_date": "05/20/2016 10:11:28", "content": "I am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
", "votes": null, "replies": [] }, { "id": 120791, "author_name": "thakadu", "author_url": "", "post_date": "05/20/2016 15:05:55", "content": "[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
", "votes": null, "replies": [] }, { "id": 120796, "author_name": "tylervigen", "author_url": "", "post_date": "05/20/2016 15:32:22", "content": "[quote=thakadu;120791]
\n\n[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
\n\n[/quote]
\n\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.
", "votes": null, "replies": [] }, { "id": 120797, "author_name": "olegpolivin", "author_url": "", "post_date": "05/20/2016 15:33:52", "content": "[quote=Tyler Vigen;120796]
\n\n[quote=thakadu;120791]
\n\n[quote=olegpolivin;120755]
\n\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.
\n\n[/quote]\nMay I ask how you know which ones are not reflected on the PL?
\n\n[/quote]
\n\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.
\n\n[/quote]
\n\nI think so.\nAnd just by noticing that changing the order of these pictures in submission does not change the PL score.
", "votes": null, "replies": [] }, { "id": 120829, "author_name": "mechenix", "author_url": "", "post_date": "05/20/2016 20:09:02", "content": "I'm doing ML only. I should have something up in the next couple of days.
", "votes": null, "replies": [] }, { "id": 120847, "author_name": "bguberfain", "author_url": "", "post_date": "05/20/2016 22:56:05", "content": "I was using ML-only solution and failed miserably. At least, I learned a lot along the process.
", "votes": null, "replies": [] }, { "id": 120979, "author_name": "vicensgaitan", "author_url": "", "post_date": "05/22/2016 11:27:10", "content": "Pure ML solution end to end
", "votes": null, "replies": [] }, { "id": 120980, "author_name": "wikunia", "author_url": "", "post_date": "05/22/2016 11:39:30", "content": "Your score is amazing @Vicens Gaitan. In my opinion it's much better than the hand labeling approach ;) Congrats!
", "votes": null, "replies": [] }, { "id": 120987, "author_name": "vicensgaitan", "author_url": "", "post_date": "05/22/2016 12:56:20", "content": "Thanks Ole, in my opinion there is a lot of room for improvement for machine learning solutions. I expect to see ML models in the .9x by the end of the competition.
", "votes": null, "replies": [] }, { "id": 121032, "author_name": "yourwanghao", "author_url": "", "post_date": "05/23/2016 04:30:18", "content": "@Gaitan, do you need to manually extract some image features or just automatically extract features from the image?
", "votes": null, "replies": [] }, { "id": 121043, "author_name": "vicensgaitan", "author_url": "", "post_date": "05/23/2016 07:29:00", "content": "Until now, all steps are fully automated: no hand labeling, no image annotation, no external data (no need to georeference the images in Google Earth). The relevant image features are automagicaly selected by the method.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "120611": "Just wondering so we can make our own *\"unofficial\"* public/private LB there for pure ML solutions (filtering out hand labeled submissions) and competitors who are here to learn (and prefer understanding where they are at against other pure ML competitors).\r\n\r\nNot that hand labeling is bad, but it is easy to get >0.80 using hand labeling (on the train set, to train yourself). Especially with the Latitude/Longitude given for about 80% of the test data (and nearly 100% of train data), it is feasible to get much better than that score in the public/private LB (and it skews the top LB). But how ML will go would be interesting to see.\r\n\r\nHand labelers who have issues ordering the images: use an ImageJ stack, register the images, and start panning fast enough so you can get hands on the differences and start sorting the images out. And/Or use online services (ex: MapBox, GIS free services, etc.).", "120716": "I am using a pure ML solution.", "120724": "Pure ML too. Not using any external data at this point either.", "120755": "I am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually.", "120791": "[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?", "120796": "[quote=thakadu;120791]\r\n\r\n[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?\r\n\r\n\r\n[/quote]\r\n\r\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.", "120797": "[quote=Tyler Vigen;120796]\r\n\r\n[quote=thakadu;120791]\r\n\r\n[quote=olegpolivin;120755]\r\n\r\nI am using a pure ML solution for the score on the Public Leaderboard with no external data. However, I know about 30 images that are not reflected in the public leaderboard, and I coded them manually. \r\n\r\n[/quote]\r\nMay I ask how you know *which* ones are not reflected on the PL?\r\n\r\n\r\n[/quote]\r\n\r\nIt sounds like he's talking about the ones in Long Beach that overlap with the training sets.\r\n\r\n[/quote]\r\n\r\nI think so.\r\nAnd just by noticing that changing the order of these pictures in submission does not change the PL score.", "120829": "I'm doing ML only. I should have something up in the next couple of days.", "120847": "I was using ML-only solution and failed miserably. At least, I learned a lot along the process.", "120979": "Pure ML solution end to end", "120980": "Your score is amazing @Vicens Gaitan. In my opinion it's much better than the hand labeling approach ;) Congrats!", "120987": "Thanks Ole, in my opinion there is a lot of room for improvement for machine learning solutions. I expect to see ML models in the .9x by the end of the competition.", "121032": "Gaitan, do you need to manually extract some image features or just automatically extract features from the image?", "121043": "Until now, all steps are fully automated: no hand labeling, no image annotation, no external data (no need to georeference the images in Google Earth). The relevant image features are automagicaly selected by the method." }, "source": "meta" }