![\"leaky\"](\"http://noahtravisphillips.com/HTMLCSSNTP/Weather3ntp/img/leaky%20bucket%20dollar%20ntp.gif\")
I assumed the ids were sorted after looking at them and never put them in memory (bad practice!) . I was looping the predictions' array to generate the index for submission.
\n\nI post to make certain people dont waste time with the same mistake :).
\n\nE.g
\n\n18790435,0\n18790436,0\n18790437,0\n18790439,0<-----------------\n18790438,0 \nThat was tricky because you will get a decline in score, but not enough to understand that there is problem with the ordering.
", "messages": [ { "id": 308299, "postDate": "2018-04-03T09:52:23.127Z", "content": "I assumed the ids were sorted after looking at them and never put them in memory (bad practice!) . I was looping the predictions' array to generate the index for submission.
\n\nI post to make certain people dont waste time with the same mistake :).
\n\nE.g
\n\n18790435,0\n18790436,0\n18790437,0\n18790439,0<-----------------\n18790438,0 \nThat was tricky because you will get a decline in score, but not enough to understand that there is problem with the ordering.
", "rawMarkdown": "I assumed the ids were sorted after looking at them and never put them in memory (bad practice!) . I was looping the predictions' array to generate the index for submission. \n\nI post to make certain people dont waste time with the same mistake :).\n\nE.g\n\n 18790435,0\n 18790436,0\n 18790437,0\n 18790439,0<-----------------\n 18790438,0 \n\nThat was tricky because you will get a decline in score, but not enough to understand that there is problem with the ordering. ", "votes": 29 }, { "id": 308379, "postDate": "2018-04-03T12:46:20.080Z", "content": "TakingData competitions have a great reputation for creative use of IDs for predictive purposes. It’s one of the less well understood and underappreciated ML techniques. I look forward to finding out what fun surprises this one has in store. Hopefully something major will be revealed in the last week of the competition.
", "rawMarkdown": "TakingData competitions have a great reputation for creative use of IDs for predictive purposes. It’s one of the less well understood and underappreciated ML techniques. I look forward to finding out what fun surprises this one has in store. Hopefully something major will be revealed in the last week of the competition.", "votes": 8, "replies": [ { "id": 308382, "postDate": "2018-04-03T12:57:08.213Z", "content": "I found one already: https://www.kaggle.com/cpmpml/ip-download-rates after it was hinted at: https://www.kaggle.com/yuliagm/be-careful-about-ips-as-a-signal
\n\nIt was further refined: https://www.kaggle.com/its7171/ip-encoding-looks-ok-data-is-clean
", "rawMarkdown": "I found one already: https://www.kaggle.com/cpmpml/ip-download-rates after it was hinted at: https://www.kaggle.com/yuliagm/be-careful-about-ips-as-a-signal\n\nIt was further refined: https://www.kaggle.com/its7171/ip-encoding-looks-ok-data-is-clean", "votes": 2 }, { "id": 308388, "postDate": "2018-04-03T13:01:44.400Z", "content": "CPMP, Bojan doesn't say IP but ID:) I think if there was a leak on ID, it could be found already and we could see people with very high scores.
", "rawMarkdown": "CPMP, Bojan doesn't say IP but ID:) I think if there was a leak on ID, it could be found already and we could see people with very high scores.", "votes": 9 }, { "id": 308401, "postDate": "2018-04-03T13:06:37.033Z", "content": "@AhmetErdem, thanks. My bad. There is no id in the training data however.
", "rawMarkdown": "@AhmetErdem, thanks. My bad. There is no id in the training data however." }, { "id": 308432, "postDate": "2018-04-03T14:09:24.170Z", "content": "\n\n\nI think if there was a leak on ID, it could be found already and we could see people with very high scores.
\n
That's exactly something that someone who's already found a leak would say.
", "rawMarkdown": ">I think if there was a leak on ID, it could be found already and we could see people with very high scores.\n\nThat's **exactly** something that someone who's already found a leak would say. \n\n", "votes": 3 }, { "id": 308452, "postDate": "2018-04-03T14:34:43.210Z", "content": "Since you noticed, I will reveal it. (IP % 27) gives you 0.9806 already. The other leak which makes it 0.9807, I will keep it for myself.
", "rawMarkdown": "Since you noticed, I will reveal it. (IP % 27) gives you 0.9806 already. The other leak which makes it 0.9807, I will keep it for myself.", "votes": 11 }, { "id": 308486, "postDate": "2018-04-03T15:08:59.033Z", "content": "
@Bojan: paranoia - excellent survival trait
\n\nhttp://cheezburger.com/8425753600
", "rawMarkdown": "@Bojan: paranoia - excellent survival trait\n\nhttp://cheezburger.com/8425753600", "votes": 2 }, { "id": 308494, "postDate": "2018-04-03T15:25:06.970Z", "content": "@Konrad - I think you are just trying to throw me off and confuse me with that comment.
", "rawMarkdown": "@Konrad - I think you are just trying to throw me off and confuse me with that comment. ", "votes": 1 }, { "id": 308504, "postDate": "2018-04-03T15:49:35.580Z", "content": "Here I am, trying to give moral support to a fellow paranoid - and this is the thanks I get... Truly, no good deed goes unpunished :-)
", "rawMarkdown": "Here I am, trying to give moral support to a fellow paranoid - and this is the thanks I get... Truly, no good deed goes unpunished :-)", "votes": 1 }, { "id": 308524, "postDate": "2018-04-03T16:30:26.180Z", "content": "Hi @AhmetErdem, could we have more explanation on (IP % 27)? Is it a joke or do you mean there is a leakage feature which value is IP mod 27? I just tried IP mod 27 and found the distribution of is_attributed is almost the same with each (IP mod 27). Thank you.
", "rawMarkdown": " Hi @AhmetErdem, could we have more explanation on (IP % 27)? Is it a joke or do you mean there is a leakage feature which value is IP mod 27? I just tried IP mod 27 and found the distribution of is_attributed is almost the same with each (IP mod 27). Thank you.\n", "votes": 2 }, { "id": 308525, "postDate": "2018-04-03T16:35:03.820Z", "content": "I think that was a joke.
\n\nIn any case, I would not rule out a leak.
", "rawMarkdown": "I think that was a joke. \n\nIn any case, I would not rule out a leak. ", "votes": 3 }, { "id": 308540, "postDate": "2018-04-03T17:01:50.980Z", "content": "Hi @Snorlax, I am really sorry for making you try that. I just wanted to answer @Bojan's joke with a joke:)
", "rawMarkdown": "Hi @Snorlax, I am really sorry for making you try that. I just wanted to answer @Bojan's joke with a joke:)", "votes": 1 }, { "id": 308571, "postDate": "2018-04-03T17:35:32.870Z", "content": "Hi @AhmetErdem, no need to say sorry. I volunteered to try that, lol.
", "rawMarkdown": "Hi @AhmetErdem, no need to say sorry. I volunteered to try that, lol.", "votes": 1 }, { "id": 308588, "postDate": "2018-04-03T17:57:53.603Z", "content": "\n\n\nI just wanted to answer @Bojan's joke with a joke:)
\n
Or is it? Hmmmm ...
", "rawMarkdown": "> I just wanted to answer @Bojan's joke with a joke:)\n\nOr is it? Hmmmm ..." }, { "id": 309147, "postDate": "2018-04-04T18:13:26.793Z", "content": "@KazAnova: Not sure if there is a leak or not but there are some magical features for sure. Someone from top teams already mentioned that with only 4 crafted features they could achieve 0.976x
", "rawMarkdown": "@KazAnova: Not sure if there is a leak or not but there are some magical features for sure. Someone from top teams already mentioned that with only 4 crafted features they could achieve 0.976x", "votes": 1 }, { "id": 309195, "postDate": "2018-04-04T19:32:45.207Z", "content": "I don't think you need a leak to explain that. With buggy data prep I could get 0.9736 with a single model and few features only.
\n\nMaybe I'll find that my bug is uncovering a leak when I'll see that with fixed data prep my score will be way lower....
", "rawMarkdown": "I don't think you need a leak to explain that. With buggy data prep I could get 0.9736 with a single model and few features only.\n\nMaybe I'll find that my bug is uncovering a leak when I'll see that with fixed data prep my score will be way lower...." } ] }, { "id": 308320, "postDate": "2018-04-03T10:42:13.553Z", "content": "People are bumping into this again and again, see for instance https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/52354#298395
", "rawMarkdown": "People are bumping into this again and again, see for instance https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/52354#298395", "votes": 1, "replies": [ { "id": 308333, "postDate": "2018-04-03T11:15:40.280Z", "content": "Yeah - I missed that :(
\n\nWorth making its own thread I think. Very misleading! Kaggle should have highlighted this, because on a quick glance it looks sorted.
", "rawMarkdown": "Yeah - I missed that :(\n\nWorth making its own thread I think. Very misleading! Kaggle should have highlighted this, because on a quick glance it looks sorted.", "votes": 2 }, { "id": 308338, "postDate": "2018-04-03T11:28:04.537Z", "content": "I agree it is worth a thread. And coming from you people will read it!
\n\nWhen I tried to explain it to someone in this forum I was dismissed... ;)
", "rawMarkdown": "I agree it is worth a thread. And coming from you people will read it!\n\nWhen I tried to explain it to someone in this forum I was dismissed... ;)", "votes": 1 } ] }, { "id": 308303, "postDate": "2018-04-03T10:00:49.157Z", "content": "It's probably my caffeine deficit speaking, but I don't see how it can impact the score - the only case I can think of is if you are trying to combine the predictions on the holdout set (for stacking or sth) and concatenate them as opposed to joining.
", "rawMarkdown": "It's probably my caffeine deficit speaking, but I don't see how it can impact the score - the only case I can think of is if you are trying to combine the predictions on the holdout set (for stacking or sth) and concatenate them as opposed to joining.", "replies": [ { "id": 308305, "postDate": "2018-04-03T10:04:39.150Z", "content": "Or...
\n\nwithout changing the ordering of the test data and you get a preds array that hold your predictions and you generate submission like I did!
for i in range (0,len(preds)):\n file.write(str(i) + \",\" + str(preds[i]) )\nWhen I say score , I mean the AUC
Ok, so I wasn't that far off :-)
", "rawMarkdown": "Ok, so I wasn't that far off :-)", "votes": 1 } ] }, { "id": 309165, "postDate": "2018-04-04T18:52:55.237Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 308379, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-04-03T12:46:20.080000", "content": "TakingData competitions have a great reputation for creative use of IDs for predictive purposes. It’s one of the less well understood and underappreciated ML techniques. I look forward to finding out what fun surprises this one has in store. Hopefully something major will be revealed in the last week of the competition.
", "votes": 8, "replies": [ { "id": 308382, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-03T12:57:08.213000", "content": "I found one already: https://www.kaggle.com/cpmpml/ip-download-rates after it was hinted at: https://www.kaggle.com/yuliagm/be-careful-about-ips-as-a-signal
\n\nIt was further refined: https://www.kaggle.com/its7171/ip-encoding-looks-ok-data-is-clean
", "votes": 2, "replies": [] }, { "id": 308388, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2018-04-03T13:01:44.400000", "content": "CPMP, Bojan doesn't say IP but ID:) I think if there was a leak on ID, it could be found already and we could see people with very high scores.
", "votes": 9, "replies": [] }, { "id": 308401, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-03T13:06:37.033000", "content": "@AhmetErdem, thanks. My bad. There is no id in the training data however.
", "votes": 0, "replies": [] }, { "id": 308432, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-04-03T14:09:24.170000", "content": "\n\n\nI think if there was a leak on ID, it could be found already and we could see people with very high scores.
\n
That's exactly something that someone who's already found a leak would say.
", "votes": 3, "replies": [] }, { "id": 308452, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2018-04-03T14:34:43.210000", "content": "Since you noticed, I will reveal it. (IP % 27) gives you 0.9806 already. The other leak which makes it 0.9807, I will keep it for myself.
", "votes": 11, "replies": [] }, { "id": 308486, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-04-03T15:08:59.033000", "content": "
@Bojan: paranoia - excellent survival trait
\n\nhttp://cheezburger.com/8425753600
", "votes": 2, "replies": [] }, { "id": 308494, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-04-03T15:25:06.970000", "content": "@Konrad - I think you are just trying to throw me off and confuse me with that comment.
", "votes": 1, "replies": [] }, { "id": 308504, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2018-04-03T15:49:35.580000", "content": "Here I am, trying to give moral support to a fellow paranoid - and this is the thanks I get... Truly, no good deed goes unpunished :-)
", "votes": 1, "replies": [] }, { "id": 308524, "author_name": "Snorlax", "author_url": "", "post_date": "2018-04-03T16:30:26.180000", "content": "Hi @AhmetErdem, could we have more explanation on (IP % 27)? Is it a joke or do you mean there is a leakage feature which value is IP mod 27? I just tried IP mod 27 and found the distribution of is_attributed is almost the same with each (IP mod 27). Thank you.
", "votes": 2, "replies": [] }, { "id": 308525, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2018-04-03T16:35:03.820000", "content": "I think that was a joke.
\n\nIn any case, I would not rule out a leak.
", "votes": 3, "replies": [] }, { "id": 308540, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2018-04-03T17:01:50.980000", "content": "Hi @Snorlax, I am really sorry for making you try that. I just wanted to answer @Bojan's joke with a joke:)
", "votes": 1, "replies": [] }, { "id": 308571, "author_name": "Snorlax", "author_url": "", "post_date": "2018-04-03T17:35:32.870000", "content": "Hi @AhmetErdem, no need to say sorry. I volunteered to try that, lol.
", "votes": 1, "replies": [] }, { "id": 308588, "author_name": "Bojan Tunguz", "author_url": "", "post_date": "2018-04-03T17:57:53.603000", "content": "\n\n\nI just wanted to answer @Bojan's joke with a joke:)
\n
Or is it? Hmmmm ...
", "votes": 0, "replies": [] }, { "id": 309147, "author_name": "Sameh Faidi", "author_url": "", "post_date": "2018-04-04T18:13:26.793000", "content": "@KazAnova: Not sure if there is a leak or not but there are some magical features for sure. Someone from top teams already mentioned that with only 4 crafted features they could achieve 0.976x
", "votes": 1, "replies": [] }, { "id": 309195, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-04T19:32:45.207000", "content": "I don't think you need a leak to explain that. With buggy data prep I could get 0.9736 with a single model and few features only.
\n\nMaybe I'll find that my bug is uncovering a leak when I'll see that with fixed data prep my score will be way lower....
", "votes": 0, "replies": [] } ] }, { "id": 308320, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-03T10:42:13.553000", "content": "People are bumping into this again and again, see for instance https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/52354#298395
", "votes": 1, "replies": [ { "id": 308333, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2018-04-03T11:15:40.280000", "content": "Yeah - I missed that :(
\n\nWorth making its own thread I think. Very misleading! Kaggle should have highlighted this, because on a quick glance it looks sorted.
", "votes": 2, "replies": [] }, { "id": 308338, "author_name": "CPMP", "author_url": "", "post_date": "2018-04-03T11:28:04.537000", "content": "I agree it is worth a thread. And coming from you people will read it!
\n\nWhen I tried to explain it to someone in this forum I was dismissed... ;)
", "votes": 1, "replies": [] } ] }, { "id": 308303, "author_name": "Konrad Banachewicz", "author_url": "", "post_date": "2018-04-03T10:00:49.157000", "content": "It's probably my caffeine deficit speaking, but I don't see how it can impact the score - the only case I can think of is if you are trying to combine the predictions on the holdout set (for stacking or sth) and concatenate them as opposed to joining.
", "votes": 0, "replies": [ { "id": 308305, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2018-04-03T10:04:39.150000", "content": "Or...
\n\nwithout changing the ordering of the test data and you get a preds array that hold your predictions and you generate submission like I did!
for i in range (0,len(preds)):\n file.write(str(i) + \",\" + str(preds[i]) )\nWhen I say score , I mean the AUC
Ok, so I wasn't that far off :-)
", "votes": 1, "replies": [] } ] }, { "id": 309165, "author_name": "", "author_url": "", "post_date": "2018-04-04T18:52:55.237000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "308299": "I assumed the ids were sorted after looking at them and never put them in memory (bad practice!) . I was looping the predictions' array to generate the index for submission. \n\nI post to make certain people dont waste time with the same mistake :).\n\nE.g\n\n 18790435,0\n 18790436,0\n 18790437,0\n 18790439,0<-----------------\n 18790438,0 \n\nThat was tricky because you will get a decline in score, but not enough to understand that there is problem with the ordering. ", "308379": "TakingData competitions have a great reputation for creative use of IDs for predictive purposes. It’s one of the less well understood and underappreciated ML techniques. I look forward to finding out what fun surprises this one has in store. Hopefully something major will be revealed in the last week of the competition.", "308320": "People are bumping into this again and again, see for instance https://www.kaggle.com/c/talkingdata-adtracking-fraud-detection/discussion/52354#298395", "308303": "It's probably my caffeine deficit speaking, but I don't see how it can impact the score - the only case I can think of is if you are trying to combine the predictions on the holdout set (for stacking or sth) and concatenate them as opposed to joining.", "309165": "" } }