{ "id": 20684, "title": "A tutorial to get you started", "url": "/competitions/expedia-hotel-recommendations/discussion/20684", "author_name": "", "post_date": "2016-05-03T20:24:43.177Z", "votes": 112, "comment_count": 40, "views": 8757, "content": "

I wrote a tutorial on how to get into the top 15 on the leaderboard in the competition (as of now): https://www.dataquest.io/blog/kaggle-tutorial/ . Hope you enjoy it!

", "messages": [ { "id": "118471", "postDate": "05/03/2016 20:24:43", "content": "

I wrote a tutorial on how to get into the top 15 on the leaderboard in the competition (as of now): https://www.dataquest.io/blog/kaggle-tutorial/ . Hope you enjoy it!

", "rawMarkdown": "I wrote a tutorial on how to get into the top 15 on the leaderboard in the competition (as of now): https://www.dataquest.io/blog/kaggle-tutorial/ . Hope you enjoy it!", "votes": null }, { "id": "118474", "postDate": "05/03/2016 20:57:43", "content": "

Thanks! I wish you had used a better language [R :) ] but I will enjoy working through your examples and reasoning. Thanks for teaching us!

", "rawMarkdown": "Thanks! I wish you had used a better language [R :) ] but I will enjoy working through your examples and reasoning. Thanks for teaching us!", "votes": null }, { "id": "118503", "postDate": "05/04/2016 01:42:59", "content": "

Very detailed and well explained.. I really enjoyed reading through it and definitely learned more than one thing or two from the material. Thanks for the tutorial!

", "rawMarkdown": "Very detailed and well explained.. I really enjoyed reading through it and definitely learned more than one thing or two from the material. Thanks for the tutorial!", "votes": null }, { "id": "118544", "postDate": "05/04/2016 07:45:09", "content": "

Very nice tutorial. Thank you for sharing your knowledge.

", "rawMarkdown": "Very nice tutorial. Thank you for sharing your knowledge.", "votes": null }, { "id": "118557", "postDate": "05/04/2016 09:03:25", "content": "

I wish there was a post like this in every competition :)

", "rawMarkdown": "I wish there was a post like this in every competition :)", "votes": null }, { "id": "118564", "postDate": "05/04/2016 09:35:47", "content": "

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

", "rawMarkdown": "Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?", "votes": null }, { "id": "118573", "postDate": "05/04/2016 10:16:42", "content": "

@Vik Thank you very much

\n\n

With a very similar (and in times more granular) approach in perl I made 0.47x. Will surely look into your solution.

\n\n

Can anyone tell how much memory is needed for this script?

\n\n

Thanks

\n\n

Gerhard

", "rawMarkdown": "Vik Thank you very much\r\n\r\nWith a very similar (and in times more granular) approach in perl I made 0.47x. Will surely look into your solution.\r\n\r\nCan anyone tell how much memory is needed for this script?\r\n\r\nThanks\r\n\r\nGerhard", "votes": null }, { "id": "118611", "postDate": "05/04/2016 13:51:39", "content": "

One of the easier to follow and understand tutorials that I have read, awesome job!

", "rawMarkdown": "One of the easier to follow and understand tutorials that I have read, awesome job!", "votes": null }, { "id": "118623", "postDate": "05/04/2016 14:40:31", "content": "

[quote=Konrad Banachewicz;118564]

\n\n

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

\n\n

[/quote]

\n\n

Thanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.

", "rawMarkdown": "[quote=Konrad Banachewicz;118564]\r\n\r\n@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb? \r\n\r\n[/quote]\r\n\r\nThanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.", "votes": null }, { "id": "118626", "postDate": "05/04/2016 14:46:16", "content": "

[quote=Vik Paruchuri;118623]

\n\n

[quote=Konrad Banachewicz;118564]

\n\n

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

\n\n

[/quote]

\n\n

Thanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.

\n\n

[/quote]

\n\n

Thank you Vik! Very helpful!

\n\n

I'm doing a much more careless validation but on 670k data. My local score is about 0.01 higher than lb.

", "rawMarkdown": "[quote=Vik Paruchuri;118623]\r\n\r\n[quote=Konrad Banachewicz;118564]\r\n\r\n@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb? \r\n\r\n[/quote]\r\n\r\nThanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.\r\n\r\n[/quote]\r\n\r\nThank you Vik! Very helpful!\r\n\r\nI'm doing a much more careless validation but on 670k data. My local score is about 0.01 higher than lb.", "votes": null }, { "id": "118628", "postDate": "05/04/2016 14:47:08", "content": "

@Vik: that's interesting - and potentially another part of the challenge. I suspect (didnt have time to verify it yet) it might have sth to do with the subsampling, i.e. the outcome is quite sensitive to which 10000 are sampled.

", "rawMarkdown": "Vik: that's interesting - and potentially another part of the challenge. I suspect (didnt have time to verify it yet) it might have sth to do with the subsampling, i.e. the outcome is quite sensitive to *which* 10000 are sampled.", "votes": null }, { "id": "118636", "postDate": "05/04/2016 15:04:32", "content": "

@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :).

\n\n

The only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).

", "rawMarkdown": "Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :). \r\n\r\nThe only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).", "votes": null }, { "id": "118637", "postDate": "05/04/2016 15:06:58", "content": "

@rcarson , but using the same approach or you actually used some ML?

\n\n

How would you suggest splitting the set into train/test? \nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?

", "rawMarkdown": "rcarson , but using the same approach or you actually used some ML?\r\n\r\nHow would you suggest splitting the set into train/test? \r\nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?", "votes": null }, { "id": "118638", "postDate": "05/04/2016 15:14:17", "content": "

[quote=Marcin Pękalski;118637]

\n\n

@rcarson , but using the same approach or you actually used some ML?

\n\n

How would you suggest splitting the set into train/test? \nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?

\n\n

[/quote]

\n\n

We're simply using the last 670k data (very arbitrary choice) in train for validation. But Vik's way (by dates) makes more sense, you just need more data in validation to reduce local-lb difference.

\n\n

Our ml is still a little worse than our rule-based model but we are getting there.

", "rawMarkdown": "[quote=Marcin Pękalski;118637]\r\n\r\n@rcarson , but using the same approach or you actually used some ML?\r\n\r\nHow would you suggest splitting the set into train/test? \r\nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?\r\n\r\n[/quote]\r\n\r\nWe're simply using the last 670k data (very arbitrary choice) in train for validation. But Vik's way (by dates) makes more sense, you just need more data in validation to reduce local-lb difference.\r\n\r\nOur ml is still a little worse than our rule-based model but we are getting there.", "votes": null }, { "id": "118644", "postDate": "05/04/2016 15:32:49", "content": "

[quote=dune_dweller;118636]

\n\n

...If logistic regression doesn't work here the reason lies somewhere else :).

\n\n

[/quote]

\n\n

Agreed. I guess the reason is that there are too many unique values for each feature and their interactions. To accommodate all the features, the logistic regression will have to store and update a huge number of parameters - which means a memory problem. If we try to hash down the number of features, random hash collisions would be unavoidable, and hence the model would be compromised.

", "rawMarkdown": "[quote=dune_dweller;118636]\r\n\r\n...If logistic regression doesn't work here the reason lies somewhere else :).\r\n\r\n[/quote]\r\n\r\nAgreed. I guess the reason is that there are too many unique values for each feature and their interactions. To accommodate all the features, the logistic regression will have to store and update a huge number of parameters - which means a memory problem. If we try to hash down the number of features, random hash collisions would be unavoidable, and hence the model would be compromised.", "votes": null }, { "id": "118646", "postDate": "05/04/2016 15:41:07", "content": "

Hi,

\n\n

so. After work I ran the first lines in python and RAM used max was nearly 30GB. Dropping to ~21GB after load finished.

\n\n

Cheers

\n\n

Gerhard

", "rawMarkdown": "Hi,\r\n\r\nso. After work I ran the first lines in python and RAM used max was nearly 30GB. Dropping to ~21GB after load finished.\r\n\r\nCheers\r\n\r\nGerhard", "votes": null }, { "id": "118649", "postDate": "05/04/2016 15:46:35", "content": "

@MightyBird, try changing data types from 64bit to 32bit. That big precision is not needed, and the RAM requirements should go down a bit. Just remember to use garbage collector afterwards, to force the cleaning.

", "rawMarkdown": "MightyBird, try changing data types from 64bit to 32bit. That big precision is not needed, and the RAM requirements should go down a bit. Just remember to use garbage collector afterwards, to force the cleaning.", "votes": null }, { "id": "118661", "postDate": "05/04/2016 16:44:52", "content": "

@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?

", "rawMarkdown": "Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?", "votes": null }, { "id": "118672", "postDate": "05/04/2016 17:20:02", "content": "

Thank you Vik! Definitely very useful for a beginner Kaggler like me :)

", "rawMarkdown": "Thank you Vik! Definitely very useful for a beginner Kaggler like me :)", "votes": null }, { "id": "118693", "postDate": "05/04/2016 19:31:30", "content": "

A new benchmark :)

", "rawMarkdown": "A new benchmark :)", "votes": null }, { "id": "118696", "postDate": "05/04/2016 19:45:47", "content": "

Thanks for the great tutorial. \nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.

", "rawMarkdown": "Thanks for the great tutorial. \r\nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.", "votes": null }, { "id": "118697", "postDate": "05/04/2016 19:50:21", "content": "

[quote=Mohamed Ali Jamaoui;118696]

\n\n

Thanks for the great tutorial. \nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.

\n\n

[/quote]

\n\n

The data section states that the test set only consists of booking events.

", "rawMarkdown": "[quote=Mohamed Ali Jamaoui;118696]\r\n\r\nThanks for the great tutorial. \r\nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard. \r\n\r\n[/quote]\r\n\r\nThe data section states that the test set only consists of booking events.", "votes": null }, { "id": "118712", "postDate": "05/04/2016 21:38:38", "content": "

Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.

", "rawMarkdown": "Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.", "votes": null }, { "id": "118729", "postDate": "05/05/2016 00:57:36", "content": "

Sorry for the double post, but I did want to send an alert to everyone following the thread.

\n\n

For me the killer code was this

\n\n
for i in range(test.shape[0]):\n     exact_matches.append(generate_exact_matches(test.iloc[i], match_cols))\n
\n\n

after about an hour+ I was only at 100k entries.

\n\n

I think a panda merge (pd.merge)--basically an SQL left join for those of us in database world, might be more efficient here for those of us with lesser hardware. I'll try and figure it out in a bit--its been a day and a half unrelated to kaggle, but I do want to contribute and make the script a bit more accessible.

", "rawMarkdown": "Sorry for the double post, but I did want to send an alert to everyone following the thread. \r\n\r\nFor me the killer code was this\r\n\r\n for i in range(test.shape[0]):\r\n exact_matches.append(generate_exact_matches(test.iloc[i], match_cols))\r\n\r\nafter about an hour+ I was only at 100k entries.\r\n\r\nI think a panda merge (pd.merge)--basically an SQL left join for those of us in database world, might be more efficient here for those of us with lesser hardware. I'll try and figure it out in a bit--its been a day and a half unrelated to kaggle, but I do want to contribute and make the script a bit more accessible.", "votes": null }, { "id": "118742", "postDate": "05/05/2016 04:03:49", "content": "

Great script Vik and very well written article!

\n\n

One critique - you should probably join the destinations dataset to the training data (or a sample of the training data) before you run PCA instead of running PCA on the destinations dataset in isolation. The distribution of destinations within the samples could have an effect on the variance within the features.

", "rawMarkdown": "Great script Vik and very well written article!\r\n\r\nOne critique - you should probably join the destinations dataset to the training data (or a sample of the training data) *before* you run PCA instead of running PCA on the destinations dataset in isolation. The distribution of destinations within the samples could have an effect on the variance within the features.", "votes": null }, { "id": "118753", "postDate": "05/05/2016 06:17:39", "content": "

If you really take a look at the data in there, many of the rows have duplicate responses for one of the values. They just look like fills with one value from what I saw in the first few rows. Even doing a transform with Rtsne noted "duplicates". To be succinct - I highly doubt that each destination was reviewed 150 times. In that highly unlikely situation, I also doubt that many got a same sentiment value (derived from solely from text analysis, if not a weighted average of rating and sentiment analysis) several times.

\n\n

The unique number of values for each row will differ if you look closely. I fear that doing a linear (or otherwise) transform, like PCA, on these values may not be the best choice of action without prior cleaning.

\n\n

Since I'm unsure of whether the mean is being used as the duplicate value for each row, it may be good to use that as the fill, instead of the value that's being displayed in each row.

\n\n

Excellent script though.

", "rawMarkdown": "If you really take a look at the data in there, many of the rows have duplicate responses for one of the values. They just look like fills with one value from what I saw in the first few rows. Even doing a transform with Rtsne noted \"duplicates\". To be succinct - I highly doubt that each destination was reviewed 150 times. In that highly unlikely situation, I also doubt that many got a same sentiment value (derived from solely from text analysis, if not a weighted average of rating and sentiment analysis) several times.\r\n\r\nThe unique number of values for each row will differ if you look closely. I fear that doing a linear (or otherwise) transform, like PCA, on these values may not be the best choice of action without prior cleaning.\r\n\r\nSince I'm unsure of whether the mean is being used as the duplicate value for each row, it may be good to use that as the fill, instead of the value that's being displayed in each row.\r\n\r\nExcellent script though.", "votes": null }, { "id": "118844", "postDate": "05/05/2016 16:55:24", "content": "

[quote=Bob, Lord of Cats;118729]\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\n[/quote]

\n\n

[quote=Bob, Lord of Cats;118729]\nafter about an hour+ I was only at 100k entries.\n[/quote]

\n\n

He actually mentions in the post:

\n\n

[quote]\nGiven the amount of memory on your system, it may or may not be feasible to read all the data in. If it isn’t, you should consider creating a machine on EC2 or DigitalOcean to process the data with.[/quote]

\n\n

and then links to this post https://www.dataquest.io/blog/data-science-ec2-quickstart/

\n\n

Obviously there's a cost involved here, but if it helps you iterate faster and doesn't lock up your machine, $30 doesn't seem like that much to me.

", "rawMarkdown": "[quote=Bob, Lord of Cats;118729]\r\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\r\n[/quote]\r\n\r\n[quote=Bob, Lord of Cats;118729]\r\nafter about an hour+ I was only at 100k entries.\r\n[/quote]\r\n\r\nHe actually mentions in the post:\r\n\r\n[quote]\r\nGiven the amount of memory on your system, it may or may not be feasible to read all the data in. If it isn’t, you should consider creating a machine on EC2 or DigitalOcean to process the data with.[/quote]\r\n\r\nand then links to this post https://www.dataquest.io/blog/data-science-ec2-quickstart/\r\n\r\nObviously there's a cost involved here, but if it helps you iterate faster and doesn't lock up your machine, $30 doesn't seem like that much to me.", "votes": null }, { "id": "118845", "postDate": "05/05/2016 17:02:49", "content": "

@Bob, Lord of Cats \nTry to run this version. The killer code you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)

", "rawMarkdown": "Bob, Lord of Cats \r\nTry to run this [version][1]. The *killer code* you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)\r\n\r\n\r\n [1]: https://www.kaggle.com/manels/expedia-hotel-recommendations/dataquest-tutorial", "votes": null }, { "id": "118847", "postDate": "05/05/2016 17:07:20", "content": "

[quote=Bob, Lord of Cats;118712]

\n\n

Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.

\n\n

[/quote]

\n\n

It took about an hour to run on my machine (16gb RAM, though). Maybe a library or system package version difference? Also, the ML code will be impractical to run on the whole training / test set, so that should be left out.

", "rawMarkdown": "[quote=Bob, Lord of Cats;118712]\r\n\r\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\r\n\r\n[/quote]\r\n\r\nIt took about an hour to run on my machine (16gb RAM, though). Maybe a library or system package version difference? Also, the ML code will be impractical to run on the whole training / test set, so that should be left out.", "votes": null }, { "id": "118848", "postDate": "05/05/2016 17:09:52", "content": "

[quote=dune_dweller;118636]

\n\n

@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :).

\n\n

The only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).

\n\n

[/quote]

\n\n

Yeah, I should have explained this better -- it comes down more to not being able to linearly separate the clusters with the given features because the boundaries are so fuzzy.

", "rawMarkdown": "[quote=dune_dweller;118636]\r\n\r\n@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :). \r\n\r\nThe only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).\r\n\r\n[/quote]\r\n\r\nYeah, I should have explained this better -- it comes down more to not being able to linearly separate the clusters with the given features because the boundaries are so fuzzy.", "votes": null }, { "id": "118849", "postDate": "05/05/2016 17:10:39", "content": "

[quote=MelvinDunn;118661]

\n\n

@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?

\n\n

[/quote]

\n\n

Please do! :)

", "rawMarkdown": "[quote=MelvinDunn;118661]\r\n\r\n@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard? \r\n\r\n[/quote]\r\n\r\nPlease do! :)", "votes": null }, { "id": "118851", "postDate": "05/05/2016 17:29:37", "content": "

@Vik: Excellent tutorial! I also confirm that 100 x binary classification does not work here, I created 100 Online Logistic models using feature hashing (with the entire training dataset) and the resulting precision was the same (about 0.05) , in addition the AUC/LogLoss metrics for each classifier were about 0.68/0.29.\nWell done!

", "rawMarkdown": "Vik: Excellent tutorial! I also confirm that 100 x binary classification does not work here, I created 100 Online Logistic models using feature hashing (with the entire training dataset) and the resulting precision was the same (about 0.05) , in addition the AUC/LogLoss metrics for each classifier were about 0.68/0.29.\r\nWell done!", "votes": null }, { "id": "118855", "postDate": "05/05/2016 18:23:18", "content": "

@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?

", "rawMarkdown": "Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?", "votes": null }, { "id": "118856", "postDate": "05/05/2016 18:24:00", "content": "

[quote=Manel;118845]

\n\n

@Bob, Lord of Cats \nTry to run this version. The killer code you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)

\n\n

[/quote]\nI'll give it a shot after work. My idea wasn't to complain but improve because I think there is a more efficient way to do it (I actually have been modifying Dune Dweller's code as I couldn't quite get this one to the point I wanted where I could get my join/merge to work). My apologies as in my studies in R/Python/ML it has been pounded into me LOOPS BAD/VECTORIZE YOUR CODE. Plus in the grand scheme of research, grabbing several users ideas, combining them into my own and then have all of you tell me what a genius I am. (I am sort of slow on the combine part, but still... ;) )

\n\n

update: I dropped out of ipython into a straight shell--the improvement for me is significant--running at about 100K samples / 45 sec. Much quicker. Methinks there is an ID:10T error in my notebook config. R simply blows away my ipython notebook for speed.

\n\n

23:09 for the "Leak" section. I knew the old gal had it in her!

", "rawMarkdown": "[quote=Manel;118845]\r\n\r\n@Bob, Lord of Cats \r\nTry to run this [version][1]. The *killer code* you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)\r\n\r\n\r\n [1]: https://www.kaggle.com/manels/expedia-hotel-recommendations/dataquest-tutorial\r\n\r\n[/quote]\r\nI'll give it a shot after work. My idea wasn't to complain but improve because I think there is a more efficient way to do it (I actually have been modifying Dune Dweller's code as I couldn't quite get this one to the point I wanted where I could get my join/merge to work). My apologies as in my studies in R/Python/ML it has been pounded into me LOOPS BAD/VECTORIZE YOUR CODE. Plus in the grand scheme of research, grabbing several users ideas, combining them into my own and then have all of you tell me what a genius I am. (I am sort of slow on the combine part, but still... ;) )\r\n\r\nupdate: I dropped out of ipython into a straight shell--the improvement for me is significant--running at about 100K samples / 45 sec. Much quicker. Methinks there is an ID:10T error in my notebook config. R simply blows away my ipython notebook for speed. \r\n\r\n23:09 for the \"Leak\" section. I knew the old gal had it in her!", "votes": null }, { "id": "118858", "postDate": "05/05/2016 18:44:12", "content": "

[quote=Marcin Pękalski;118855]

\n\n

@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?

\n\n

[/quote]\nHello Marcin, I used basically the raw data excluding the dates, I created a few features (year, month, day of week, total days of stay) using the dates. Regarding passes, just passed the training dataset 2 times.\nNo fine tuning here, but I guess not so much chance for this to work.

", "rawMarkdown": "[quote=Marcin Pękalski;118855]\r\n\r\n@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?\r\n\r\n[/quote]\r\nHello Marcin, I used basically the raw data excluding the dates, I created a few features (year, month, day of week, total days of stay) using the dates. Regarding passes, just passed the training dataset 2 times.\r\nNo fine tuning here, but I guess not so much chance for this to work.", "votes": null }, { "id": "118903", "postDate": "05/06/2016 01:28:07", "content": "

I owe @Vik an apology--the program wasn't the problem--apparently my Jupyter Notebook/IPython shell and chrome do not play well at all (10X or more performance hit). Running from a prompt gives excellent performance (under an hour as advertised).

", "rawMarkdown": "I owe @Vik an apology--the program wasn't the problem--apparently my Jupyter Notebook/IPython shell and chrome do not play well at all (10X or more performance hit). Running from a prompt gives excellent performance (under an hour as advertised).", "votes": null }, { "id": "120832", "postDate": "05/20/2016 20:14:04", "content": "

Glad you got it sorted in the end Bob!

", "rawMarkdown": "Glad you got it sorted in the end Bob!", "votes": null }, { "id": "160460", "postDate": "02/07/2017 19:37:13", "content": "

@Vik many thanks! Following a lot of your tips/tutorials is helping a lot.

", "rawMarkdown": "Vik many thanks! Following a lot of your tips/tutorials is helping a lot.", "votes": null }, { "id": "160461", "postDate": "02/07/2017 19:37:16", "content": "", "rawMarkdown": "", "votes": null }, { "id": "344911", "postDate": "06/18/2018 22:40:18", "content": "

Thanks for the link @Vik

", "rawMarkdown": "Thanks for the link @Vik", "votes": null }, { "id": "3080956", "postDate": "12/26/2024 02:14:38", "content": "

thanks for the tutorial

", "rawMarkdown": "thanks for the tutorial", "votes": null } ], "comments": [ { "id": 3080956, "author_name": "anowersha", "author_url": "", "post_date": "12/26/2024 02:14:38", "content": "

thanks for the tutorial

", "votes": null, "replies": [] }, { "id": 118474, "author_name": "piinformatics", "author_url": "", "post_date": "05/03/2016 20:57:43", "content": "

Thanks! I wish you had used a better language [R :) ] but I will enjoy working through your examples and reasoning. Thanks for teaching us!

", "votes": null, "replies": [] }, { "id": 118503, "author_name": "fernandoprocy", "author_url": "", "post_date": "05/04/2016 01:42:59", "content": "

Very detailed and well explained.. I really enjoyed reading through it and definitely learned more than one thing or two from the material. Thanks for the tutorial!

", "votes": null, "replies": [] }, { "id": 118544, "author_name": "aikinogard", "author_url": "", "post_date": "05/04/2016 07:45:09", "content": "

Very nice tutorial. Thank you for sharing your knowledge.

", "votes": null, "replies": [] }, { "id": 118557, "author_name": "vykhand", "author_url": "", "post_date": "05/04/2016 09:03:25", "content": "

I wish there was a post like this in every competition :)

", "votes": null, "replies": [] }, { "id": 118564, "author_name": "konradb", "author_url": "", "post_date": "05/04/2016 09:35:47", "content": "

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

", "votes": null, "replies": [] }, { "id": 118573, "author_name": "mightybird", "author_url": "", "post_date": "05/04/2016 10:16:42", "content": "

@Vik Thank you very much

\n\n

With a very similar (and in times more granular) approach in perl I made 0.47x. Will surely look into your solution.

\n\n

Can anyone tell how much memory is needed for this script?

\n\n

Thanks

\n\n

Gerhard

", "votes": null, "replies": [] }, { "id": 118611, "author_name": "nitini", "author_url": "", "post_date": "05/04/2016 13:51:39", "content": "

One of the easier to follow and understand tutorials that I have read, awesome job!

", "votes": null, "replies": [] }, { "id": 118623, "author_name": "vikpar", "author_url": "", "post_date": "05/04/2016 14:40:31", "content": "

[quote=Konrad Banachewicz;118564]

\n\n

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

\n\n

[/quote]

\n\n

Thanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.

", "votes": null, "replies": [] }, { "id": 118626, "author_name": "jiweiliu", "author_url": "", "post_date": "05/04/2016 14:46:16", "content": "

[quote=Vik Paruchuri;118623]

\n\n

[quote=Konrad Banachewicz;118564]

\n\n

@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?

\n\n

[/quote]

\n\n

Thanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.

\n\n

[/quote]

\n\n

Thank you Vik! Very helpful!

\n\n

I'm doing a much more careless validation but on 670k data. My local score is about 0.01 higher than lb.

", "votes": null, "replies": [] }, { "id": 118628, "author_name": "konradb", "author_url": "", "post_date": "05/04/2016 14:47:08", "content": "

@Vik: that's interesting - and potentially another part of the challenge. I suspect (didnt have time to verify it yet) it might have sth to do with the subsampling, i.e. the outcome is quite sensitive to which 10000 are sampled.

", "votes": null, "replies": [] }, { "id": 118636, "author_name": "dvasyukova", "author_url": "", "post_date": "05/04/2016 15:04:32", "content": "

@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :).

\n\n

The only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).

", "votes": null, "replies": [] }, { "id": 118637, "author_name": "mpekalski", "author_url": "", "post_date": "05/04/2016 15:06:58", "content": "

@rcarson , but using the same approach or you actually used some ML?

\n\n

How would you suggest splitting the set into train/test? \nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?

", "votes": null, "replies": [] }, { "id": 118638, "author_name": "jiweiliu", "author_url": "", "post_date": "05/04/2016 15:14:17", "content": "

[quote=Marcin Pękalski;118637]

\n\n

@rcarson , but using the same approach or you actually used some ML?

\n\n

How would you suggest splitting the set into train/test? \nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?

\n\n

[/quote]

\n\n

We're simply using the last 670k data (very arbitrary choice) in train for validation. But Vik's way (by dates) makes more sense, you just need more data in validation to reduce local-lb difference.

\n\n

Our ml is still a little worse than our rule-based model but we are getting there.

", "votes": null, "replies": [] }, { "id": 118644, "author_name": "mandalsubhajit", "author_url": "", "post_date": "05/04/2016 15:32:49", "content": "

[quote=dune_dweller;118636]

\n\n

...If logistic regression doesn't work here the reason lies somewhere else :).

\n\n

[/quote]

\n\n

Agreed. I guess the reason is that there are too many unique values for each feature and their interactions. To accommodate all the features, the logistic regression will have to store and update a huge number of parameters - which means a memory problem. If we try to hash down the number of features, random hash collisions would be unavoidable, and hence the model would be compromised.

", "votes": null, "replies": [] }, { "id": 118646, "author_name": "mightybird", "author_url": "", "post_date": "05/04/2016 15:41:07", "content": "

Hi,

\n\n

so. After work I ran the first lines in python and RAM used max was nearly 30GB. Dropping to ~21GB after load finished.

\n\n

Cheers

\n\n

Gerhard

", "votes": null, "replies": [] }, { "id": 118649, "author_name": "mpekalski", "author_url": "", "post_date": "05/04/2016 15:46:35", "content": "

@MightyBird, try changing data types from 64bit to 32bit. That big precision is not needed, and the RAM requirements should go down a bit. Just remember to use garbage collector afterwards, to force the cleaning.

", "votes": null, "replies": [] }, { "id": 118661, "author_name": "", "author_url": "", "post_date": "05/04/2016 16:44:52", "content": "

@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?

", "votes": null, "replies": [] }, { "id": 118672, "author_name": "dannychan", "author_url": "", "post_date": "05/04/2016 17:20:02", "content": "

Thank you Vik! Definitely very useful for a beginner Kaggler like me :)

", "votes": null, "replies": [] }, { "id": 118693, "author_name": "yangyang", "author_url": "", "post_date": "05/04/2016 19:31:30", "content": "

A new benchmark :)

", "votes": null, "replies": [] }, { "id": 118696, "author_name": "mohamedali", "author_url": "", "post_date": "05/04/2016 19:45:47", "content": "

Thanks for the great tutorial. \nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.

", "votes": null, "replies": [] }, { "id": 118697, "author_name": "brandenkmurray", "author_url": "", "post_date": "05/04/2016 19:50:21", "content": "

[quote=Mohamed Ali Jamaoui;118696]

\n\n

Thanks for the great tutorial. \nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.

\n\n

[/quote]

\n\n

The data section states that the test set only consists of booking events.

", "votes": null, "replies": [] }, { "id": 118712, "author_name": "rdslater", "author_url": "", "post_date": "05/04/2016 21:38:38", "content": "

Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.

", "votes": null, "replies": [] }, { "id": 118729, "author_name": "rdslater", "author_url": "", "post_date": "05/05/2016 00:57:36", "content": "

Sorry for the double post, but I did want to send an alert to everyone following the thread.

\n\n

For me the killer code was this

\n\n
for i in range(test.shape[0]):\n     exact_matches.append(generate_exact_matches(test.iloc[i], match_cols))\n
\n\n

after about an hour+ I was only at 100k entries.

\n\n

I think a panda merge (pd.merge)--basically an SQL left join for those of us in database world, might be more efficient here for those of us with lesser hardware. I'll try and figure it out in a bit--its been a day and a half unrelated to kaggle, but I do want to contribute and make the script a bit more accessible.

", "votes": null, "replies": [] }, { "id": 118742, "author_name": "ben519", "author_url": "", "post_date": "05/05/2016 04:03:49", "content": "

Great script Vik and very well written article!

\n\n

One critique - you should probably join the destinations dataset to the training data (or a sample of the training data) before you run PCA instead of running PCA on the destinations dataset in isolation. The distribution of destinations within the samples could have an effect on the variance within the features.

", "votes": null, "replies": [] }, { "id": 118753, "author_name": "", "author_url": "", "post_date": "05/05/2016 06:17:39", "content": "

If you really take a look at the data in there, many of the rows have duplicate responses for one of the values. They just look like fills with one value from what I saw in the first few rows. Even doing a transform with Rtsne noted "duplicates". To be succinct - I highly doubt that each destination was reviewed 150 times. In that highly unlikely situation, I also doubt that many got a same sentiment value (derived from solely from text analysis, if not a weighted average of rating and sentiment analysis) several times.

\n\n

The unique number of values for each row will differ if you look closely. I fear that doing a linear (or otherwise) transform, like PCA, on these values may not be the best choice of action without prior cleaning.

\n\n

Since I'm unsure of whether the mean is being used as the duplicate value for each row, it may be good to use that as the fill, instead of the value that's being displayed in each row.

\n\n

Excellent script though.

", "votes": null, "replies": [] }, { "id": 118844, "author_name": "jaypeedevlin", "author_url": "", "post_date": "05/05/2016 16:55:24", "content": "

[quote=Bob, Lord of Cats;118729]\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\n[/quote]

\n\n

[quote=Bob, Lord of Cats;118729]\nafter about an hour+ I was only at 100k entries.\n[/quote]

\n\n

He actually mentions in the post:

\n\n

[quote]\nGiven the amount of memory on your system, it may or may not be feasible to read all the data in. If it isn’t, you should consider creating a machine on EC2 or DigitalOcean to process the data with.[/quote]

\n\n

and then links to this post https://www.dataquest.io/blog/data-science-ec2-quickstart/

\n\n

Obviously there's a cost involved here, but if it helps you iterate faster and doesn't lock up your machine, $30 doesn't seem like that much to me.

", "votes": null, "replies": [] }, { "id": 118845, "author_name": "manels", "author_url": "", "post_date": "05/05/2016 17:02:49", "content": "

@Bob, Lord of Cats \nTry to run this version. The killer code you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)

", "votes": null, "replies": [] }, { "id": 118847, "author_name": "vikpar", "author_url": "", "post_date": "05/05/2016 17:07:20", "content": "

[quote=Bob, Lord of Cats;118712]

\n\n

Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.

\n\n

[/quote]

\n\n

It took about an hour to run on my machine (16gb RAM, though). Maybe a library or system package version difference? Also, the ML code will be impractical to run on the whole training / test set, so that should be left out.

", "votes": null, "replies": [] }, { "id": 118848, "author_name": "vikpar", "author_url": "", "post_date": "05/05/2016 17:09:52", "content": "

[quote=dune_dweller;118636]

\n\n

@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :).

\n\n

The only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).

\n\n

[/quote]

\n\n

Yeah, I should have explained this better -- it comes down more to not being able to linearly separate the clusters with the given features because the boundaries are so fuzzy.

", "votes": null, "replies": [] }, { "id": 118849, "author_name": "vikpar", "author_url": "", "post_date": "05/05/2016 17:10:39", "content": "

[quote=MelvinDunn;118661]

\n\n

@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?

\n\n

[/quote]

\n\n

Please do! :)

", "votes": null, "replies": [] }, { "id": 118851, "author_name": "dimitrislev", "author_url": "", "post_date": "05/05/2016 17:29:37", "content": "

@Vik: Excellent tutorial! I also confirm that 100 x binary classification does not work here, I created 100 Online Logistic models using feature hashing (with the entire training dataset) and the resulting precision was the same (about 0.05) , in addition the AUC/LogLoss metrics for each classifier were about 0.68/0.29.\nWell done!

", "votes": null, "replies": [] }, { "id": 118855, "author_name": "mpekalski", "author_url": "", "post_date": "05/05/2016 18:23:18", "content": "

@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?

", "votes": null, "replies": [] }, { "id": 118856, "author_name": "rdslater", "author_url": "", "post_date": "05/05/2016 18:24:00", "content": "

[quote=Manel;118845]

\n\n

@Bob, Lord of Cats \nTry to run this version. The killer code you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)

\n\n

[/quote]\nI'll give it a shot after work. My idea wasn't to complain but improve because I think there is a more efficient way to do it (I actually have been modifying Dune Dweller's code as I couldn't quite get this one to the point I wanted where I could get my join/merge to work). My apologies as in my studies in R/Python/ML it has been pounded into me LOOPS BAD/VECTORIZE YOUR CODE. Plus in the grand scheme of research, grabbing several users ideas, combining them into my own and then have all of you tell me what a genius I am. (I am sort of slow on the combine part, but still... ;) )

\n\n

update: I dropped out of ipython into a straight shell--the improvement for me is significant--running at about 100K samples / 45 sec. Much quicker. Methinks there is an ID:10T error in my notebook config. R simply blows away my ipython notebook for speed.

\n\n

23:09 for the "Leak" section. I knew the old gal had it in her!

", "votes": null, "replies": [] }, { "id": 118858, "author_name": "dimitrislev", "author_url": "", "post_date": "05/05/2016 18:44:12", "content": "

[quote=Marcin Pękalski;118855]

\n\n

@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?

\n\n

[/quote]\nHello Marcin, I used basically the raw data excluding the dates, I created a few features (year, month, day of week, total days of stay) using the dates. Regarding passes, just passed the training dataset 2 times.\nNo fine tuning here, but I guess not so much chance for this to work.

", "votes": null, "replies": [] }, { "id": 118903, "author_name": "rdslater", "author_url": "", "post_date": "05/06/2016 01:28:07", "content": "

I owe @Vik an apology--the program wasn't the problem--apparently my Jupyter Notebook/IPython shell and chrome do not play well at all (10X or more performance hit). Running from a prompt gives excellent performance (under an hour as advertised).

", "votes": null, "replies": [] }, { "id": 120832, "author_name": "jaypeedevlin", "author_url": "", "post_date": "05/20/2016 20:14:04", "content": "

Glad you got it sorted in the end Bob!

", "votes": null, "replies": [] }, { "id": 160460, "author_name": "rayjohnsoncomedy", "author_url": "", "post_date": "02/07/2017 19:37:13", "content": "

@Vik many thanks! Following a lot of your tips/tutorials is helping a lot.

", "votes": null, "replies": [] }, { "id": 160461, "author_name": "rayjohnsoncomedy", "author_url": "", "post_date": "02/07/2017 19:37:16", "content": "", "votes": null, "replies": [] }, { "id": 344911, "author_name": "funkegoodvibe", "author_url": "", "post_date": "06/18/2018 22:40:18", "content": "

Thanks for the link @Vik

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "118471": "I wrote a tutorial on how to get into the top 15 on the leaderboard in the competition (as of now): https://www.dataquest.io/blog/kaggle-tutorial/ . Hope you enjoy it!", "118474": "Thanks! I wish you had used a better language [R :) ] but I will enjoy working through your examples and reasoning. Thanks for teaching us!", "118503": "Very detailed and well explained.. I really enjoyed reading through it and definitely learned more than one thing or two from the material. Thanks for the tutorial!", "118544": "Very nice tutorial. Thank you for sharing your knowledge.", "118557": "I wish there was a post like this in every competition :)", "118564": "Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb?", "118573": "Vik Thank you very much\r\n\r\nWith a very similar (and in times more granular) approach in perl I made 0.47x. Will surely look into your solution.\r\n\r\nCan anyone tell how much memory is needed for this script?\r\n\r\nThanks\r\n\r\nGerhard", "118611": "One of the easier to follow and understand tutorials that I have read, awesome job!", "118623": "[quote=Konrad Banachewicz;118564]\r\n\r\n@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb? \r\n\r\n[/quote]\r\n\r\nThanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.", "118626": "[quote=Vik Paruchuri;118623]\r\n\r\n[quote=Konrad Banachewicz;118564]\r\n\r\n@Vik: fantastic tutorial, well done. One thing that got me curious: the validation score shown in the script is ~ 0.28 and the snapshot from the leaderboard says 0.48. Is there really such a large discrepancy between validation and lb? \r\n\r\n[/quote]\r\n\r\nThanks! Yup, in this case, there was. The validation set was much smaller that the testing set due to sampling. I outlined a few reasons in the post for why the validation accuracy might be lower than leaderboard accuracy.\r\n\r\n[/quote]\r\n\r\nThank you Vik! Very helpful!\r\n\r\nI'm doing a much more careless validation but on 670k data. My local score is about 0.01 higher than lb.", "118628": "Vik: that's interesting - and potentially another part of the challenge. I suspect (didnt have time to verify it yet) it might have sth to do with the subsampling, i.e. the outcome is quite sensitive to *which* 10000 are sampled.", "118636": "Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :). \r\n\r\nThe only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).", "118637": "rcarson , but using the same approach or you actually used some ML?\r\n\r\nHow would you suggest splitting the set into train/test? \r\nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?", "118638": "[quote=Marcin Pękalski;118637]\r\n\r\n@rcarson , but using the same approach or you actually used some ML?\r\n\r\nHow would you suggest splitting the set into train/test? \r\nSeems like the most logical would be simply by dates, to have similar issues like with real data (new srch_destination_id in test set), but maybe not?\r\n\r\n[/quote]\r\n\r\nWe're simply using the last 670k data (very arbitrary choice) in train for validation. But Vik's way (by dates) makes more sense, you just need more data in validation to reduce local-lb difference.\r\n\r\nOur ml is still a little worse than our rule-based model but we are getting there.", "118644": "[quote=dune_dweller;118636]\r\n\r\n...If logistic regression doesn't work here the reason lies somewhere else :).\r\n\r\n[/quote]\r\n\r\nAgreed. I guess the reason is that there are too many unique values for each feature and their interactions. To accommodate all the features, the logistic regression will have to store and update a huge number of parameters - which means a memory problem. If we try to hash down the number of features, random hash collisions would be unavoidable, and hence the model would be compromised.", "118646": "Hi,\r\n\r\nso. After work I ran the first lines in python and RAM used max was nearly 30GB. Dropping to ~21GB after load finished.\r\n\r\nCheers\r\n\r\nGerhard", "118649": "MightyBird, try changing data types from 64bit to 32bit. That big precision is not needed, and the RAM requirements should go down a bit. Just remember to use garbage collector afterwards, to force the cleaning.", "118661": "Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard?", "118672": "Thank you Vik! Definitely very useful for a beginner Kaggler like me :)", "118693": "A new benchmark :)", "118696": "Thanks for the great tutorial. \r\nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard.", "118697": "[quote=Mohamed Ali Jamaoui;118696]\r\n\r\nThanks for the great tutorial. \r\nI wonder, though, if removing the click events is a good strategy as we are not sure if the private test set is also only composed of booking events (has no click events). If it's the case, your solution can overfit to the public leaderboard. \r\n\r\n[/quote]\r\n\r\nThe data section states that the test set only consists of booking events.", "118712": "Nice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.", "118729": "Sorry for the double post, but I did want to send an alert to everyone following the thread. \r\n\r\nFor me the killer code was this\r\n\r\n for i in range(test.shape[0]):\r\n exact_matches.append(generate_exact_matches(test.iloc[i], match_cols))\r\n\r\nafter about an hour+ I was only at 100k entries.\r\n\r\nI think a panda merge (pd.merge)--basically an SQL left join for those of us in database world, might be more efficient here for those of us with lesser hardware. I'll try and figure it out in a bit--its been a day and a half unrelated to kaggle, but I do want to contribute and make the script a bit more accessible.", "118742": "Great script Vik and very well written article!\r\n\r\nOne critique - you should probably join the destinations dataset to the training data (or a sample of the training data) *before* you run PCA instead of running PCA on the destinations dataset in isolation. The distribution of destinations within the samples could have an effect on the variance within the features.", "118753": "If you really take a look at the data in there, many of the rows have duplicate responses for one of the values. They just look like fills with one value from what I saw in the first few rows. Even doing a transform with Rtsne noted \"duplicates\". To be succinct - I highly doubt that each destination was reviewed 150 times. In that highly unlikely situation, I also doubt that many got a same sentiment value (derived from solely from text analysis, if not a weighted average of rating and sentiment analysis) several times.\r\n\r\nThe unique number of values for each row will differ if you look closely. I fear that doing a linear (or otherwise) transform, like PCA, on these values may not be the best choice of action without prior cleaning.\r\n\r\nSince I'm unsure of whether the mean is being used as the duplicate value for each row, it may be good to use that as the fill, instead of the value that's being displayed in each row.\r\n\r\nExcellent script though.", "118844": "[quote=Bob, Lord of Cats;118729]\r\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\r\n[/quote]\r\n\r\n[quote=Bob, Lord of Cats;118729]\r\nafter about an hour+ I was only at 100k entries.\r\n[/quote]\r\n\r\nHe actually mentions in the post:\r\n\r\n[quote]\r\nGiven the amount of memory on your system, it may or may not be feasible to read all the data in. If it isn’t, you should consider creating a machine on EC2 or DigitalOcean to process the data with.[/quote]\r\n\r\nand then links to this post https://www.dataquest.io/blog/data-science-ec2-quickstart/\r\n\r\nObviously there's a cost involved here, but if it helps you iterate faster and doesn't lock up your machine, $30 doesn't seem like that much to me.", "118845": "Bob, Lord of Cats \r\nTry to run this [version][1]. The *killer code* you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)\r\n\r\n\r\n [1]: https://www.kaggle.com/manels/expedia-hotel-recommendations/dataquest-tutorial", "118847": "[quote=Bob, Lord of Cats;118712]\r\n\r\nNice tutorial but it did crush my machine (e8400+8gb ram) running in an ipython notebook. 12 hours and it still was cranking away on the final step.\r\n\r\n[/quote]\r\n\r\nIt took about an hour to run on my machine (16gb RAM, though). Maybe a library or system package version difference? Also, the ML code will be impractical to run on the whole training / test set, so that should be left out.", "118848": "[quote=dune_dweller;118636]\r\n\r\n@Vik, nice tutorial! I liked your method for subsampling data, and visiting the Expedia site was a nice touch :). \r\n\r\nThe only thing that seemed off was the comment about how logistic regression is unlikely to work because no features correlate with hotel cluster. Logistic regression is a classification method, so it shouldn't care at all about the numeric values of class labels. If logistic regression doesn't work here the reason lies somewhere else :).\r\n\r\n[/quote]\r\n\r\nYeah, I should have explained this better -- it comes down more to not being able to linearly separate the clusters with the given features because the boundaries are so fuzzy.", "118849": "[quote=MelvinDunn;118661]\r\n\r\n@Vik, you mind if I put your tutorial in the scripts (you'll be cited) and add a bit of chaos to the Leaderboard? \r\n\r\n[/quote]\r\n\r\nPlease do! :)", "118851": "Vik: Excellent tutorial! I also confirm that 100 x binary classification does not work here, I created 100 Online Logistic models using feature hashing (with the entire training dataset) and the resulting precision was the same (about 0.05) , in addition the AUC/LogLoss metrics for each classifier were about 0.68/0.29.\r\nWell done!", "118855": "Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?", "118856": "[quote=Manel;118845]\r\n\r\n@Bob, Lord of Cats \r\nTry to run this [version][1]. The *killer code* you said runs in 0:24:40.167909h on my computer (i7 +8gb ram)\r\n\r\n\r\n [1]: https://www.kaggle.com/manels/expedia-hotel-recommendations/dataquest-tutorial\r\n\r\n[/quote]\r\nI'll give it a shot after work. My idea wasn't to complain but improve because I think there is a more efficient way to do it (I actually have been modifying Dune Dweller's code as I couldn't quite get this one to the point I wanted where I could get my join/merge to work). My apologies as in my studies in R/Python/ML it has been pounded into me LOOPS BAD/VECTORIZE YOUR CODE. Plus in the grand scheme of research, grabbing several users ideas, combining them into my own and then have all of you tell me what a genius I am. (I am sort of slow on the combine part, but still... ;) )\r\n\r\nupdate: I dropped out of ipython into a straight shell--the improvement for me is significant--running at about 100K samples / 45 sec. Much quicker. Methinks there is an ID:10T error in my notebook config. R simply blows away my ipython notebook for speed. \r\n\r\n23:09 for the \"Leak\" section. I knew the old gal had it in her!", "118858": "[quote=Marcin Pękalski;118855]\r\n\r\n@Dimitris, but what new features have you created? Or you just run it on the raw data? How many passes you made through the dataset?\r\n\r\n[/quote]\r\nHello Marcin, I used basically the raw data excluding the dates, I created a few features (year, month, day of week, total days of stay) using the dates. Regarding passes, just passed the training dataset 2 times.\r\nNo fine tuning here, but I guess not so much chance for this to work.", "118903": "I owe @Vik an apology--the program wasn't the problem--apparently my Jupyter Notebook/IPython shell and chrome do not play well at all (10X or more performance hit). Running from a prompt gives excellent performance (under an hour as advertised).", "120832": "Glad you got it sorted in the end Bob!", "160460": "Vik many thanks! Following a lot of your tips/tutorials is helping a lot.", "160461": "", "344911": "Thanks for the link @Vik", "3080956": "thanks for the tutorial" }, "source": "meta" }