![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/120033/4267/user_locations_color.png?sv=2012-02-12&se=2016-05-17T19:22:48Z&sr=b&sp=r&sig=j9rf8VctiYJjt7DxcMrttq5OlHIVgeznXdRaUaYo5o8%3D\")
\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/120033/4266/hotel_locations_color.png?sv=2012-02-12&se=2016-05-17T19:22:48Z&sr=b&sp=r&sig=qUCfhd676kRC7m6NO5iCHOsD7cQpxYBffHyOQ2Jy%2B34%3D\")
Hi Adam,
\n\nA few questions have been asked about reverse engineering the cities. \nhttps://www.kaggle.com/c/expedia-hotel-recommendations/forums/t/20282/a-leak-in-the-data/119760#post119760
\n\nPlease confirm what is considered external data:
\n\nWith these information I was able to reverse engineer the cities and map them to rough lat long coords. Actually I am not really interested in the exact real world coords. I just would like to map the cities to a spherical coordinate system that represents the distances between them. I believe it could be done using only the competition data and the haversine function.\nSuch representation would allow easier clustering (e.g. new user/ hotel regions)
\n\nPlease note that the origin/destination city coords is not really leakage you obviously have them and you could benefit from any model that uses them.
\n\nSince we only have city - hotel distances my method has ~0.5 mile average error. Imho it does not mean privacy threat to the anonymous users.
\n\n\n
Hi Adam,
\n\nA few questions have been asked about reverse engineering the cities. \nhttps://www.kaggle.com/c/expedia-hotel-recommendations/forums/t/20282/a-leak-in-the-data/119760#post119760
\n\nPlease confirm what is considered external data:
\n\nWith these information I was able to reverse engineer the cities and map them to rough lat long coords. Actually I am not really interested in the exact real world coords. I just would like to map the cities to a spherical coordinate system that represents the distances between them. I believe it could be done using only the competition data and the haversine function.\nSuch representation would allow easier clustering (e.g. new user/ hotel regions)
\n\nPlease note that the origin/destination city coords is not really leakage you obviously have them and you could benefit from any model that uses them.
\n\nSince we only have city - hotel distances my method has ~0.5 mile average error. Imho it does not mean privacy threat to the anonymous users.
\n\n\n
Interesting question!
\n\nCould you provide more details on how did you use information in point 2.? For example, did you assume that most frequent destinations (or user locations) in the US are NY, LV and SF and assign the above coordinates to these geo locations?
", "rawMarkdown": "Interesting question!\r\n\r\nCould you provide more details on how did you use information in point 2.? For example, did you assume that most frequent destinations (or user locations) in the US are NY, LV and SF and assign the above coordinates to these geo locations?", "votes": null }, { "id": "120083", "postDate": "05/15/2016 07:34:39", "content": "The point 2 is not necessary. We can reveal the configuration of points without using any real coordinates. Then, by looking at the contours one can recognize the continents.
", "rawMarkdown": "The point 2 is not necessary. We can reveal the configuration of points without using any real coordinates. Then, by looking at the contours one can recognize the continents.", "votes": null }, { "id": "120097", "postDate": "05/15/2016 11:16:35", "content": "[quote=Adam;120040]\nCould you provide more details on how did you use information in point 2.?\n[/quote]
\n\nI assumed that NY, LV, SF would be one of the top 50 cities (both user city and hotel city)\nI calculated the real world distances between NY, LV, SF based on the real world coords and matched them with the distances in the data.
\n\nAs Victor pointed out the real coordinates are not really necessary.
\n\nI could choose (0,0) for city A then set B somewhere based on dist(A, B) then find a suitable location for city C. I just used the real world coordinates because I don't think they are really external data and they allow us to draw cool maps and visualizations.
\n\n![\"enter](\"https://www.math.nyu.edu/~crorres/Archimedes/Lever/Lever.jpg\")
“Give me a place to stand on, and I will move the Earth.” Archimedes
", "rawMarkdown": "[quote=Adam;120040]\r\nCould you provide more details on how did you use information in point 2.?\r\n[/quote]\r\n\r\nI assumed that NY, LV, SF would be one of the top 50 cities (both user city and hotel city)\r\nI calculated the real world distances between NY, LV, SF based on the real world coords and matched them with the distances in the data.\r\n\r\nAs Victor pointed out the real coordinates are not really necessary. \r\n\r\n\r\nI could choose (0,0) for city A then set B somewhere based on dist(A, B) then find a suitable location for city C. I just used the real world coordinates because I don't think they are really external data and they allow us to draw cool maps and visualizations. \r\n\r\n![enter image description here][1]\r\n\r\n*“Give me a place to stand on, and I will move the Earth.”* Archimedes\r\n\r\n [1]: https://www.math.nyu.edu/~crorres/Archimedes/Lever/Lever.jpg", "votes": null }, { "id": "120138", "postDate": "05/15/2016 20:02:53", "content": "Got it. I prefer you don't use any real world coordinates to match distances that are found in the data. But as you said real coordinates are not necessary (i.e. you'll be able to cluster markets and destinations based on proximity, but you won't be able to draw nice maps).
", "rawMarkdown": "Got it. I prefer you don't use any real world coordinates to match distances that are found in the data. But as you said real coordinates are not necessary (i.e. you'll be able to cluster markets and destinations based on proximity, but you won't be able to draw nice maps).", "votes": null }, { "id": "120211", "postDate": "05/16/2016 12:34:48", "content": "That’s a cool viz.
\n\nCan you be more specific how did you get to it from the raw data? More specifically, how did you place each point on the map?\nThanks
", "rawMarkdown": "That’s a cool viz.\r\n\r\nCan you be more specific how did you get to it from the raw data? More specifically, how did you place each point on the map?\r\nThanks", "votes": null }, { "id": "120272", "postDate": "05/17/2016 01:04:59", "content": "Am i correct to assume that the general idea is to match the distance between user and destination city/hotel to an external database where the distances between cities are available?
\n\nSo if there is a close match, we could figure out which encoded city in the data set is the actual city (in real life).
", "rawMarkdown": "Am i correct to assume that the general idea is to match the distance between user and destination city/hotel to an external database where the distances between cities are available?\r\n\r\nSo if there is a close match, we could figure out which encoded city in the data set is the actual city (in real life).", "votes": null }, { "id": "120273", "postDate": "05/17/2016 01:05:26", "content": "", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 120040, "author_name": "adamwoz", "author_url": "", "post_date": "05/14/2016 20:11:34", "content": "Interesting question!
\n\nCould you provide more details on how did you use information in point 2.? For example, did you assume that most frequent destinations (or user locations) in the US are NY, LV and SF and assign the above coordinates to these geo locations?
", "votes": null, "replies": [] }, { "id": 120083, "author_name": "nedelko", "author_url": "", "post_date": "05/15/2016 07:34:39", "content": "The point 2 is not necessary. We can reveal the configuration of points without using any real coordinates. Then, by looking at the contours one can recognize the continents.
", "votes": null, "replies": [] }, { "id": 120097, "author_name": "gaborfodor", "author_url": "", "post_date": "05/15/2016 11:16:35", "content": "[quote=Adam;120040]\nCould you provide more details on how did you use information in point 2.?\n[/quote]
\n\nI assumed that NY, LV, SF would be one of the top 50 cities (both user city and hotel city)\nI calculated the real world distances between NY, LV, SF based on the real world coords and matched them with the distances in the data.
\n\nAs Victor pointed out the real coordinates are not really necessary.
\n\nI could choose (0,0) for city A then set B somewhere based on dist(A, B) then find a suitable location for city C. I just used the real world coordinates because I don't think they are really external data and they allow us to draw cool maps and visualizations.
\n\n
“Give me a place to stand on, and I will move the Earth.” Archimedes
", "votes": null, "replies": [] }, { "id": 120138, "author_name": "adamwoz", "author_url": "", "post_date": "05/15/2016 20:02:53", "content": "Got it. I prefer you don't use any real world coordinates to match distances that are found in the data. But as you said real coordinates are not necessary (i.e. you'll be able to cluster markets and destinations based on proximity, but you won't be able to draw nice maps).
", "votes": null, "replies": [] }, { "id": 120211, "author_name": "dmenin", "author_url": "", "post_date": "05/16/2016 12:34:48", "content": "That’s a cool viz.
\n\nCan you be more specific how did you get to it from the raw data? More specifically, how did you place each point on the map?\nThanks
", "votes": null, "replies": [] }, { "id": 120272, "author_name": "zyazzy", "author_url": "", "post_date": "05/17/2016 01:04:59", "content": "Am i correct to assume that the general idea is to match the distance between user and destination city/hotel to an external database where the distances between cities are available?
\n\nSo if there is a close match, we could figure out which encoded city in the data set is the actual city (in real life).
", "votes": null, "replies": [] }, { "id": 120273, "author_name": "zyazzy", "author_url": "", "post_date": "05/17/2016 01:05:26", "content": "", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "120033": "Hi Adam,\r\n\r\nA few questions have been asked about reverse engineering the cities. \r\nhttps://www.kaggle.com/c/expedia-hotel-recommendations/forums/t/20282/a-leak-in-the-data/119760#post119760\r\n\r\nPlease confirm what is considered external data:\r\n\r\n 1. AVG_EARTH_RADIUS = 6371\r\n 2. {'New York': (40.6643, -73.9385),\r\n 'San Francisco': (37.7751, -122.4193),\r\n 'Las Vegas': (36.2277, -115.2640)}\r\n 3. US is the country with the most bookings\r\n\r\n\r\nWith these information I was able to reverse engineer the cities and map them to rough lat long coords. Actually I am not really interested in the exact real world coords. I just would like to map the cities to a spherical coordinate system that represents the distances between them. I believe it could be done using only the competition data and the haversine function.\r\nSuch representation would allow easier clustering (e.g. new user/ hotel regions)\r\n\r\nPlease note that the origin/destination city coords is not really leakage you obviously have them and you could benefit from any model that uses them.\r\n\r\nSince we only have city - hotel distances my method has ~0.5 mile average error. Imho it does not mean privacy threat to the anonymous users. \r\n\r\n![enter image description here][1]\r\n![enter image description here][2]\r\n\r\n\r\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/120033/4267/user_locations_color.png?sv=2012-02-12&se=2016-05-17T19%3A22%3A48Z&sr=b&sp=r&sig=j9rf8VctiYJjt7DxcMrttq5OlHIVgeznXdRaUaYo5o8%3D\r\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/120033/4266/hotel_locations_color.png?sv=2012-02-12&se=2016-05-17T19%3A22%3A48Z&sr=b&sp=r&sig=qUCfhd676kRC7m6NO5iCHOsD7cQpxYBffHyOQ2Jy%2B34%3D", "120040": "Interesting question!\r\n\r\nCould you provide more details on how did you use information in point 2.? For example, did you assume that most frequent destinations (or user locations) in the US are NY, LV and SF and assign the above coordinates to these geo locations?", "120083": "The point 2 is not necessary. We can reveal the configuration of points without using any real coordinates. Then, by looking at the contours one can recognize the continents.", "120097": "[quote=Adam;120040]\r\nCould you provide more details on how did you use information in point 2.?\r\n[/quote]\r\n\r\nI assumed that NY, LV, SF would be one of the top 50 cities (both user city and hotel city)\r\nI calculated the real world distances between NY, LV, SF based on the real world coords and matched them with the distances in the data.\r\n\r\nAs Victor pointed out the real coordinates are not really necessary. \r\n\r\n\r\nI could choose (0,0) for city A then set B somewhere based on dist(A, B) then find a suitable location for city C. I just used the real world coordinates because I don't think they are really external data and they allow us to draw cool maps and visualizations. \r\n\r\n![enter image description here][1]\r\n\r\n*“Give me a place to stand on, and I will move the Earth.”* Archimedes\r\n\r\n [1]: https://www.math.nyu.edu/~crorres/Archimedes/Lever/Lever.jpg", "120138": "Got it. I prefer you don't use any real world coordinates to match distances that are found in the data. But as you said real coordinates are not necessary (i.e. you'll be able to cluster markets and destinations based on proximity, but you won't be able to draw nice maps).", "120211": "That’s a cool viz.\r\n\r\nCan you be more specific how did you get to it from the raw data? More specifically, how did you place each point on the map?\r\nThanks", "120272": "Am i correct to assume that the general idea is to match the distance between user and destination city/hotel to an external database where the distances between cities are available?\r\n\r\nSo if there is a close match, we could figure out which encoded city in the data set is the actual city (in real life).", "120273": "" }, "source": "meta" }