So Amazon actually has some pretty useful information in their Alexa API. They provide 1000 free API http request hits per month, with is sufficient to capture the period in train/test. Some of the data points provided on a per day basis are:
\n\nSo this data is freely available... the issue is within their EULA:
\n\n\n\n\n8.4 License Restrictions. Neither you nor any End User will use the Service Offerings in any manner or for any purpose other than as\n expressly permitted by this Agreement. Neither you nor any End User\n will, or will attempt to (a) modify, distribute, alter, tamper with,\n repair, or otherwise create derivative works of any Content included\n in the Service Offerings (except to the extent Content included in the\n Service Offerings is provided to you under a separate license that\n expressly permits the creation of derivative works),
\n
The emphasis above is my own. If we look at the competition rules, they state:
\n\n\n\n\nThe following provision supersedes General Rules Section 7.C. below:\n “You may use data, other than the Competition Data, as allowed on the\n Competition Website to develop and test your models and Submissions;\n provided, you have the right and authority to use such external data\n for the purposes of the Competition, and to share such data with\n Sponsor and Kaggle as may be required.\" If you are using a source of\n external data, you must post the source to the official external data\n forum thread no later than one week prior to the deadline.
\n
Again, emphasis is my own. So considering the testimony of at last one Kaggler ( @matthewa313 ) that just the rank variable alone packed some good feature importance, and considering the potential benefit of the other features available in the API.... and moreover, since this data really is FREE, can we get clarity if this data can be used within the competition @sohier ? If the answer is yes, again due to the Amazon EULA I doubt we would be able to do something post an Avito.ru Alexa Dataset, but I wouldn't mind publishing a kernel that allows ppl to just dump in their AWIS key and it'll aggregate the data for them / for you.
\n\nI highly doubt the sponsors, namely Avito themselves, would be against this =).
\n\nPlease advise.
", "messages": [ { "id": "330830", "postDate": "05/19/2018 19:48:45", "content": "So Amazon actually has some pretty useful information in their Alexa API. They provide 1000 free API http request hits per month, with is sufficient to capture the period in train/test. Some of the data points provided on a per day basis are:
\n\nSo this data is freely available... the issue is within their EULA:
\n\n\n\n\n8.4 License Restrictions. Neither you nor any End User will use the Service Offerings in any manner or for any purpose other than as\n expressly permitted by this Agreement. Neither you nor any End User\n will, or will attempt to (a) modify, distribute, alter, tamper with,\n repair, or otherwise create derivative works of any Content included\n in the Service Offerings (except to the extent Content included in the\n Service Offerings is provided to you under a separate license that\n expressly permits the creation of derivative works),
\n
The emphasis above is my own. If we look at the competition rules, they state:
\n\n\n\n\nThe following provision supersedes General Rules Section 7.C. below:\n “You may use data, other than the Competition Data, as allowed on the\n Competition Website to develop and test your models and Submissions;\n provided, you have the right and authority to use such external data\n for the purposes of the Competition, and to share such data with\n Sponsor and Kaggle as may be required.\" If you are using a source of\n external data, you must post the source to the official external data\n forum thread no later than one week prior to the deadline.
\n
Again, emphasis is my own. So considering the testimony of at last one Kaggler ( @matthewa313 ) that just the rank variable alone packed some good feature importance, and considering the potential benefit of the other features available in the API.... and moreover, since this data really is FREE, can we get clarity if this data can be used within the competition @sohier ? If the answer is yes, again due to the Amazon EULA I doubt we would be able to do something post an Avito.ru Alexa Dataset, but I wouldn't mind publishing a kernel that allows ppl to just dump in their AWIS key and it'll aggregate the data for them / for you.
\n\nI highly doubt the sponsors, namely Avito themselves, would be against this =).
\n\nPlease advise.
", "rawMarkdown": "So Amazon actually has some pretty useful information in their [Alexa API][1]. They provide 1000 free API http request hits per month, with is sufficient to capture the period in train/test. Some of the data points provided on a per day basis are:\n\n- Reach\n- Rank\n- PageViews\n- AvgPageViewsPerUser\n\nSo this data is freely available... the issue is within their [EULA][2]:\n\n> 8.4 License Restrictions. Neither you nor any End User will use the Service Offerings in any manner or for any purpose other than as\n> expressly permitted by this Agreement. Neither you nor any End User\n> will, or will attempt to (a) modify, **distribute**, alter, tamper with,\n> repair, or otherwise create derivative works of any Content included\n> in the Service Offerings (except to the extent Content included in the\n> Service Offerings is provided to you under a separate license that\n> expressly permits the creation of derivative works),\n\nThe emphasis above is my own. If we look at [the competition rules][3], they state:\n\n> The following provision supersedes General Rules Section 7.C. below:\n> “You may use data, other than the Competition Data, as allowed on the\n> Competition Website to develop and test your models and Submissions;\n> provided, you have the right and authority to use such external data\n> for the purposes of the Competition, and to **share** such data with\n> Sponsor and Kaggle as may be required.\" If you are using a source of\n> external data, you must post the source to the official external data\n> forum thread no later than one week prior to the deadline.\n\nAgain, emphasis is my own. So considering the testimony of at last one Kaggler ( @matthewa313 ) that just the rank variable alone packed some good feature importance, and considering the potential benefit of the other features available in the API.... and moreover, since this data really is FREE, can we get clarity if this data can be used within the competition @sohier ? If the answer is yes, again due to the Amazon EULA I doubt we would be able to do something post an Avito.ru Alexa Dataset, but I wouldn't mind publishing a kernel that allows ppl to just dump in their AWIS key and it'll aggregate the data for them / for you.\n\nI highly doubt the sponsors, namely Avito themselves, would be against this =).\n\nPlease advise.\n\n\n [1]: https://aws.amazon.com/awis/\n [2]: https://aws.amazon.com/agreement/\n [3]: https://www.kaggle.com/c/avito-demand-prediction/rules", "votes": null }, { "id": "331672", "postDate": "05/21/2018 17:10:56", "content": "@authman, thank you for providing such a detailed writeup!
\n\nBased on this information, Alexa data is not sufficiently open for use in this competition. The two key facts here are that only a limited amount of the data is available for free and that all of the data requires a login.
\n\nI will also post this to @matthewa313's section of the external data thread.
", "rawMarkdown": "authman, thank you for providing such a detailed writeup!\n\nBased on this information, Alexa data is not sufficiently open for use in this competition. The two key facts here are that only a limited amount of the data is available for free and that all of the data requires a login.\n\nI will also post this to @matthewa313's section of the external data thread.", "votes": null }, { "id": "331675", "postDate": "05/21/2018 17:12:47", "content": "Thank you for the clarification.
", "rawMarkdown": "Thank you for the clarification.", "votes": null }, { "id": "331680", "postDate": "05/21/2018 17:20:28", "content": "Thanks @authman for doing the work to get this important clarification.
", "rawMarkdown": "Thanks @authman for doing the work to get this important clarification.", "votes": null } ], "comments": [ { "id": 331672, "author_name": "sohier", "author_url": "", "post_date": "05/21/2018 17:10:56", "content": "@authman, thank you for providing such a detailed writeup!
\n\nBased on this information, Alexa data is not sufficiently open for use in this competition. The two key facts here are that only a limited amount of the data is available for free and that all of the data requires a login.
\n\nI will also post this to @matthewa313's section of the external data thread.
", "votes": null, "replies": [ { "id": 331675, "author_name": "authman", "author_url": "", "post_date": "05/21/2018 17:12:47", "content": "Thank you for the clarification.
", "votes": null, "replies": [] }, { "id": 331680, "author_name": "peterhurford", "author_url": "", "post_date": "05/21/2018 17:20:28", "content": "Thanks @authman for doing the work to get this important clarification.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "330830": "So Amazon actually has some pretty useful information in their [Alexa API][1]. They provide 1000 free API http request hits per month, with is sufficient to capture the period in train/test. Some of the data points provided on a per day basis are:\n\n- Reach\n- Rank\n- PageViews\n- AvgPageViewsPerUser\n\nSo this data is freely available... the issue is within their [EULA][2]:\n\n> 8.4 License Restrictions. Neither you nor any End User will use the Service Offerings in any manner or for any purpose other than as\n> expressly permitted by this Agreement. Neither you nor any End User\n> will, or will attempt to (a) modify, **distribute**, alter, tamper with,\n> repair, or otherwise create derivative works of any Content included\n> in the Service Offerings (except to the extent Content included in the\n> Service Offerings is provided to you under a separate license that\n> expressly permits the creation of derivative works),\n\nThe emphasis above is my own. If we look at [the competition rules][3], they state:\n\n> The following provision supersedes General Rules Section 7.C. below:\n> “You may use data, other than the Competition Data, as allowed on the\n> Competition Website to develop and test your models and Submissions;\n> provided, you have the right and authority to use such external data\n> for the purposes of the Competition, and to **share** such data with\n> Sponsor and Kaggle as may be required.\" If you are using a source of\n> external data, you must post the source to the official external data\n> forum thread no later than one week prior to the deadline.\n\nAgain, emphasis is my own. So considering the testimony of at last one Kaggler ( @matthewa313 ) that just the rank variable alone packed some good feature importance, and considering the potential benefit of the other features available in the API.... and moreover, since this data really is FREE, can we get clarity if this data can be used within the competition @sohier ? If the answer is yes, again due to the Amazon EULA I doubt we would be able to do something post an Avito.ru Alexa Dataset, but I wouldn't mind publishing a kernel that allows ppl to just dump in their AWIS key and it'll aggregate the data for them / for you.\n\nI highly doubt the sponsors, namely Avito themselves, would be against this =).\n\nPlease advise.\n\n\n [1]: https://aws.amazon.com/awis/\n [2]: https://aws.amazon.com/agreement/\n [3]: https://www.kaggle.com/c/avito-demand-prediction/rules", "331672": "authman, thank you for providing such a detailed writeup!\n\nBased on this information, Alexa data is not sufficiently open for use in this competition. The two key facts here are that only a limited amount of the data is available for free and that all of the data requires a login.\n\nI will also post this to @matthewa313's section of the external data thread.", "331675": "Thank you for the clarification.", "331680": "Thanks @authman for doing the work to get this important clarification." }, "source": "meta" }