Below is a list of Kaggle datasets containing translated texts. If I'm missing any, please comment. If they are listed in the external thread discussion, i wrote \"yes\" after them.
\n\nTrain\njigsaw-train-translated now yes\njigsaw-train-translated-yandex-api yes\njigsaw-train-multilingual-coments-google-api yes\njigsawtanslatedgoogle yes\ntranslated-train-bias-all-langs yes
\n\nValid\nval-en-df now yes\njigsaw-multilingual-toxic-test-translated yes\njigsawtanslatedgoogle yes
\n\nTest\ntest-en-df now yes\njigsaw-multilingual-toxic-test-translated yes
", "messages": [ { "id": "892699", "postDate": "06/19/2020 05:08:24", "content": "Below is a list of Kaggle datasets containing translated texts. If I'm missing any, please comment. If they are listed in the external thread discussion, i wrote \"yes\" after them.
\n\nTrain\njigsaw-train-translated now yes\njigsaw-train-translated-yandex-api yes\njigsaw-train-multilingual-coments-google-api yes\njigsawtanslatedgoogle yes\ntranslated-train-bias-all-langs yes
\n\nValid\nval-en-df now yes\njigsaw-multilingual-toxic-test-translated yes\njigsawtanslatedgoogle yes
\n\nTest\ntest-en-df now yes\njigsaw-multilingual-toxic-test-translated yes
", "rawMarkdown": "Below is a list of Kaggle datasets containing translated texts. If I'm missing any, please comment. If they are listed in the external thread discussion, i wrote \"yes\" after them.\n\nTrain\n[jigsaw-train-translated][3] now yes\n[jigsaw-train-translated-yandex-api][4] yes\n[jigsaw-train-multilingual-coments-google-api][5] yes\n[jigsawtanslatedgoogle][7] yes\n[translated-train-bias-all-langs][8] yes\n\nValid\n[val-en-df][2] now yes\n[jigsaw-multilingual-toxic-test-translated][6] yes\n[jigsawtanslatedgoogle][7] yes\n\nTest\n[test-en-df][1] now yes\n[jigsaw-multilingual-toxic-test-translated][6] yes\n\n[1]: https://www.kaggle.com/bamps53/test-en-df\n[2]: https://www.kaggle.com/bamps53/val-en-df\n[3]: https://www.kaggle.com/ma7555/jigsaw-train-translated\n[4]: https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api\n[5]: https://www.kaggle.com/miklgr500/jigsaw-train-multilingual-coments-google-api\n[6]: https://www.kaggle.com/kashnitsky/jigsaw-multilingual-toxic-test-translated\n[7]: https://www.kaggle.com/ludovick/jigsawtanslatedgoogle\n[8]: https://www.kaggle.com/rafiko1/translated-train-bias-all-langs", "votes": null }, { "id": "892718", "postDate": "06/19/2020 05:29:31", "content": "Thanks for sharing.
", "rawMarkdown": "Thanks for sharing.", "votes": null }, { "id": "893748", "postDate": "06/19/2020 21:16:02", "content": "Thanks for this list. One question: have all of these been approved as valid external datasets? I guess yes but just double checking. :)
", "rawMarkdown": "Thanks for this list. One question: have all of these been approved as valid external datasets? I guess yes but just double checking. :)", "votes": null }, { "id": "893766", "postDate": "06/19/2020 21:39:24", "content": "@bamps53 Camaro, did you list your translated texts in the external dataset thread?\nhttps://www.kaggle.com/bamps53/val-en-df\nhttps://www.kaggle.com/bamps53/test-en-df
", "rawMarkdown": "bamps53 Camaro, did you list your translated texts in the external dataset thread?\nhttps://www.kaggle.com/bamps53/val-en-df\nhttps://www.kaggle.com/bamps53/test-en-df", "votes": null }, { "id": "893768", "postDate": "06/19/2020 21:40:54", "content": "@ma7555 did you list your https://www.kaggle.com/ma7555/jigsaw-train-translated in the external datasets thread? I see https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api in the thread but not the former.
", "rawMarkdown": "ma7555 did you list your https://www.kaggle.com/ma7555/jigsaw-train-translated in the external datasets thread? I see https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api in the thread but not the former.", "votes": null }, { "id": "893771", "postDate": "06/19/2020 21:44:30", "content": "Great question. I just reviewed the external dataset thread. All but 3 are listed there. But some people list outputs from public kernels and https://cloud.google.com/translate/. So maybe that covers those 3. Also you can turn internet on in your notebook and connect with a translation service during submit so maybe explicitly listing translations is not required.
Also if those 3 datasets are disallowed, I'm not sure what happens if you use the output from a public notebook that trains with one of those 3 datasets. Does the disqualification transfer to the notebook's output?
", "rawMarkdown": "Also if those 3 datasets are disallowed, I'm not sure what happens if you use the output from a public notebook that trains with one of those 3 datasets. Does the disqualification transfer to the notebook's output?", "votes": null }, { "id": "893775", "postDate": "06/19/2020 21:52:45", "content": "Thanks for checking @cdeotte. I am trying to be extra careful given what happened recently. 😄
", "rawMarkdown": "Thanks for checking @cdeotte. I am trying to be extra careful given what happened recently. 😄", "votes": null }, { "id": "893783", "postDate": "06/19/2020 22:11:28", "content": "@cdeotte I just posted those two in the external dataset thread.
", "rawMarkdown": "cdeotte I just posted those two in the external dataset thread.", "votes": null }, { "id": "893784", "postDate": "06/19/2020 22:14:08", "content": "Ok, technically they need to be posted 1 week before competition deadline. But these have been discussed previously in the forums and they have been public for 3 months, so its probably ok.
", "rawMarkdown": "Ok, technically they need to be posted 1 week before competition deadline. But these have been discussed previously in the forums and they have been public for 3 months, so its probably ok.", "votes": null }, { "id": "893798", "postDate": "06/19/2020 23:20:56", "content": "Thanks @cdeotte , last week I was searching for them and was having a hard time going through discussions, and to be honest I did not occured t to me looking if they were listed at the external data thread. You listed all that I got but that are two more datasets, but they are not the competition data translated, listing here just in case:
\n\n", "rawMarkdown": "Thanks @cdeotte , last week I was searching for them and was having a hard time going through discussions, and to be honest I did not occured t to me looking if they were listed at the external data thread. You listed all that I got but that are two more datasets, but they are not the competition data translated, listing here just in case:\n\n- [open_subtitles_toxic_pseudo_labeling](https://www.kaggle.com/shonenkov/open-subtitles-toxic-pseudo-labeling)\n- [Jigsaw Bias Toxicity EDA NLP aug16 alpha0.05](https://www.kaggle.com/yeayates21/jigsaw-bias-toxicity-eda-nlp-aug16-alpha005)", "votes": null }, { "id": "893816", "postDate": "06/20/2020 00:40:37", "content": "You are right @cdeotte. I have been seeing them in public kernels but did not use it until yesterday so surprised to know from your post that they are not posted.
", "rawMarkdown": "You are right @cdeotte. I have been seeing them in public kernels but did not use it until yesterday so surprised to know from your post that they are not posted.", "votes": null }, { "id": "893840", "postDate": "06/20/2020 02:16:02", "content": "Hello @cdeotte , I have added both now
\n\nthanks for the heads up, stay safe
", "rawMarkdown": "Hello @cdeotte , I have added both now\n\nthanks for the heads up, stay safe", "votes": null }, { "id": "893877", "postDate": "06/20/2020 03:45:08", "content": "good NB chris
", "rawMarkdown": "good NB chris", "votes": null }, { "id": "894131", "postDate": "06/20/2020 07:32:34", "content": "thanks for sharing
", "rawMarkdown": "thanks for sharing", "votes": null }, { "id": "895544", "postDate": "06/21/2020 12:50:29", "content": "Very useful. However, it's worth noting that some might not be fully translated due to API limits. Regardless, thanks a lot for making us aware of this!
", "rawMarkdown": "Very useful. However, it's worth noting that some might not be fully translated due to API limits. Regardless, thanks a lot for making us aware of this!", "votes": null }, { "id": "895744", "postDate": "06/21/2020 15:20:27", "content": "@cdeotte Do you know if anyone has translated the test dataset to the other 5 languages for each row?
", "rawMarkdown": "cdeotte Do you know if anyone has translated the test dataset to the other 5 languages for each row?", "votes": null }, { "id": "896112", "postDate": "06/21/2020 21:21:16", "content": "Good question. I have not seen a version like this but that doesn't mean it isn't out there. If you find one, let me know.
", "rawMarkdown": "Good question. I have not seen a version like this but that doesn't mean it isn't out there. If you find one, let me know.", "votes": null }, { "id": "896346", "postDate": "06/22/2020 05:14:37", "content": "@yassinealouini i make a dataset using googld api and make it public yesterday\nhttps://www.kaggle.com/alphaecho/jigsawtranslatedtest
", "rawMarkdown": "yassinealouini i make a dataset using googld api and make it public yesterday\nhttps://www.kaggle.com/alphaecho/jigsawtranslatedtest", "votes": null }, { "id": "897227", "postDate": "06/22/2020 17:58:43", "content": "Thats seem correct
", "rawMarkdown": "Thats seem correct", "votes": null }, { "id": "900054", "postDate": "06/24/2020 15:36:57", "content": "good job
", "rawMarkdown": "good job", "votes": null }, { "id": "902413", "postDate": "06/26/2020 06:05:37", "content": "thanks for sharing
", "rawMarkdown": "thanks for sharing", "votes": null }, { "id": "910821", "postDate": "07/01/2020 11:46:36", "content": "Hi @cdeotte thanks for sharing Datasets ,\nConcerning the translation of texts, I found a library (which could be useful) to translate texts,\nhttps://pypi.org/project/googletrans/\nThis library uses google translate.
", "rawMarkdown": "Hi @cdeotte thanks for sharing Datasets ,\nConcerning the translation of texts, I found a library (which could be useful) to translate texts,\nhttps://pypi.org/project/googletrans/\nThis library uses google translate.", "votes": null }, { "id": "916943", "postDate": "07/06/2020 06:22:08", "content": "Hi
", "rawMarkdown": "Hi", "votes": null } ], "comments": [ { "id": 892718, "author_name": "shivyshiv", "author_url": "", "post_date": "06/19/2020 05:29:31", "content": "Thanks for sharing.
", "votes": null, "replies": [] }, { "id": 893748, "author_name": "yassinealouini", "author_url": "", "post_date": "06/19/2020 21:16:02", "content": "Thanks for this list. One question: have all of these been approved as valid external datasets? I guess yes but just double checking. :)
", "votes": null, "replies": [ { "id": 893771, "author_name": "cdeotte", "author_url": "", "post_date": "06/19/2020 21:44:30", "content": "Great question. I just reviewed the external dataset thread. All but 3 are listed there. But some people list outputs from public kernels and https://cloud.google.com/translate/. So maybe that covers those 3. Also you can turn internet on in your notebook and connect with a translation service during submit so maybe explicitly listing translations is not required.
Also if those 3 datasets are disallowed, I'm not sure what happens if you use the output from a public notebook that trains with one of those 3 datasets. Does the disqualification transfer to the notebook's output?
", "votes": null, "replies": [] }, { "id": 893775, "author_name": "yassinealouini", "author_url": "", "post_date": "06/19/2020 21:52:45", "content": "Thanks for checking @cdeotte. I am trying to be extra careful given what happened recently. 😄
", "votes": null, "replies": [] } ] }, { "id": 893766, "author_name": "cdeotte", "author_url": "", "post_date": "06/19/2020 21:39:24", "content": "@bamps53 Camaro, did you list your translated texts in the external dataset thread?\nhttps://www.kaggle.com/bamps53/val-en-df\nhttps://www.kaggle.com/bamps53/test-en-df
", "votes": null, "replies": [ { "id": 893783, "author_name": "sheriytm", "author_url": "", "post_date": "06/19/2020 22:11:28", "content": "@cdeotte I just posted those two in the external dataset thread.
", "votes": null, "replies": [] }, { "id": 893784, "author_name": "cdeotte", "author_url": "", "post_date": "06/19/2020 22:14:08", "content": "Ok, technically they need to be posted 1 week before competition deadline. But these have been discussed previously in the forums and they have been public for 3 months, so its probably ok.
", "votes": null, "replies": [] }, { "id": 893816, "author_name": "sheriytm", "author_url": "", "post_date": "06/20/2020 00:40:37", "content": "You are right @cdeotte. I have been seeing them in public kernels but did not use it until yesterday so surprised to know from your post that they are not posted.
", "votes": null, "replies": [] } ] }, { "id": 893768, "author_name": "cdeotte", "author_url": "", "post_date": "06/19/2020 21:40:54", "content": "@ma7555 did you list your https://www.kaggle.com/ma7555/jigsaw-train-translated in the external datasets thread? I see https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api in the thread but not the former.
", "votes": null, "replies": [ { "id": 893840, "author_name": "ma7555", "author_url": "", "post_date": "06/20/2020 02:16:02", "content": "Hello @cdeotte , I have added both now
\n\nthanks for the heads up, stay safe
", "votes": null, "replies": [] } ] }, { "id": 893798, "author_name": "dimitreoliveira", "author_url": "", "post_date": "06/19/2020 23:20:56", "content": "Thanks @cdeotte , last week I was searching for them and was having a hard time going through discussions, and to be honest I did not occured t to me looking if they were listed at the external data thread. You listed all that I got but that are two more datasets, but they are not the competition data translated, listing here just in case:
\n\n", "votes": null, "replies": [] }, { "id": 893877, "author_name": "muralidhar123", "author_url": "", "post_date": "06/20/2020 03:45:08", "content": "good NB chris
", "votes": null, "replies": [] }, { "id": 894131, "author_name": "muralidhar123", "author_url": "", "post_date": "06/20/2020 07:32:34", "content": "thanks for sharing
", "votes": null, "replies": [] }, { "id": 895544, "author_name": "danoozy44", "author_url": "", "post_date": "06/21/2020 12:50:29", "content": "Very useful. However, it's worth noting that some might not be fully translated due to API limits. Regardless, thanks a lot for making us aware of this!
", "votes": null, "replies": [] }, { "id": 895744, "author_name": "yassinealouini", "author_url": "", "post_date": "06/21/2020 15:20:27", "content": "@cdeotte Do you know if anyone has translated the test dataset to the other 5 languages for each row?
", "votes": null, "replies": [ { "id": 896112, "author_name": "cdeotte", "author_url": "", "post_date": "06/21/2020 21:21:16", "content": "Good question. I have not seen a version like this but that doesn't mean it isn't out there. If you find one, let me know.
", "votes": null, "replies": [] }, { "id": 896346, "author_name": "alphaecho", "author_url": "", "post_date": "06/22/2020 05:14:37", "content": "@yassinealouini i make a dataset using googld api and make it public yesterday\nhttps://www.kaggle.com/alphaecho/jigsawtranslatedtest
", "votes": null, "replies": [] } ] }, { "id": 897227, "author_name": "oludhe", "author_url": "", "post_date": "06/22/2020 17:58:43", "content": "Thats seem correct
", "votes": null, "replies": [] }, { "id": 900054, "author_name": "ludobb", "author_url": "", "post_date": "06/24/2020 15:36:57", "content": "good job
", "votes": null, "replies": [] }, { "id": 902413, "author_name": "theparth", "author_url": "", "post_date": "06/26/2020 06:05:37", "content": "thanks for sharing
", "votes": null, "replies": [] }, { "id": 910821, "author_name": "syphax93", "author_url": "", "post_date": "07/01/2020 11:46:36", "content": "Hi @cdeotte thanks for sharing Datasets ,\nConcerning the translation of texts, I found a library (which could be useful) to translate texts,\nhttps://pypi.org/project/googletrans/\nThis library uses google translate.
", "votes": null, "replies": [] }, { "id": 916943, "author_name": "", "author_url": "", "post_date": "07/06/2020 06:22:08", "content": "Hi
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "892699": "Below is a list of Kaggle datasets containing translated texts. If I'm missing any, please comment. If they are listed in the external thread discussion, i wrote \"yes\" after them.\n\nTrain\n[jigsaw-train-translated][3] now yes\n[jigsaw-train-translated-yandex-api][4] yes\n[jigsaw-train-multilingual-coments-google-api][5] yes\n[jigsawtanslatedgoogle][7] yes\n[translated-train-bias-all-langs][8] yes\n\nValid\n[val-en-df][2] now yes\n[jigsaw-multilingual-toxic-test-translated][6] yes\n[jigsawtanslatedgoogle][7] yes\n\nTest\n[test-en-df][1] now yes\n[jigsaw-multilingual-toxic-test-translated][6] yes\n\n[1]: https://www.kaggle.com/bamps53/test-en-df\n[2]: https://www.kaggle.com/bamps53/val-en-df\n[3]: https://www.kaggle.com/ma7555/jigsaw-train-translated\n[4]: https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api\n[5]: https://www.kaggle.com/miklgr500/jigsaw-train-multilingual-coments-google-api\n[6]: https://www.kaggle.com/kashnitsky/jigsaw-multilingual-toxic-test-translated\n[7]: https://www.kaggle.com/ludovick/jigsawtanslatedgoogle\n[8]: https://www.kaggle.com/rafiko1/translated-train-bias-all-langs", "892718": "Thanks for sharing.", "893748": "Thanks for this list. One question: have all of these been approved as valid external datasets? I guess yes but just double checking. :)", "893766": "bamps53 Camaro, did you list your translated texts in the external dataset thread?\nhttps://www.kaggle.com/bamps53/val-en-df\nhttps://www.kaggle.com/bamps53/test-en-df", "893768": "ma7555 did you list your https://www.kaggle.com/ma7555/jigsaw-train-translated in the external datasets thread? I see https://www.kaggle.com/ma7555/jigsaw-train-translated-yandex-api in the thread but not the former.", "893771": "Great question. I just reviewed the external dataset thread. All but 3 are listed there. But some people list `outputs from public kernels` and `https://cloud.google.com/translate/`. So maybe that covers those 3. Also you can turn internet on in your notebook and connect with a translation service during submit so maybe explicitly listing translations is not required.", "893772": "Also if those 3 datasets are disallowed, I'm not sure what happens if you use the output from a public notebook that trains with one of those 3 datasets. Does the disqualification transfer to the notebook's output?", "893775": "Thanks for checking @cdeotte. I am trying to be extra careful given what happened recently. 😄", "893783": "cdeotte I just posted those two in the external dataset thread.", "893784": "Ok, technically they need to be posted 1 week before competition deadline. But these have been discussed previously in the forums and they have been public for 3 months, so its probably ok.", "893798": "Thanks @cdeotte , last week I was searching for them and was having a hard time going through discussions, and to be honest I did not occured t to me looking if they were listed at the external data thread. You listed all that I got but that are two more datasets, but they are not the competition data translated, listing here just in case:\n\n- [open_subtitles_toxic_pseudo_labeling](https://www.kaggle.com/shonenkov/open-subtitles-toxic-pseudo-labeling)\n- [Jigsaw Bias Toxicity EDA NLP aug16 alpha0.05](https://www.kaggle.com/yeayates21/jigsaw-bias-toxicity-eda-nlp-aug16-alpha005)", "893816": "You are right @cdeotte. I have been seeing them in public kernels but did not use it until yesterday so surprised to know from your post that they are not posted.", "893840": "Hello @cdeotte , I have added both now\n\nthanks for the heads up, stay safe", "893877": "good NB chris", "894131": "thanks for sharing", "895544": "Very useful. However, it's worth noting that some might not be fully translated due to API limits. Regardless, thanks a lot for making us aware of this!", "895744": "cdeotte Do you know if anyone has translated the test dataset to the other 5 languages for each row?", "896112": "Good question. I have not seen a version like this but that doesn't mean it isn't out there. If you find one, let me know.", "896346": "yassinealouini i make a dataset using googld api and make it public yesterday\nhttps://www.kaggle.com/alphaecho/jigsawtranslatedtest", "897227": "Thats seem correct", "900054": "good job", "902413": "thanks for sharing", "910821": "Hi @cdeotte thanks for sharing Datasets ,\nConcerning the translation of texts, I found a library (which could be useful) to translate texts,\nhttps://pypi.org/project/googletrans/\nThis library uses google translate.", "916943": "Hi" }, "source": "meta" }