![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F745525%2F3aacda044ccde7227ebfa6ec7eecdb0c%2F2020-03-05%200.48.39.png?generation=1583336940871096&alt=media\")
I've noticed that there are 3 combinations on the train set that correspond to 2 different graphemes each, are these variations of the same grapheme?
\n\n|grapheme_root|vowel_diacritic|consonant_diacritic|grapheme1|grapheme2|\n|---|---|---|---|---|\n|64|3|2|র্তী | র্ত্রী|\n|64|7|2|র্তে | র্ত্রে|\n|72|0|2|র্দ |র্দ্র|
\n\nAll other triplets of components correspond to a unique grapheme in the train.csv
", "messages": [ { "id": "706874", "postDate": "12/30/2019 23:51:04", "content": "I've noticed that there are 3 combinations on the train set that correspond to 2 different graphemes each, are these variations of the same grapheme?
\n\n|grapheme_root|vowel_diacritic|consonant_diacritic|grapheme1|grapheme2|\n|---|---|---|---|---|\n|64|3|2|র্তী | র্ত্রী|\n|64|7|2|র্তে | র্ত্রে|\n|72|0|2|র্দ |র্দ্র|
\n\nAll other triplets of components correspond to a unique grapheme in the train.csv
", "rawMarkdown": "I've noticed that there are 3 combinations on the train set that correspond to 2 different graphemes each, are these variations of the same grapheme?\n\n|grapheme_root|vowel_diacritic|consonant_diacritic|grapheme1|grapheme2|\n|---|---|---|---|---|\n|64|3|2|র্তী | র্ত্রী|\n|64|7|2|র্তে | র্ত্রে|\n|72|0|2|র্দ |র্দ্র|\n\nAll other triplets of components correspond to a unique grapheme in the train.csv", "votes": null }, { "id": "708740", "postDate": "01/02/2020 15:55:43", "content": "The graphemes here look different due to the presence of that particular consonant diacritic.
", "rawMarkdown": "The graphemes here look different due to the presence of that particular consonant diacritic.", "votes": null }, { "id": "710115", "postDate": "01/04/2020 10:22:16", "content": "Hi Miguel,\nThis is actually an important detection, we are looking into why it bypassed our sanity checks, these indeed are different graphemes. This is quite an interesting linguistic phenomena where two diacritic forms of the same consonant র (r) are added to one grapheme root. Will come back with updates!
", "rawMarkdown": "Hi Miguel,\nThis is actually an important detection, we are looking into why it bypassed our sanity checks, these indeed are different graphemes. This is quite an interesting linguistic phenomena where two diacritic forms of the same consonant র (r) are added to one grapheme root. Will come back with updates!", "votes": null }, { "id": "729692", "postDate": "01/26/2020 14:31:23", "content": "@imtiazprio any updates about this?
", "rawMarkdown": "imtiazprio any updates about this?", "votes": null }, { "id": "737060", "postDate": "02/04/2020 21:35:51", "content": "@imtiazprio have you been able to resolve this, if yes pls share with a wider audience.
", "rawMarkdown": "imtiazprio have you been able to resolve this, if yes pls share with a wider audience.", "votes": null }, { "id": "758966", "postDate": "02/28/2020 12:01:24", "content": "@imtiazprio any updates about this?
", "rawMarkdown": "imtiazprio any updates about this?", "votes": null }, { "id": "765727", "postDate": "03/07/2020 02:07:33", "content": "I did an EDA anlysis of grapheme combinations and came to the same conclusion. \n- https://www.kaggle.com/c/bengaliai-cv19/discussion/133735 \n- https://www.kaggle.com/jamesmcguigan/bengali-ai-dataset-eda-grapheme-combinations
\n\nAs @ren4yu discovered, some graphemes do indeed have multiple renderings, and this is implemented in unicode by permitting duplicate diacritics in the multi charachter encoding:
\n\n
The graphemes here look different due to the presence of that particular consonant diacritic.
", "votes": null, "replies": [] }, { "id": 710115, "author_name": "imtiazprio", "author_url": "", "post_date": "01/04/2020 10:22:16", "content": "Hi Miguel,\nThis is actually an important detection, we are looking into why it bypassed our sanity checks, these indeed are different graphemes. This is quite an interesting linguistic phenomena where two diacritic forms of the same consonant র (r) are added to one grapheme root. Will come back with updates!
", "votes": null, "replies": [ { "id": 729692, "author_name": "artgor", "author_url": "", "post_date": "01/26/2020 14:31:23", "content": "@imtiazprio any updates about this?
", "votes": null, "replies": [] }, { "id": 737060, "author_name": "rohitagarwal", "author_url": "", "post_date": "02/04/2020 21:35:51", "content": "@imtiazprio have you been able to resolve this, if yes pls share with a wider audience.
", "votes": null, "replies": [] }, { "id": 758966, "author_name": "artgor", "author_url": "", "post_date": "02/28/2020 12:01:24", "content": "@imtiazprio any updates about this?
", "votes": null, "replies": [] } ] }, { "id": 765727, "author_name": "jamesmcguigan", "author_url": "", "post_date": "03/07/2020 02:07:33", "content": "I did an EDA anlysis of grapheme combinations and came to the same conclusion. \n- https://www.kaggle.com/c/bengaliai-cv19/discussion/133735 \n- https://www.kaggle.com/jamesmcguigan/bengali-ai-dataset-eda-grapheme-combinations
\n\nAs @ren4yu discovered, some graphemes do indeed have multiple renderings, and this is implemented in unicode by permitting duplicate diacritics in the multi charachter encoding:
\n\n