{ "id": 403348, "title": "Color analysis: Ink = Ink + Noise. ROI", "url": "/competitions/vesuvius-challenge-ink-detection/discussion/403348", "author_name": "", "post_date": "2023-04-22T16:01:22.114585700Z", "votes": 50, "comment_count": 13, "views": 0, "content": "

We noticed that the boundary pixels behave abnormally and do not contribute well to the color analysis. Therefore, we excluded the boundary and instead focused on the inner part of the fragment.

\n

\"Trimming

\n

After analyzing the ink's color distribution of the first fragment, it is clear that there are two overlapping normal distributions.

\n

The first Gaussian-like distribution, ranging from 0-130, probably represents the genuine ink distribution, while the second one, ranging from 0-255, seems to account for noise.

\n

The noise may originate from the annotation process of the letters, where the annotator likely focused on outlining the shape of each letter rather than capturing every individual pixel. Furthermore, ink degradation and sparsity over time could have also contributed to the noise.

\n

Fragment 1

\n

\"Fragment

\n

Fragment 2

\n

Fragment 2 is more challenging and its histogram requires a bit of linear transformation (roughly 0.6* x + 42) processing but ultimately conforms to the same pattern.
\n\"Fragment

\n

A few insights:

\n

1) It is reasonable to assume that pixels of fragment 1 with a color value greater than 130 are unlikely to be ink pixels and can be ignored.

\n

2) We can generate a refined ground truth (GT) mask for fragment 1 by eliminating the papyrus pixels (>130).

\n

\"Refined

\n

3) A refined mask for Fragment 2 needs to be generated considering the inverse transformation corresponding to the mentioned histogram transformation.

", "messages": [ { "id": "2230650", "postDate": "04/22/2023 16:01:22", "content": "

We noticed that the boundary pixels behave abnormally and do not contribute well to the color analysis. Therefore, we excluded the boundary and instead focused on the inner part of the fragment.

\n

\"Trimming

\n

After analyzing the ink's color distribution of the first fragment, it is clear that there are two overlapping normal distributions.

\n

The first Gaussian-like distribution, ranging from 0-130, probably represents the genuine ink distribution, while the second one, ranging from 0-255, seems to account for noise.

\n

The noise may originate from the annotation process of the letters, where the annotator likely focused on outlining the shape of each letter rather than capturing every individual pixel. Furthermore, ink degradation and sparsity over time could have also contributed to the noise.

\n

Fragment 1

\n

\"Fragment

\n

Fragment 2

\n

Fragment 2 is more challenging and its histogram requires a bit of linear transformation (roughly 0.6* x + 42) processing but ultimately conforms to the same pattern.
\n\"Fragment

\n

A few insights:

\n

1) It is reasonable to assume that pixels of fragment 1 with a color value greater than 130 are unlikely to be ink pixels and can be ignored.

\n

2) We can generate a refined ground truth (GT) mask for fragment 1 by eliminating the papyrus pixels (>130).

\n

\"Refined

\n

3) A refined mask for Fragment 2 needs to be generated considering the inverse transformation corresponding to the mentioned histogram transformation.

", "rawMarkdown": "We noticed that the boundary pixels behave abnormally and do not contribute well to the color analysis. Therefore, we excluded the boundary and instead focused on the inner part of the fragment.\n\n![Trimming process](https://user-images.githubusercontent.com/16557697/233792973-1c53dd47-2ee8-4dd4-b460-0248ab7e3fb1.png)\n\nAfter analyzing the ink's color distribution of the first fragment, it is clear that there are two overlapping normal distributions.\n\nThe first Gaussian-like distribution, ranging from 0-130, probably represents the genuine ink distribution, while the second one, ranging from 0-255, seems to account for noise.\n\nThe noise may originate from the annotation process of the letters, where the annotator likely focused on outlining the shape of each letter rather than capturing every individual pixel. Furthermore, ink degradation and sparsity over time could have also contributed to the noise.\n\n\n## Fragment 1\n![Fragment 1](https://user-images.githubusercontent.com/16557697/233792717-2e0b1fd0-692d-40c5-91da-c5be40fb61ba.png)\n\n## Fragment 2\n\nFragment 2 is more challenging and its histogram requires a bit of linear transformation (roughly 0.6* x + 42) processing but ultimately conforms to the same pattern.\n![Fragment 2](https://user-images.githubusercontent.com/16557697/233793079-9b331fc6-c956-4ad6-974e-ee71bc3abf6b.png)\n\n\nA few insights:\n\n1) It is reasonable to assume that pixels of **fragment 1** with a color value greater than 130 are unlikely to be ink pixels and can be ignored.\n\n2) We can generate a refined ground truth (GT) mask for **fragment 1** by eliminating the papyrus pixels (>130).\n\n![Refined mask](https://user-images.githubusercontent.com/16557697/233793741-c2c91ff1-27db-4643-96c5-e63ae3231fd7.png)\n\n3) A refined mask for **Fragment 2** needs to be generated considering the inverse transformation corresponding to the mentioned histogram transformation.", "votes": null }, { "id": "2230833", "postDate": "04/22/2023 19:27:21", "content": "

Fragment 3

\n

The same pattern applies to Fragment 3, where ink and noise can be distinguished. The spectrum of ink colors spans from 0 to 140.

\n

\"Fragment

\n

\"GIF\"

", "rawMarkdown": "## Fragment 3\n\nThe same pattern applies to Fragment 3, where ink and noise can be distinguished. The spectrum of ink colors spans from 0 to 140.\n\n![Fragment 3](https://user-images.githubusercontent.com/16557697/233802053-e74b36ac-1497-49dd-803b-74d1762f3a38.png)\n\n\n![GIF](https://user-images.githubusercontent.com/16557697/233802776-1aab36bb-f912-448c-94d7-abdb9e28a27d.gif)", "votes": null }, { "id": "2231499", "postDate": "04/23/2023 11:29:21", "content": "

This is all very interesting. An obvious but important next step for anyone reading this is to see how this mask refinement affects model performance. This could probably be done with just about any working model, I would imagine.

\n

A few technical notes/questions:

\n", "rawMarkdown": "This is all very interesting. An obvious but important next step for anyone reading this is to see how this mask refinement affects model performance. This could probably be done with just about any working model, I would imagine.\n\nA few technical notes/questions:\n- What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.\n- The border shrinking seems fairly extreme. Did you try with thinner borders? I'm curious when the noise floor becomes too high. \n- It would also be interesting to see how this total effect is observed across characters. We have noticed that not all characters exhibit the same response in the baseline performance.\n- The data is 16bpc, but you're working with a 256 bin histogram. I'm curious if the trend is so cleanly visible at full dynamic range or if it needs the aggregation from being binned down.", "votes": null }, { "id": "2235071", "postDate": "04/25/2023 17:45:48", "content": "

We can reconstruct the \"true\" papyrus histogram by moving \"false\" ink pixel frequencies to papyrus pixel frequencies. As a result, the papyrus histogram appears as it should (roughly resembling a normal distribution).

\n

\"Papyrus

\n

The distance d between the peak of the ink (64) and the peak of the refined papyrus (120) shows how recognizable the letters are. The larger this distance, the more easily the letters can be identified. A larger distance between these peaks signifies a more significant contrast between the two features.

\n

When the contrast between the objects is higher, it becomes easier for the human eye and/or neural network to differentiate between them. This is because the color or intensity values of the two objects are more distinct from each other, allowing for better visibility and easier separation.

\n

A more detailed analysis of all slices leads to an interesting observation. This pattern is not present in every slide. In fact, each fragment has its range of slices where the distance 'd' is sufficient. It is important to note that if both peaks are close, it indicates that the ink is so deeply buried in the noise that it becomes undetectable.

\n

It seems reasonable that the detectable ink would appear in various locations across different scans. I managed to determine the following good slices. Some are significantly better than others, and I believe that selecting the optimal range for each slice is crucial in this competition.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
FragmentSlicesInk Peak
Fragment 121-3465
Fragment 225-3888
Fragment 320-3377
", "rawMarkdown": "We can reconstruct the \"true\" papyrus histogram by moving \"false\" ink pixel frequencies to papyrus pixel frequencies. As a result, the papyrus histogram appears as it should (roughly resembling a normal distribution).\n\n\n![Papyrus Reconstruction](https://user-images.githubusercontent.com/16557697/234257216-62c25fd2-6218-462d-8308-1cba7f268f63.png)\n\n\nThe distance `d` between the peak of the ink (64) and the peak of the refined papyrus (120) shows how recognizable the letters are. The larger this distance, the more easily the letters can be identified. A larger distance between these peaks signifies a more significant contrast between the two features.\n\nWhen the contrast between the objects is higher, it becomes easier for the human eye and/or neural network to differentiate between them. This is because the color or intensity values of the two objects are more distinct from each other, allowing for better visibility and easier separation.\n\nA more detailed analysis of all slices leads to an interesting observation. This pattern is not present in every slide. In fact, each fragment has its range of slices where the distance 'd' is sufficient. It is important to note that if both peaks are close, it indicates that the ink is so deeply buried in the noise that it becomes undetectable.\n\nIt seems reasonable that the detectable ink would appear in various locations across different scans. I managed to determine the following `good` slices. Some are significantly better than others, and I believe that selecting the optimal range for each slice is crucial in this competition.\n\n| Fragment | Slices | Ink Peak |\n|-------------|---------|-----------|\n| Fragment 1 | 21-34 | 65 |\n| Fragment 2 | 25-38 | 88 |\n| Fragment 3 | 20-33 | 77 |", "votes": null }, { "id": "2235077", "postDate": "04/25/2023 17:52:00", "content": "

Thank you for your remarks. I truly appreciate them.

\n
\n

What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.

\n
\n

It appears that every fragment has a unique range in which the inks are visible. A thorough analysis of all the slices may help identify the optimal range for each fragment. Kindly refer to my comment below for additional information

", "rawMarkdown": "Thank you for your remarks. I truly appreciate them.\n\n>What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.\n\nIt appears that every fragment has a unique range in which the inks are visible. A thorough analysis of all the slices may help identify the optimal range for each fragment. Kindly refer to my comment below for additional information", "votes": null }, { "id": "2235081", "postDate": "04/25/2023 17:58:49", "content": "

thanks a lot ! i was trying to find the best slices randomly !

", "rawMarkdown": "thanks a lot ! i was trying to find the best slices randomly !", "votes": null }, { "id": "2237178", "postDate": "04/27/2023 12:32:41", "content": "

You would see a thin edge when viewing a fragment from a lateral perspective. This animation demonstrates how fragment 1 appears from this viewpoint. It also shows the corresponding 1D ink mask. The ROI (Region of Interest) of the fragment (21-34) is highlighted.\"

\n

\"fragment1\"

\n

It's interesting if the neural net can learn from this 1D input rather than using 2D patches. If so, we could use all 65 channels with coordY convolution to assist the neural network in distinguishing between the channels.

", "rawMarkdown": "You would see a thin edge when viewing a fragment from a lateral perspective. This animation demonstrates how fragment 1 appears from this viewpoint. It also shows the corresponding 1D ink mask. The ROI (Region of Interest) of the fragment (21-34) is highlighted.\"\n\n![fragment1](https://user-images.githubusercontent.com/16557697/234857771-00283d9a-af24-406d-98e5-f1175dd25a4f.gif)\n\nIt's interesting if the neural net can learn from this 1D input rather than using 2D patches. If so, we could use all 65 channels with coordY convolution to assist the neural network in distinguishing between the channels.", "votes": null }, { "id": "2240436", "postDate": "04/30/2023 14:34:19", "content": "

It is very interesting. However my analysis of the histogram doesn't show any menaingful difference in distribution. It would be great if you could provide some code and explanation of how did you receive these results.
\nThere are also something that I would like you to elaborate:

\n
    \n
  1. Whenever you say channel - do you assume slice number?
    2. \n
  2. The gray-levels are in range 0..65535 for fragment 1, did you normalize them to 0..255 or you are using some different source of data?
    \nBelow are two examples of histogram of fragment 1 for slices 26 and 30. Images are cropped to remove boundary pixels.
    \nfragment 1, slice 26:
    \n\"\"
    \nfragment 1, slice 30:
    \n\"\"
    3. \n
", "rawMarkdown": "It is very interesting. However my analysis of the histogram doesn't show any menaingful difference in distribution. It would be great if you could provide some code and explanation of how did you receive these results.\nThere are also something that I would like you to elaborate:\n1. Whenever you say *channel* - do you assume slice number?\n2. The gray-levels are in range 0..65535 for fragment 1, did you normalize them to 0..255 or you are using some different source of data?\nBelow are two examples of histogram of fragment 1 for slices 26 and 30. Images are cropped to remove boundary pixels.\n**fragment 1, slice 26:**\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5613794%2F8854476a54dbd9445b426dad5ee1079a%2Fhistogram%20f%201%20s%2026.jpg?generation=1682867802742448&alt=media)\n**fragment 1, slice 30:**\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5613794%2Fef39e1d7ea39ef3223e609ff9f1cb3bb%2Fhistogram%20f%201%20s%2030.jpg?generation=1682867821586031&alt=media)", "votes": null }, { "id": "2240522", "postDate": "04/30/2023 15:52:14", "content": "

@pavelgonchar nice post and interesting insights! 👍 Thanks for sharing!🤝

", "rawMarkdown": "pavelgonchar nice post and interesting insights! 👍 Thanks for sharing!🤝", "votes": null }, { "id": "2240864", "postDate": "04/30/2023 23:25:34", "content": "

I believe that 2 components of the signal that we see are scroll and air. There is no distinguishable ink signal.
\nAlso there is a high chance that scrolls are scanned from the same direction from which infrared picture is taken, meaning layers 0-10 actually contain most ink information if any present at all.
\nI think only features around bottom of the scroll that are useful for letter detection are rare geometrical deformations of scroll surface from excessive pressure applied by tool used to apply ink on it.
\nThis deformation is probably what most notebooks focusing on layers 20-45 are learning to recognise.

", "rawMarkdown": "I believe that 2 components of the signal that we see are scroll and air. There is no distinguishable ink signal.\nAlso there is a high chance that scrolls are scanned from the same direction from which infrared picture is taken, meaning layers 0-10 actually contain most ink information if any present at all.\nI think only features around bottom of the scroll that are useful for letter detection are rare geometrical deformations of scroll surface from excessive pressure applied by tool used to apply ink on it.\nThis deformation is probably what most notebooks focusing on layers 20-45 are learning to recognise.", "votes": null }, { "id": "2247008", "postDate": "05/05/2023 16:02:08", "content": "

The EDA is really good! But how do we utilise it in training. As you said most ink pixels are beween 0-130 is it any usefull fore some sort of preprocessing? because i tried using only those pixels but didnt work

", "rawMarkdown": "The EDA is really good! But how do we utilise it in training. As you said most ink pixels are beween 0-130 is it any usefull fore some sort of preprocessing? because i tried using only those pixels but didnt work", "votes": null }, { "id": "2247810", "postDate": "05/06/2023 09:38:21", "content": "

it is \"obvious\" that the pixel label are not correct.
\nbelow shows how the model predicts as training iterations proceed.

\n

[url=https://ibb.co/c17zBbL][img]https://i.ibb.co/XbvmQy8/Selection-999-1945.png[/img][/url]
\n\"https://i.ibb.co/Ms3qv86/Selection-999-1945.png\"

\n

you can see the predictions are flip flopping, i.e. the model \"sometimes learn some background as link\"

\n

you can relabel using the intensity histogram as discussed or some semi-supervised methods (i.e. selective pesudo label)

", "rawMarkdown": "it is \"obvious\" that the pixel label are not correct.\nbelow shows how the model predicts as training iterations proceed.\n\n[url=https://ibb.co/c17zBbL][img]https://i.ibb.co/XbvmQy8/Selection-999-1945.png[/img][/url]\n![https://i.ibb.co/Ms3qv86/Selection-999-1945.png](https://i.ibb.co/Ms3qv86/Selection-999-1945.png)\n\nyou can see the predictions are flip flopping, i.e. the model \"sometimes learn some background as link\"\n\nyou can relabel using the intensity histogram as discussed or some semi-supervised methods (i.e. selective pesudo label)", "votes": null }, { "id": "2269304", "postDate": "05/22/2023 11:17:36", "content": "

what if the private dataset is labelled in the same way (I guess it is) ?
\nwouldn't it be better to let the model learn labels 'as is', which in a sense 'fills the letters' in the same way as in the private set?
\nwouldn't it be harder to first find the 'corrected ink labels' and then to postprocess to fill the letters, rather then let the model learn all at once?
\nI am guessing,

", "rawMarkdown": "what if the private dataset is labelled in the same way (I guess it is) ?\nwouldn't it be better to let the model learn labels 'as is', which in a sense 'fills the letters' in the same way as in the private set?\nwouldn't it be harder to first find the 'corrected ink labels' and then to postprocess to fill the letters, rather then let the model learn all at once? \nI am guessing,", "votes": null }, { "id": "2272242", "postDate": "05/24/2023 11:11:14", "content": "

If the model can extract abstract features, does this kind of \"label noise\" successfully compliment by the model's expressiveness? I have a feeling of the difficulty in training with \"noisy\" label is simply coming from the lack of expressiveness of the model.

\n

In other viewpoint, we can consider this competition's label is \"weak\" label. The provided label only depicts corse segmentation map, instead of precise saliency map.

", "rawMarkdown": "If the model can extract abstract features, does this kind of \"label noise\" successfully compliment by the model's expressiveness? I have a feeling of the difficulty in training with \"noisy\" label is simply coming from the lack of expressiveness of the model.\n\nIn other viewpoint, we can consider this competition's label is \"weak\" label. The provided label only depicts corse segmentation map, instead of precise saliency map.", "votes": null } ], "comments": [ { "id": 2230833, "author_name": "pavelgonchar", "author_url": "", "post_date": "04/22/2023 19:27:21", "content": "

Fragment 3

\n

The same pattern applies to Fragment 3, where ink and noise can be distinguished. The spectrum of ink colors spans from 0 to 140.

\n

\"Fragment

\n

\"GIF\"

", "votes": null, "replies": [ { "id": 2235071, "author_name": "pavelgonchar", "author_url": "", "post_date": "04/25/2023 17:45:48", "content": "

We can reconstruct the \"true\" papyrus histogram by moving \"false\" ink pixel frequencies to papyrus pixel frequencies. As a result, the papyrus histogram appears as it should (roughly resembling a normal distribution).

\n

\"Papyrus

\n

The distance d between the peak of the ink (64) and the peak of the refined papyrus (120) shows how recognizable the letters are. The larger this distance, the more easily the letters can be identified. A larger distance between these peaks signifies a more significant contrast between the two features.

\n

When the contrast between the objects is higher, it becomes easier for the human eye and/or neural network to differentiate between them. This is because the color or intensity values of the two objects are more distinct from each other, allowing for better visibility and easier separation.

\n

A more detailed analysis of all slices leads to an interesting observation. This pattern is not present in every slide. In fact, each fragment has its range of slices where the distance 'd' is sufficient. It is important to note that if both peaks are close, it indicates that the ink is so deeply buried in the noise that it becomes undetectable.

\n

It seems reasonable that the detectable ink would appear in various locations across different scans. I managed to determine the following good slices. Some are significantly better than others, and I believe that selecting the optimal range for each slice is crucial in this competition.

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
FragmentSlicesInk Peak
Fragment 121-3465
Fragment 225-3888
Fragment 320-3377
", "votes": null, "replies": [ { "id": 2235081, "author_name": "iraqbot", "author_url": "", "post_date": "04/25/2023 17:58:49", "content": "

thanks a lot ! i was trying to find the best slices randomly !

", "votes": null, "replies": [] } ] } ] }, { "id": 2231499, "author_name": "csparker", "author_url": "", "post_date": "04/23/2023 11:29:21", "content": "

This is all very interesting. An obvious but important next step for anyone reading this is to see how this mask refinement affects model performance. This could probably be done with just about any working model, I would imagine.

\n

A few technical notes/questions:

\n", "votes": null, "replies": [ { "id": 2235077, "author_name": "pavelgonchar", "author_url": "", "post_date": "04/25/2023 17:52:00", "content": "

Thank you for your remarks. I truly appreciate them.

\n
\n

What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.

\n
\n

It appears that every fragment has a unique range in which the inks are visible. A thorough analysis of all the slices may help identify the optimal range for each fragment. Kindly refer to my comment below for additional information

", "votes": null, "replies": [] } ] }, { "id": 2237178, "author_name": "pavelgonchar", "author_url": "", "post_date": "04/27/2023 12:32:41", "content": "

You would see a thin edge when viewing a fragment from a lateral perspective. This animation demonstrates how fragment 1 appears from this viewpoint. It also shows the corresponding 1D ink mask. The ROI (Region of Interest) of the fragment (21-34) is highlighted.\"

\n

\"fragment1\"

\n

It's interesting if the neural net can learn from this 1D input rather than using 2D patches. If so, we could use all 65 channels with coordY convolution to assist the neural network in distinguishing between the channels.

", "votes": null, "replies": [] }, { "id": 2240436, "author_name": "yurikreinin", "author_url": "", "post_date": "04/30/2023 14:34:19", "content": "

It is very interesting. However my analysis of the histogram doesn't show any menaingful difference in distribution. It would be great if you could provide some code and explanation of how did you receive these results.
\nThere are also something that I would like you to elaborate:

\n
    \n
  1. Whenever you say channel - do you assume slice number?
    2. \n
  2. The gray-levels are in range 0..65535 for fragment 1, did you normalize them to 0..255 or you are using some different source of data?
    \nBelow are two examples of histogram of fragment 1 for slices 26 and 30. Images are cropped to remove boundary pixels.
    \nfragment 1, slice 26:
    \n\"\"
    \nfragment 1, slice 30:
    \n\"\"
    3. \n
", "votes": null, "replies": [ { "id": 2240864, "author_name": "elvenmonk", "author_url": "", "post_date": "04/30/2023 23:25:34", "content": "

I believe that 2 components of the signal that we see are scroll and air. There is no distinguishable ink signal.
\nAlso there is a high chance that scrolls are scanned from the same direction from which infrared picture is taken, meaning layers 0-10 actually contain most ink information if any present at all.
\nI think only features around bottom of the scroll that are useful for letter detection are rare geometrical deformations of scroll surface from excessive pressure applied by tool used to apply ink on it.
\nThis deformation is probably what most notebooks focusing on layers 20-45 are learning to recognise.

", "votes": null, "replies": [] } ] }, { "id": 2240522, "author_name": "ivanisaev", "author_url": "", "post_date": "04/30/2023 15:52:14", "content": "

@pavelgonchar nice post and interesting insights! 👍 Thanks for sharing!🤝

", "votes": null, "replies": [] }, { "id": 2247008, "author_name": "khailashsanthakumar", "author_url": "", "post_date": "05/05/2023 16:02:08", "content": "

The EDA is really good! But how do we utilise it in training. As you said most ink pixels are beween 0-130 is it any usefull fore some sort of preprocessing? because i tried using only those pixels but didnt work

", "votes": null, "replies": [] }, { "id": 2247810, "author_name": "hengck23", "author_url": "", "post_date": "05/06/2023 09:38:21", "content": "

it is \"obvious\" that the pixel label are not correct.
\nbelow shows how the model predicts as training iterations proceed.

\n

[url=https://ibb.co/c17zBbL][img]https://i.ibb.co/XbvmQy8/Selection-999-1945.png[/img][/url]
\n\"https://i.ibb.co/Ms3qv86/Selection-999-1945.png\"

\n

you can see the predictions are flip flopping, i.e. the model \"sometimes learn some background as link\"

\n

you can relabel using the intensity histogram as discussed or some semi-supervised methods (i.e. selective pesudo label)

", "votes": null, "replies": [ { "id": 2269304, "author_name": "abdulkadirguner", "author_url": "", "post_date": "05/22/2023 11:17:36", "content": "

what if the private dataset is labelled in the same way (I guess it is) ?
\nwouldn't it be better to let the model learn labels 'as is', which in a sense 'fills the letters' in the same way as in the private set?
\nwouldn't it be harder to first find the 'corrected ink labels' and then to postprocess to fill the letters, rather then let the model learn all at once?
\nI am guessing,

", "votes": null, "replies": [] } ] }, { "id": 2272242, "author_name": "tatamikenn", "author_url": "", "post_date": "05/24/2023 11:11:14", "content": "

If the model can extract abstract features, does this kind of \"label noise\" successfully compliment by the model's expressiveness? I have a feeling of the difficulty in training with \"noisy\" label is simply coming from the lack of expressiveness of the model.

\n

In other viewpoint, we can consider this competition's label is \"weak\" label. The provided label only depicts corse segmentation map, instead of precise saliency map.

", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "2230650": "We noticed that the boundary pixels behave abnormally and do not contribute well to the color analysis. Therefore, we excluded the boundary and instead focused on the inner part of the fragment.\n\n![Trimming process](https://user-images.githubusercontent.com/16557697/233792973-1c53dd47-2ee8-4dd4-b460-0248ab7e3fb1.png)\n\nAfter analyzing the ink's color distribution of the first fragment, it is clear that there are two overlapping normal distributions.\n\nThe first Gaussian-like distribution, ranging from 0-130, probably represents the genuine ink distribution, while the second one, ranging from 0-255, seems to account for noise.\n\nThe noise may originate from the annotation process of the letters, where the annotator likely focused on outlining the shape of each letter rather than capturing every individual pixel. Furthermore, ink degradation and sparsity over time could have also contributed to the noise.\n\n\n## Fragment 1\n![Fragment 1](https://user-images.githubusercontent.com/16557697/233792717-2e0b1fd0-692d-40c5-91da-c5be40fb61ba.png)\n\n## Fragment 2\n\nFragment 2 is more challenging and its histogram requires a bit of linear transformation (roughly 0.6* x + 42) processing but ultimately conforms to the same pattern.\n![Fragment 2](https://user-images.githubusercontent.com/16557697/233793079-9b331fc6-c956-4ad6-974e-ee71bc3abf6b.png)\n\n\nA few insights:\n\n1) It is reasonable to assume that pixels of **fragment 1** with a color value greater than 130 are unlikely to be ink pixels and can be ignored.\n\n2) We can generate a refined ground truth (GT) mask for **fragment 1** by eliminating the papyrus pixels (>130).\n\n![Refined mask](https://user-images.githubusercontent.com/16557697/233793741-c2c91ff1-27db-4643-96c5-e63ae3231fd7.png)\n\n3) A refined mask for **Fragment 2** needs to be generated considering the inverse transformation corresponding to the mentioned histogram transformation.", "2230833": "## Fragment 3\n\nThe same pattern applies to Fragment 3, where ink and noise can be distinguished. The spectrum of ink colors spans from 0 to 140.\n\n![Fragment 3](https://user-images.githubusercontent.com/16557697/233802053-e74b36ac-1497-49dd-803b-74d1762f3a38.png)\n\n\n![GIF](https://user-images.githubusercontent.com/16557697/233802776-1aab36bb-f912-448c-94d7-abdb9e28a27d.gif)", "2231499": "This is all very interesting. An obvious but important next step for anyone reading this is to see how this mask refinement affects model performance. This could probably be done with just about any working model, I would imagine.\n\nA few technical notes/questions:\n- What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.\n- The border shrinking seems fairly extreme. Did you try with thinner borders? I'm curious when the noise floor becomes too high. \n- It would also be interesting to see how this total effect is observed across characters. We have noticed that not all characters exhibit the same response in the baseline performance.\n- The data is 16bpc, but you're working with a 256 bin histogram. I'm curious if the trend is so cleanly visible at full dynamic range or if it needs the aggregation from being binned down.", "2235071": "We can reconstruct the \"true\" papyrus histogram by moving \"false\" ink pixel frequencies to papyrus pixel frequencies. As a result, the papyrus histogram appears as it should (roughly resembling a normal distribution).\n\n\n![Papyrus Reconstruction](https://user-images.githubusercontent.com/16557697/234257216-62c25fd2-6218-462d-8308-1cba7f268f63.png)\n\n\nThe distance `d` between the peak of the ink (64) and the peak of the refined papyrus (120) shows how recognizable the letters are. The larger this distance, the more easily the letters can be identified. A larger distance between these peaks signifies a more significant contrast between the two features.\n\nWhen the contrast between the objects is higher, it becomes easier for the human eye and/or neural network to differentiate between them. This is because the color or intensity values of the two objects are more distinct from each other, allowing for better visibility and easier separation.\n\nA more detailed analysis of all slices leads to an interesting observation. This pattern is not present in every slide. In fact, each fragment has its range of slices where the distance 'd' is sufficient. It is important to note that if both peaks are close, it indicates that the ink is so deeply buried in the noise that it becomes undetectable.\n\nIt seems reasonable that the detectable ink would appear in various locations across different scans. I managed to determine the following `good` slices. Some are significantly better than others, and I believe that selecting the optimal range for each slice is crucial in this competition.\n\n| Fragment | Slices | Ink Peak |\n|-------------|---------|-----------|\n| Fragment 1 | 21-34 | 65 |\n| Fragment 2 | 25-38 | 88 |\n| Fragment 3 | 20-33 | 77 |", "2235077": "Thank you for your remarks. I truly appreciate them.\n\n>What are you calculating the histogram over? The full image stack for each fragment? Some subset of that? In the same way you shrank the boundary of the fragment, it would be interesting to shrink the image stack on the ends and see the effect on the signal.\n\nIt appears that every fragment has a unique range in which the inks are visible. A thorough analysis of all the slices may help identify the optimal range for each fragment. Kindly refer to my comment below for additional information", "2235081": "thanks a lot ! i was trying to find the best slices randomly !", "2237178": "You would see a thin edge when viewing a fragment from a lateral perspective. This animation demonstrates how fragment 1 appears from this viewpoint. It also shows the corresponding 1D ink mask. The ROI (Region of Interest) of the fragment (21-34) is highlighted.\"\n\n![fragment1](https://user-images.githubusercontent.com/16557697/234857771-00283d9a-af24-406d-98e5-f1175dd25a4f.gif)\n\nIt's interesting if the neural net can learn from this 1D input rather than using 2D patches. If so, we could use all 65 channels with coordY convolution to assist the neural network in distinguishing between the channels.", "2240436": "It is very interesting. However my analysis of the histogram doesn't show any menaingful difference in distribution. It would be great if you could provide some code and explanation of how did you receive these results.\nThere are also something that I would like you to elaborate:\n1. Whenever you say *channel* - do you assume slice number?\n2. The gray-levels are in range 0..65535 for fragment 1, did you normalize them to 0..255 or you are using some different source of data?\nBelow are two examples of histogram of fragment 1 for slices 26 and 30. Images are cropped to remove boundary pixels.\n**fragment 1, slice 26:**\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5613794%2F8854476a54dbd9445b426dad5ee1079a%2Fhistogram%20f%201%20s%2026.jpg?generation=1682867802742448&alt=media)\n**fragment 1, slice 30:**\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5613794%2Fef39e1d7ea39ef3223e609ff9f1cb3bb%2Fhistogram%20f%201%20s%2030.jpg?generation=1682867821586031&alt=media)", "2240522": "pavelgonchar nice post and interesting insights! 👍 Thanks for sharing!🤝", "2240864": "I believe that 2 components of the signal that we see are scroll and air. There is no distinguishable ink signal.\nAlso there is a high chance that scrolls are scanned from the same direction from which infrared picture is taken, meaning layers 0-10 actually contain most ink information if any present at all.\nI think only features around bottom of the scroll that are useful for letter detection are rare geometrical deformations of scroll surface from excessive pressure applied by tool used to apply ink on it.\nThis deformation is probably what most notebooks focusing on layers 20-45 are learning to recognise.", "2247008": "The EDA is really good! But how do we utilise it in training. As you said most ink pixels are beween 0-130 is it any usefull fore some sort of preprocessing? because i tried using only those pixels but didnt work", "2247810": "it is \"obvious\" that the pixel label are not correct.\nbelow shows how the model predicts as training iterations proceed.\n\n[url=https://ibb.co/c17zBbL][img]https://i.ibb.co/XbvmQy8/Selection-999-1945.png[/img][/url]\n![https://i.ibb.co/Ms3qv86/Selection-999-1945.png](https://i.ibb.co/Ms3qv86/Selection-999-1945.png)\n\nyou can see the predictions are flip flopping, i.e. the model \"sometimes learn some background as link\"\n\nyou can relabel using the intensity histogram as discussed or some semi-supervised methods (i.e. selective pesudo label)", "2269304": "what if the private dataset is labelled in the same way (I guess it is) ?\nwouldn't it be better to let the model learn labels 'as is', which in a sense 'fills the letters' in the same way as in the private set?\nwouldn't it be harder to first find the 'corrected ink labels' and then to postprocess to fill the letters, rather then let the model learn all at once? \nI am guessing,", "2272242": "If the model can extract abstract features, does this kind of \"label noise\" successfully compliment by the model's expressiveness? I have a feeling of the difficulty in training with \"noisy\" label is simply coming from the lack of expressiveness of the model.\n\nIn other viewpoint, we can consider this competition's label is \"weak\" label. The provided label only depicts corse segmentation map, instead of precise saliency map." }, "source": "meta" }