{ "id": 413949, "title": "For those guys who scored > 0.2", "url": "/competitions/vesuvius-challenge-ink-detection/discussion/413949", "author_name": "TB Dukale", "post_date": "2023-05-30T19:35:05.432000", "votes": 5, "comment_count": 9, "views": 0, "content": "

is the given 'rle' function right?
\nif u notice it at the begining of the function it says 'flat_img = img.flatten()' which is actually flattening the given image in row-major (C-style) order
\nwasnt it correct if we use column-major style like… … 'flat_img = img.flatten('F')'?
\nand i have good prediction on the test images but have difficulty on the LB score (0.01-0.08). anyone faced this problem? how did u solved it?
\nthanks.

", "messages": [ { "id": 2281346, "postDate": "2023-05-30T19:35:05.433Z", "content": "

is the given 'rle' function right?
\nif u notice it at the begining of the function it says 'flat_img = img.flatten()' which is actually flattening the given image in row-major (C-style) order
\nwasnt it correct if we use column-major style like… … 'flat_img = img.flatten('F')'?
\nand i have good prediction on the test images but have difficulty on the LB score (0.01-0.08). anyone faced this problem? how did u solved it?
\nthanks.

", "rawMarkdown": "is the given 'rle' function right? \nif u notice it at the begining of the function it says 'flat_img = img.flatten()' which is actually flattening the given image **in row-major (C-style) order**\nwasnt it correct if we use **column-major style** like... ... 'flat_img = img.flatten('F')'?\nand i have good prediction on the test images but have difficulty on the LB score (0.01-0.08). anyone faced this problem? how did u solved it?\nthanks.", "votes": 5 }, { "id": 2290579, "postDate": "2023-06-07T00:45:46.273Z", "content": "

The tutorial notebook just uses the standard Numpy flatten command with no arguments.

", "rawMarkdown": "The tutorial notebook just uses the standard Numpy flatten command with no arguments.", "votes": 1, "replies": [ { "id": 2293005, "postDate": "2023-06-08T21:35:28.067Z", "content": "

yeah, i checked it and we just have to use the 'rle' function as it is.
\nthank u.

", "rawMarkdown": "yeah, i checked it and we just have to use the 'rle' function as it is. \nthank u." } ] }, { "id": 2282972, "postDate": "2023-06-01T01:27:13.110Z", "content": "

I've been using that function to check what my encoded rle looks like

\n
def decode_rle(rle, shape):\n    ar = np.zeros(shape[0]*shape[1], dtype=int)\n    pairs = rle.split()\n    for i in range(0,math.floor(len(pairs)/2)):\n        start = int(pairs[i*2])\n        count = int(pairs[i*2+1])\n        ar[start:start+count] = 255\n    new_img = np.array(ar)\n    new_img = np.reshape(new_img , shape)\n    new_img = new_img.astype('uint8')\n    new_img = PIL.Image.fromarray(new_img, mode='P')\n    plt.figure(figsize=(7, 7))\n    plt.imshow(new_img)\n    plt.axis('off')\n    plt.show()\n
\n

For example, you can look at one of the provided csv this way:

\n
file=io.open('/kaggle/input/vesuvius-challenge-ink-detection/train/1/inklabels_rle.csv', mode='r', encoding=\"utf-8\")\ncsv = file.read()\ndata = csv[15:]\ndecode_rle(data, (8181, 6330))\n
\n

Then you can pass your own rle after generating it, to see what you get

\n

I hope this helps some

", "rawMarkdown": "I've been using that function to check what my encoded rle looks like\n```\ndef decode_rle(rle, shape):\n ar = np.zeros(shape[0]*shape[1], dtype=int)\n pairs = rle.split()\n for i in range(0,math.floor(len(pairs)/2)):\n start = int(pairs[i*2])\n count = int(pairs[i*2+1])\n ar[start:start+count] = 255\n new_img = np.array(ar)\n new_img = np.reshape(new_img , shape)\n new_img = new_img.astype('uint8')\n new_img = PIL.Image.fromarray(new_img, mode='P')\n plt.figure(figsize=(7, 7))\n plt.imshow(new_img)\n plt.axis('off')\n plt.show()\n```\n\nFor example, you can look at one of the provided csv this way:\n```\nfile=io.open('/kaggle/input/vesuvius-challenge-ink-detection/train/1/inklabels_rle.csv', mode='r', encoding=\"utf-8\")\ncsv = file.read()\ndata = csv[15:]\ndecode_rle(data, (8181, 6330))\n```\n\nThen you can pass your own rle after generating it, to see what you get\n\nI hope this helps some", "votes": 2, "replies": [ { "id": 2283581, "postDate": "2023-06-01T11:13:32.293Z", "content": "

let me check it.
\nand Thank u!

", "rawMarkdown": "let me check it.\nand Thank u!" } ] }, { "id": 2281622, "postDate": "2023-05-31T03:21:19.510Z", "content": "

Many have this problem, check other discussions, I have not seen anyone posting that they had this issue and resolved it. But usually it is suggested to check overall dimensions of an image before flattening and to try to transpose an image before flattening. Also it is a good idea to take a look at high scoring notebooks or notebook that submits mask as a prediction for rle function that is known to work. Submitting mask in your notebook instead of actual prediction and not getting an 0.11 score is a good indicator that there is an issue in how you load or transform data or in rle function itself.

\n

I believe \"row-major (C-style) order\" is an expected order for submission in this competition which can be justified by the fact that this is how png, tiff formats store the data.

", "rawMarkdown": "Many have this problem, check other discussions, I have not seen anyone posting that they had this issue and resolved it. But usually it is suggested to check overall dimensions of an image before flattening and to try to transpose an image before flattening. Also it is a good idea to take a look at high scoring notebooks or notebook that submits mask as a prediction for rle function that is known to work. Submitting mask in your notebook instead of actual prediction and not getting an 0.11 score is a good indicator that there is an issue in how you load or transform data or in rle function itself.\n\nI believe \"row-major (C-style) order\" is an expected order for submission in this competition which can be justified by the fact that this is how png, tiff formats store the data.", "votes": 2, "replies": [ { "id": 2281742, "postDate": "2023-05-31T05:24:44.490Z", "content": "

Thank u sir! I will go back check my data preparation and training and try to resolve the scoring issue.
\nBut abt the 'rle': when u say Transpose before flattening, r we not changing the C-style to F-style?
\nThank u for the tip of submitting the mask and checking the score!

", "rawMarkdown": "Thank u sir! I will go back check my data preparation and training and try to resolve the scoring issue. \nBut abt the 'rle': when u say **Transpose** before flattening, r we not **changing the C-style to F-style?**\nThank u for the tip of submitting the mask and checking the score!", "replies": [ { "id": 2282721, "postDate": "2023-05-31T19:22:38.747Z", "content": "

First thing I would double check exact dimensions of image passed to rle and that it matches source data

", "rawMarkdown": "First thing I would double check exact dimensions of image passed to rle and that it matches source data" }, { "id": 2283580, "postDate": "2023-06-01T11:12:41.977Z", "content": "

i checked the shapes and it was correct. Thank u for ur advice.

", "rawMarkdown": "i checked the shapes and it was correct. Thank u for ur advice." } ] }, { "id": 2290576, "postDate": "2023-06-07T00:43:09.013Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 2290579, "author_name": "Matthew DeHaven", "author_url": "", "post_date": "2023-06-07T00:45:46.273000", "content": "

The tutorial notebook just uses the standard Numpy flatten command with no arguments.

", "votes": 1, "replies": [ { "id": 2293005, "author_name": "TB Dukale", "author_url": "", "post_date": "2023-06-08T21:35:28.067000", "content": "

yeah, i checked it and we just have to use the 'rle' function as it is.
\nthank u.

", "votes": 0, "replies": [] } ] }, { "id": 2282972, "author_name": "Jean-Fabien Barrois", "author_url": "", "post_date": "2023-06-01T01:27:13.110000", "content": "

I've been using that function to check what my encoded rle looks like

\n
def decode_rle(rle, shape):\n    ar = np.zeros(shape[0]*shape[1], dtype=int)\n    pairs = rle.split()\n    for i in range(0,math.floor(len(pairs)/2)):\n        start = int(pairs[i*2])\n        count = int(pairs[i*2+1])\n        ar[start:start+count] = 255\n    new_img = np.array(ar)\n    new_img = np.reshape(new_img , shape)\n    new_img = new_img.astype('uint8')\n    new_img = PIL.Image.fromarray(new_img, mode='P')\n    plt.figure(figsize=(7, 7))\n    plt.imshow(new_img)\n    plt.axis('off')\n    plt.show()\n
\n

For example, you can look at one of the provided csv this way:

\n
file=io.open('/kaggle/input/vesuvius-challenge-ink-detection/train/1/inklabels_rle.csv', mode='r', encoding=\"utf-8\")\ncsv = file.read()\ndata = csv[15:]\ndecode_rle(data, (8181, 6330))\n
\n

Then you can pass your own rle after generating it, to see what you get

\n

I hope this helps some

", "votes": 2, "replies": [ { "id": 2283581, "author_name": "TB Dukale", "author_url": "", "post_date": "2023-06-01T11:13:32.293000", "content": "

let me check it.
\nand Thank u!

", "votes": 0, "replies": [] } ] }, { "id": 2281622, "author_name": "Serhii Hrynko", "author_url": "", "post_date": "2023-05-31T03:21:19.510000", "content": "

Many have this problem, check other discussions, I have not seen anyone posting that they had this issue and resolved it. But usually it is suggested to check overall dimensions of an image before flattening and to try to transpose an image before flattening. Also it is a good idea to take a look at high scoring notebooks or notebook that submits mask as a prediction for rle function that is known to work. Submitting mask in your notebook instead of actual prediction and not getting an 0.11 score is a good indicator that there is an issue in how you load or transform data or in rle function itself.

\n

I believe \"row-major (C-style) order\" is an expected order for submission in this competition which can be justified by the fact that this is how png, tiff formats store the data.

", "votes": 2, "replies": [ { "id": 2281742, "author_name": "TB Dukale", "author_url": "", "post_date": "2023-05-31T05:24:44.490000", "content": "

Thank u sir! I will go back check my data preparation and training and try to resolve the scoring issue.
\nBut abt the 'rle': when u say Transpose before flattening, r we not changing the C-style to F-style?
\nThank u for the tip of submitting the mask and checking the score!

", "votes": 0, "replies": [ { "id": 2282721, "author_name": "Serhii Hrynko", "author_url": "", "post_date": "2023-05-31T19:22:38.747000", "content": "

First thing I would double check exact dimensions of image passed to rle and that it matches source data

", "votes": 0, "replies": [] }, { "id": 2283580, "author_name": "TB Dukale", "author_url": "", "post_date": "2023-06-01T11:12:41.977000", "content": "

i checked the shapes and it was correct. Thank u for ur advice.

", "votes": 0, "replies": [] } ] }, { "id": 2290576, "author_name": "", "author_url": "", "post_date": "2023-06-07T00:43:09.013000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2281346": "is the given 'rle' function right? \nif u notice it at the begining of the function it says 'flat_img = img.flatten()' which is actually flattening the given image **in row-major (C-style) order**\nwasnt it correct if we use **column-major style** like... ... 'flat_img = img.flatten('F')'?\nand i have good prediction on the test images but have difficulty on the LB score (0.01-0.08). anyone faced this problem? how did u solved it?\nthanks.", "2290579": "The tutorial notebook just uses the standard Numpy flatten command with no arguments.", "2282972": "I've been using that function to check what my encoded rle looks like\n```\ndef decode_rle(rle, shape):\n ar = np.zeros(shape[0]*shape[1], dtype=int)\n pairs = rle.split()\n for i in range(0,math.floor(len(pairs)/2)):\n start = int(pairs[i*2])\n count = int(pairs[i*2+1])\n ar[start:start+count] = 255\n new_img = np.array(ar)\n new_img = np.reshape(new_img , shape)\n new_img = new_img.astype('uint8')\n new_img = PIL.Image.fromarray(new_img, mode='P')\n plt.figure(figsize=(7, 7))\n plt.imshow(new_img)\n plt.axis('off')\n plt.show()\n```\n\nFor example, you can look at one of the provided csv this way:\n```\nfile=io.open('/kaggle/input/vesuvius-challenge-ink-detection/train/1/inklabels_rle.csv', mode='r', encoding=\"utf-8\")\ncsv = file.read()\ndata = csv[15:]\ndecode_rle(data, (8181, 6330))\n```\n\nThen you can pass your own rle after generating it, to see what you get\n\nI hope this helps some", "2281622": "Many have this problem, check other discussions, I have not seen anyone posting that they had this issue and resolved it. But usually it is suggested to check overall dimensions of an image before flattening and to try to transpose an image before flattening. Also it is a good idea to take a look at high scoring notebooks or notebook that submits mask as a prediction for rle function that is known to work. Submitting mask in your notebook instead of actual prediction and not getting an 0.11 score is a good indicator that there is an issue in how you load or transform data or in rle function itself.\n\nI believe \"row-major (C-style) order\" is an expected order for submission in this competition which can be justified by the fact that this is how png, tiff formats store the data." } }