{ "id": 177447, "title": "Incorrect RescaleIntercept?", "url": "/competitions/osic-pulmonary-fibrosis-progression/discussion/177447", "author_name": "", "post_date": "2020-08-26T00:32:24.782740100Z", "votes": 2, "comment_count": 6, "views": 0, "content": "

I'm probably still not understanding something correct, for patient ID00026637202179561894768, image train/ID00026637202179561894768/138.dcm e.g., if I just plot the raw pixel array data, it look okay:
\n\"\"

\n

and its histogram plot of the pixel values
\n\"\"

\n

however, i think i need to convert this to Hounsfield Unit by multiply the slope and add the intercept, the slope for all our images are 1 so it can be ignored, but the intercept for this patient is 0, (also the Rescale type is US (unspecified) ), see below :
\n\"\"
\nthere is no way the hist can be scaled back with peak around -1000 (air), did I misunderstand something? Or since this is unspecified Rescale type, I shall just do whatever it takes to move the peak to -1000? e.g. by simply adding 1000 and the peak in the hist would move to roughly -1000

", "messages": [ { "id": "985701", "postDate": "08/26/2020 00:32:24", "content": "

I'm probably still not understanding something correct, for patient ID00026637202179561894768, image train/ID00026637202179561894768/138.dcm e.g., if I just plot the raw pixel array data, it look okay:
\n\"\"

\n

and its histogram plot of the pixel values
\n\"\"

\n

however, i think i need to convert this to Hounsfield Unit by multiply the slope and add the intercept, the slope for all our images are 1 so it can be ignored, but the intercept for this patient is 0, (also the Rescale type is US (unspecified) ), see below :
\n\"\"
\nthere is no way the hist can be scaled back with peak around -1000 (air), did I misunderstand something? Or since this is unspecified Rescale type, I shall just do whatever it takes to move the peak to -1000? e.g. by simply adding 1000 and the peak in the hist would move to roughly -1000

", "rawMarkdown": "I'm probably still not understanding something correct, for patient `ID00026637202179561894768`, image `train/ID00026637202179561894768/138.dcm` e.g., if I just plot the raw pixel array data, it look okay:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2F08bf2f058785705ef0c319390cecd19b%2Fshow.png?generation=1598401640873885&alt=media)\n\nand its histogram plot of the pixel values\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2F059a6a5eea500c97f67f0833f3038b29%2Fhist.png?generation=1598401664842784&alt=media)\n\nhowever, i think i need to convert this to Hounsfield Unit by multiply the slope and add the intercept, the slope for all our images are 1 so it can be ignored, but the intercept for this patient is 0, (also the Rescale type is US (unspecified) ), see below :\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2Ffdc64e2683c5d6ff1230496a588d0fd5%2FScreen%20Shot%202020-08-25%20at%208.27.02%20PM.png?generation=1598401740546543&alt=media)\nthere is no way the hist can be scaled back with peak around -1000 (air), did I misunderstand something? Or since this is unspecified Rescale type, I shall just do whatever it takes to move the peak to -1000? e.g. by simply adding 1000 and the peak in the hist would move to roughly -1000", "votes": null }, { "id": "986400", "postDate": "08/26/2020 13:03:02", "content": "

Hi, you are right. This HU is shifted in this scan (also in ID00128637202219474716089), although adding 1000 solved it for me and the histogram looked reasonable.

", "rawMarkdown": "Hi, you are right. This HU is shifted in this scan (also in ID00128637202219474716089), although adding 1000 solved it for me and the histogram looked reasonable.", "votes": null }, { "id": "986451", "postDate": "08/26/2020 14:05:22", "content": "

got it, thanks for confirming

", "rawMarkdown": "got it, thanks for confirming", "votes": null }, { "id": "992098", "postDate": "08/30/2020 23:14:37", "content": "

@ahmedhshahin Does the test set have these kinds of shifts?

\n

Should I just hard code in how to deal with these of is it worth detecting for them?

", "rawMarkdown": "ahmedhshahin Does the test set have these kinds of shifts?\n\nShould I just hard code in how to deal with these of is it worth detecting for them?", "votes": null }, { "id": "992105", "postDate": "08/30/2020 23:36:21", "content": "

It also looks like id ID00132637202222178761324 is shifted as well, but by a different amount, looks about 2000

", "rawMarkdown": "It also looks like id ID00132637202222178761324 is shifted as well, but by a different amount, looks about 2000", "votes": null }, { "id": "992790", "postDate": "08/31/2020 12:52:44", "content": "

Yes, I think hard coding them should work. By visually inspecting histograms of test set patients, there are no such cases there. Also in the training set, only these three cases exist.

", "rawMarkdown": "Yes, I think hard coding them should work. By visually inspecting histograms of test set patients, there are no such cases there. Also in the training set, only these three cases exist.", "votes": null }, { "id": "992812", "postDate": "08/31/2020 13:11:08", "content": "

Thank you for the clarification.

", "rawMarkdown": "Thank you for the clarification.", "votes": null } ], "comments": [ { "id": 986400, "author_name": "ahmedhshahin", "author_url": "", "post_date": "08/26/2020 13:03:02", "content": "

Hi, you are right. This HU is shifted in this scan (also in ID00128637202219474716089), although adding 1000 solved it for me and the histogram looked reasonable.

", "votes": null, "replies": [ { "id": 986451, "author_name": "samshipengs", "author_url": "", "post_date": "08/26/2020 14:05:22", "content": "

got it, thanks for confirming

", "votes": null, "replies": [] }, { "id": 992098, "author_name": "kipparker", "author_url": "", "post_date": "08/30/2020 23:14:37", "content": "

@ahmedhshahin Does the test set have these kinds of shifts?

\n

Should I just hard code in how to deal with these of is it worth detecting for them?

", "votes": null, "replies": [] }, { "id": 992105, "author_name": "kipparker", "author_url": "", "post_date": "08/30/2020 23:36:21", "content": "

It also looks like id ID00132637202222178761324 is shifted as well, but by a different amount, looks about 2000

", "votes": null, "replies": [] }, { "id": 992790, "author_name": "ahmedhshahin", "author_url": "", "post_date": "08/31/2020 12:52:44", "content": "

Yes, I think hard coding them should work. By visually inspecting histograms of test set patients, there are no such cases there. Also in the training set, only these three cases exist.

", "votes": null, "replies": [] }, { "id": 992812, "author_name": "kipparker", "author_url": "", "post_date": "08/31/2020 13:11:08", "content": "

Thank you for the clarification.

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "985701": "I'm probably still not understanding something correct, for patient `ID00026637202179561894768`, image `train/ID00026637202179561894768/138.dcm` e.g., if I just plot the raw pixel array data, it look okay:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2F08bf2f058785705ef0c319390cecd19b%2Fshow.png?generation=1598401640873885&alt=media)\n\nand its histogram plot of the pixel values\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2F059a6a5eea500c97f67f0833f3038b29%2Fhist.png?generation=1598401664842784&alt=media)\n\nhowever, i think i need to convert this to Hounsfield Unit by multiply the slope and add the intercept, the slope for all our images are 1 so it can be ignored, but the intercept for this patient is 0, (also the Rescale type is US (unspecified) ), see below :\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1878085%2Ffdc64e2683c5d6ff1230496a588d0fd5%2FScreen%20Shot%202020-08-25%20at%208.27.02%20PM.png?generation=1598401740546543&alt=media)\nthere is no way the hist can be scaled back with peak around -1000 (air), did I misunderstand something? Or since this is unspecified Rescale type, I shall just do whatever it takes to move the peak to -1000? e.g. by simply adding 1000 and the peak in the hist would move to roughly -1000", "986400": "Hi, you are right. This HU is shifted in this scan (also in ID00128637202219474716089), although adding 1000 solved it for me and the histogram looked reasonable.", "986451": "got it, thanks for confirming", "992098": "ahmedhshahin Does the test set have these kinds of shifts?\n\nShould I just hard code in how to deal with these of is it worth detecting for them?", "992105": "It also looks like id ID00132637202222178761324 is shifted as well, but by a different amount, looks about 2000", "992790": "Yes, I think hard coding them should work. By visually inspecting histograms of test set patients, there are no such cases there. Also in the training set, only these three cases exist.", "992812": "Thank you for the clarification." }, "source": "meta" }