If I take any frame of any of video, such as the first frame of devel01/K_1.avi and extract it as a bitmap I get a 320x240 B&W image with 211 unique colors.
\r\n\r\n
I make a histogram of the colors. Essentially, I count the number of pixels where Red == 1, the number where Red == 2, and so on. I've excerpted the beginning part of this table below. The left-hand column is the color index, with "0" being "black", the\r\n next column is Red, then Green, then Blue . There are a large number of counts assigned to RGB=[0,0,0], reflecting the proportionally large number of black pixels in this image.
\r\n\r\n
If you look at any individual column, you'll notice that there is a regular repeating "0" every seven positions. In the red column, these zeroes are at positions 5, 12, 19, 26, and so on. This pattern goes on for the entire histogram, and is fixed for all\r\n scenes in all videos I've examined so far (a very large number... it hasn't finished yet.)
\r\n\r\n
Normally I would consider this a quantization effect of the hardware. The hardware only has a resolution of 211 possible depths, and this is mapped into the 256 range of an RGB image. Zeroes are inserted, no problem, this is standard hardware behaviour.
\r\n\r\n
My question is this: If you compare the three columns, note that the Green and Blue columns also have zeroes... but these are\r\noffset from the zeroes in the red column. The green zeroes are two ahead of the red ones, and the blue zeroes are one behind.
\r\n\r\n
Taking a color as an example, there are 22 pixels which have Red == 6. All of these have Green == 8 and Blue == 5 in the image, indicating that the colors match the offsets in the table as well as the zeroes.
\r\n\r\n
I'm led to the hypothesis that the Kinect data has a bug in the transcription mechanism. The underlying hardware generates depth information quantized to 211 levels, but when this information is translated into RGB the color channels get "out of sync" with\r\n each other. The normalization algorithm is somehow skewed - averaging the R+G+B components is incorrect - one of the channels is correct, and the skewness of the other channels will introduce noise into the renormalization calculation.
\r\n\r\n
It would appear that the correct normalization procedure is to use one channel (probably Green) to calculate the original depth data.
\r\n\r\n
Taking Red == 1 as an example, the corresponding Green should be 3 and Blue should be -1. All pixels which have Red == 1 have Green = 3 and Blue == 0. This indicates that the offset in color intensities is clamped to the endpoints of the range.
\r\n\r\n
If this is indeed a transcription problem, correcting it would remove the clamping effect from the range endpoints, resulting in better resolution at the extreme ends of the measurement spectrum.
\r\n\r\n
0: 1270 1132 1297
\r\n 1: 27 98 17
\r\n 2: 17 40 15
\r\n 3: 15 27 15
\r\n 4: 15 17 0
\r\n 5: 0 15 22
\r\n 6: 22 15 14
\r\n 7: 14 0 13
\r\n 8: 13 22 15
\r\n 9: 15 14 13
\r\n 10: 13 13 9
\r\n 11: 9 15 0
\r\n 12: 0 13 2
\r\n 13: 2 9 5
\r\n 14: 5 0 4
\r\n 15: 4 2 2
\r\n 16: 2 5 4
\r\n 17: 4 4 3
\r\n 18: 3 2 0
\r\n 19: 0 4 2
\r\n 20: 2 3 2
\r\n 21: 2 0 2
\r\n 22: 2 2 3
\r\n 23: 3 2 4
\r\n 24: 4 2 4
\r\n 25: 4 3 0
\r\n 26: 0 4 4
\r\n 27: 4 4 2
\r\n 28: 2 0 7
\r\n 29: 7 4 2
\r\n 30: 2 2 7
\r\n 31: 7 7 5
\r\n 32: 5 2 0
", "messages": [ { "id": "7606", "postDate": "01/01/2012 04:07:59", "content": "
If I take any frame of any of video, such as the first frame of devel01/K_1.avi and extract it as a bitmap I get a 320x240 B&W image with 211 unique colors.
\r\n\r\n
I make a histogram of the colors. Essentially, I count the number of pixels where Red == 1, the number where Red == 2, and so on. I've excerpted the beginning part of this table below. The left-hand column is the color index, with "0" being "black", the\r\n next column is Red, then Green, then Blue . There are a large number of counts assigned to RGB=[0,0,0], reflecting the proportionally large number of black pixels in this image.
\r\n\r\n
If you look at any individual column, you'll notice that there is a regular repeating "0" every seven positions. In the red column, these zeroes are at positions 5, 12, 19, 26, and so on. This pattern goes on for the entire histogram, and is fixed for all\r\n scenes in all videos I've examined so far (a very large number... it hasn't finished yet.)
\r\n\r\n
Normally I would consider this a quantization effect of the hardware. The hardware only has a resolution of 211 possible depths, and this is mapped into the 256 range of an RGB image. Zeroes are inserted, no problem, this is standard hardware behaviour.
\r\n\r\n
My question is this: If you compare the three columns, note that the Green and Blue columns also have zeroes... but these are\r\noffset from the zeroes in the red column. The green zeroes are two ahead of the red ones, and the blue zeroes are one behind.
\r\n\r\n
Taking a color as an example, there are 22 pixels which have Red == 6. All of these have Green == 8 and Blue == 5 in the image, indicating that the colors match the offsets in the table as well as the zeroes.
\r\n\r\n
I'm led to the hypothesis that the Kinect data has a bug in the transcription mechanism. The underlying hardware generates depth information quantized to 211 levels, but when this information is translated into RGB the color channels get "out of sync" with\r\n each other. The normalization algorithm is somehow skewed - averaging the R+G+B components is incorrect - one of the channels is correct, and the skewness of the other channels will introduce noise into the renormalization calculation.
\r\n\r\n
It would appear that the correct normalization procedure is to use one channel (probably Green) to calculate the original depth data.
\r\n\r\n
Taking Red == 1 as an example, the corresponding Green should be 3 and Blue should be -1. All pixels which have Red == 1 have Green = 3 and Blue == 0. This indicates that the offset in color intensities is clamped to the endpoints of the range.
\r\n\r\n
If this is indeed a transcription problem, correcting it would remove the clamping effect from the range endpoints, resulting in better resolution at the extreme ends of the measurement spectrum.
\r\n\r\n
0: 1270 1132 1297
\r\n 1: 27 98 17
\r\n 2: 17 40 15
\r\n 3: 15 27 15
\r\n 4: 15 17 0
\r\n 5: 0 15 22
\r\n 6: 22 15 14
\r\n 7: 14 0 13
\r\n 8: 13 22 15
\r\n 9: 15 14 13
\r\n 10: 13 13 9
\r\n 11: 9 15 0
\r\n 12: 0 13 2
\r\n 13: 2 9 5
\r\n 14: 5 0 4
\r\n 15: 4 2 2
\r\n 16: 2 5 4
\r\n 17: 4 4 3
\r\n 18: 3 2 0
\r\n 19: 0 4 2
\r\n 20: 2 3 2
\r\n 21: 2 0 2
\r\n 22: 2 2 3
\r\n 23: 3 2 4
\r\n 24: 4 2 4
\r\n 25: 4 3 0
\r\n 26: 0 4 4
\r\n 27: 4 4 2
\r\n 28: 2 0 7
\r\n 29: 7 4 2
\r\n 30: 2 2 7
\r\n 31: 7 7 5
\r\n 32: 5 2 0
", "rawMarkdown": "", "votes": null }, { "id": "7625", "postDate": "01/02/2012 21:10:01", "content": "
Oh, I forgot to ask my question! :-)
\r\n\r\n
The first step in signal processing is to run the data through a transform which removes sensor effects.
\r\n\r\n
For the Kinect data, that would involve skipping over the periodic zeroes to reconstruct the data as a continuous spectrum of intensities.
\r\n\r\n
My question: Is this appropriate for all Kinects? In particular, will the final evaluation data have the repeating zeroes artifact?
", "rawMarkdown": "", "votes": null } ], "comments": [ { "id": 7625, "author_name": "rajstennajbarrabas", "author_url": "", "post_date": "01/02/2012 21:10:01", "content": "Oh, I forgot to ask my question! :-)
\r\n\r\n
The first step in signal processing is to run the data through a transform which removes sensor effects.
\r\n\r\n
For the Kinect data, that would involve skipping over the periodic zeroes to reconstruct the data as a continuous spectrum of intensities.
\r\n\r\n
My question: Is this appropriate for all Kinects? In particular, will the final evaluation data have the repeating zeroes artifact?
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "7606": "", "7625": "" }, "source": "meta" }