{ "id": 49386, "title": "Handcrafted feature", "url": "/competitions/sp-society-camera-model-identification/discussion/49386", "author_name": "", "post_date": "2018-02-10T07:01:14.078461500Z", "votes": 2, "comment_count": 5, "views": 0, "content": "

Data

\n\n

Org-Data\nPart1 . Training data provided by the organisers\nPart2. Collected from Flick, about 1000 images

\n\n

New Training Data(275/class) = [130~270/class] Part1 + [0~145/class] Part2.

\n\n

Post-Processing for New Training Data:

\n\n

JPEG compression with quality factor = [70, 90]

\n\n

resizing (via bicubic interpolation) by a factor of [0.5, 0.8, 1.5, 2.0]

\n\n

gamma correction using gamma = [0.8, 1.2]

\n\n

Features

\n\n

SRMQ Features to describe the Adjacent pixel correlation\nSRQM\nPRNU Features to generate the camera Fingerprint\nPRNU

\n\n

Classifier\nEnsemble Classifer\nClssifer

\n\n

Tricks in order

\n\n
    \n

  1. training a multi-classifier with SRMQ features for multi-post-processing, specially, for gamma correlation we use histgram feature to train CNN model(~99%)

    2. \n

  2. training 9 classifier with SRMQ featuresfor unalt images and manipulated images (~85.0%)

    3. \n

  3. finetune the results for unalt and manipulated images using PRNU feature(~94.8%)

    4. \n

  4. using test data(110/class) to train 9 classifier with SRMQ features for unalt and manipulated images(~95.7%/private VS ~96.49%/public)

    5. \n
\n\n

Inspiration

\n\n

Because of the limitation of time and hardware, we did not use DL method and high-dimension handcrafted features. We just test the CNN method for New Training Data(unalt data), the validation accuray is same to handcrafted feature(SRMQ). So for small dataset, handcrafted features should be a better choice.

\n\n

Note: it is important to classify different manipulations. And gamma correlation should be considering

\n\n

For tranditional methods,

\n\n
    \n

  1. high-dimension features would be test, such as SRM, RM, and so on;

    2. \n

  2. SVM classifier or other classiers would be used;

    3. \n
\n\n

For DL methods,

\n\n
    \n

  1. different image contents should be seperated.

    2. \n

  2. multi-stream networks would be a better choice.

    3. \n

  3. adding a convolutional layer into first layer of DL architecture.\nmore detail\nSource camera identification based on content-adaptive fusion residual networks

    4. \n
\n\n

Questions

\n\n
    \n

  1. In a real-world environment, there are a lot of manipulations for images, even multi-operations for one images. How to deal with it?

    2. \n

  2. The device of each models for testing is limitation, is it still work for other devices， not including training data?

    3. \n

  3. Device individual identification is a much harder and meanful task than device model identification, How to tranfer DL method for individual identification task?

    4. \n
\n\n

Thanks for IEEE's Signal Processing Society; Thanks for Prof. Stamm and Prof. Bestagini; Thanks for everyone~

", "messages": [ { "id": "280520", "postDate": "02/10/2018 07:01:14", "content": "

Data

\n\n

Org-Data\nPart1 . Training data provided by the organisers\nPart2. Collected from Flick, about 1000 images

\n\n

New Training Data(275/class) = [130~270/class] Part1 + [0~145/class] Part2.

\n\n

Post-Processing for New Training Data:

\n\n

JPEG compression with quality factor = [70, 90]

\n\n

resizing (via bicubic interpolation) by a factor of [0.5, 0.8, 1.5, 2.0]

\n\n

gamma correction using gamma = [0.8, 1.2]

\n\n

Features

\n\n

SRMQ Features to describe the Adjacent pixel correlation\nSRQM\nPRNU Features to generate the camera Fingerprint\nPRNU

\n\n

Classifier\nEnsemble Classifer\nClssifer

\n\n

Tricks in order

\n\n
    \n

  1. training a multi-classifier with SRMQ features for multi-post-processing, specially, for gamma correlation we use histgram feature to train CNN model(~99%)

    2. \n

  2. training 9 classifier with SRMQ featuresfor unalt images and manipulated images (~85.0%)

    3. \n

  3. finetune the results for unalt and manipulated images using PRNU feature(~94.8%)

    4. \n

  4. using test data(110/class) to train 9 classifier with SRMQ features for unalt and manipulated images(~95.7%/private VS ~96.49%/public)

    5. \n
\n\n

Inspiration

\n\n

Because of the limitation of time and hardware, we did not use DL method and high-dimension handcrafted features. We just test the CNN method for New Training Data(unalt data), the validation accuray is same to handcrafted feature(SRMQ). So for small dataset, handcrafted features should be a better choice.

\n\n

Note: it is important to classify different manipulations. And gamma correlation should be considering

\n\n

For tranditional methods,

\n\n
    \n

  1. high-dimension features would be test, such as SRM, RM, and so on;

    2. \n

  2. SVM classifier or other classiers would be used;

    3. \n
\n\n

For DL methods,

\n\n
    \n

  1. different image contents should be seperated.

    2. \n

  2. multi-stream networks would be a better choice.

    3. \n

  3. adding a convolutional layer into first layer of DL architecture.\nmore detail\nSource camera identification based on content-adaptive fusion residual networks

    4. \n
\n\n

Questions

\n\n
    \n

  1. In a real-world environment, there are a lot of manipulations for images, even multi-operations for one images. How to deal with it?

    2. \n

  2. The device of each models for testing is limitation, is it still work for other devices， not including training data?

    3. \n

  3. Device individual identification is a much harder and meanful task than device model identification, How to tranfer DL method for individual identification task?

    4. \n
\n\n

Thanks for IEEE's Signal Processing Society; Thanks for Prof. Stamm and Prof. Bestagini; Thanks for everyone~

", "rawMarkdown": "**Data**\n\n**Org-Data**\nPart1 . Training data provided by the organisers\nPart2. Collected from Flick, about 1000 images\n\n**New Training Data**(275/class) = [130~270/class] Part1 + [0~145/class] Part2.\n\n**Post-Processing for New Training Data:**\n\nJPEG compression with quality factor = [70, 90]\n\nresizing (via bicubic interpolation) by a factor of [0.5, 0.8, 1.5, 2.0]\n\ngamma correction using gamma = [0.8, 1.2]\n\n\n**Features**\n\nSRMQ Features to describe the Adjacent pixel correlation\n[SRQM][1]\nPRNU Features to generate the camera Fingerprint\n[PRNU][2]\n\n**Classifier**\nEnsemble Classifer\n[Clssifer][3]\n\n**Tricks in order**\n\n1. training a multi-classifier with SRMQ features for multi-post-processing, specially, for gamma correlation we use histgram feature to train CNN model(~99%)\n\n2. training 9 classifier with SRMQ featuresfor unalt images and manipulated images (~85.0%)\n\n3. finetune the results for unalt and manipulated images using PRNU feature(~94.8%)\n\n4. using test data(110/class) to train 9 classifier with SRMQ features for unalt and manipulated images(~95.7%/private VS ~96.49%/public)\n\n**Inspiration**\n\nBecause of the limitation of time and hardware, we did not use DL method and high-dimension handcrafted features. We just test the CNN method for New Training Data(unalt data), the validation accuray is same to handcrafted feature(SRMQ). So for small dataset, handcrafted features should be a better choice.\n\n**Note: it is important to classify different manipulations. And gamma correlation should be considering** \n\nFor tranditional methods, \n\n1. high-dimension features would be test, such as SRM, RM, and so on;\n\n2. SVM classifier or other classiers would be used;\n\n\nFor DL methods,\n\n1. different image contents should be seperated. \n\n2. multi-stream networks would be a better choice.\n\n3. adding a convolutional layer into first layer of DL architecture.\nmore detail\n[Source camera identification based on content-adaptive fusion residual networks][4]\n\n\n**Questions**\n\n1. In a real-world environment, there are a lot of manipulations for images, even multi-operations for one images. How to deal with it? \n\n2. The device of each models for testing is limitation, is it still work for other devices， not including training data? \n\n3. Device individual identification is a much harder and meanful task than device model identification, How to tranfer DL method for individual identification task?\n\n\n [1]: http://dde.binghamton.edu/download/feature_extractors/\n [2]: http://dde.binghamton.edu/download/camera_fingerprint/\n [3]: http://dde.binghamton.edu/download/ensemble/\n [4]: https://www.sciencedirect.com/science/article/pii/S0167865517303884?via%3Dihub\n\n\nThanks for IEEE's Signal Processing Society; Thanks for Prof. Stamm and Prof. Bestagini; Thanks for everyone~", "votes": null }, { "id": "280823", "postDate": "02/11/2018 05:56:53", "content": "

My answers:

\n\n
    \n

  1. Very nice question. I've been researching source attribution (not only for cameras) since 2014 and don't know your answer (actually nobody nows the right answer). The best possible in my opinion would have a classifier separated to classify pristine and manipulated images (Including most combinations of manipulations as possible). Just to have an idea of how difficult is the source attribution problem, for printers, the problem gets really worse, as scanners and different toner levels, paper ages, etc are involved.

    2. \n

  2. If you are saying individual devices of same brand and model of the training data, I would say that the accuracy could get a little similar. As some discussions here on Kaggle have mentioned, there were 'good jpegs' on additional data shared here (e.g., taken in the same conditions of the training data). Photos with flash, of homogeneous areas and even with the focus changed will confuse the classifiers.

    3. \n

  3. If you are meaning 'specific individual device' identification, my first idea would be using the PRNU as input for CNNs. Dont know if it could work.

    4. \n
\n\n

Cheers!

", "rawMarkdown": "My answers:\n\n1. Very nice question. I've been researching source attribution (not only for cameras) since 2014 and don't know your answer (actually nobody nows the right answer). The best possible in my opinion would have a classifier separated to classify pristine and manipulated images (Including most combinations of manipulations as possible). Just to have an idea of how difficult is the source attribution problem, for printers, the problem gets really worse, as scanners and different toner levels, paper ages, etc are involved.\n\n2. If you are saying individual devices of same brand and model of the training data, I would say that the accuracy could get a little similar. As some discussions here on Kaggle have mentioned, there were 'good jpegs' on additional data shared here (e.g., taken in the same conditions of the training data). Photos with flash, of homogeneous areas and even with the focus changed will confuse the classifiers.\n\n3. If you are meaning 'specific individual device' identification, my first idea would be using the PRNU as input for CNNs. Dont know if it could work.\n\nCheers!", "votes": null }, { "id": "280838", "postDate": "02/11/2018 07:02:29", "content": "

Thanks for your answers. About your suggestions, I have some things want to discuss.

\n\n
    \n

  1. It should not be a hard thing to separate the pristine and manipulated images. The difficult thing, I think, is to have an Fine classification for manipulated images. The model training from manipulated images with Gamma_0.8 doesnot work for the images with Gamma_1.2. So it is necessary for handcrafted features to have an Fine classification. However, in real life, there are a lot of image processions to generate different manipulated images. Maybe the method based on DL would be deal with it, just put as many manipulated images as possible into DL architectures. However, it is not an clear way for researching.

    2. \n

  2. In my experiments, I added some new images into training data, 0-5 devices/each model and 20-50 images/each device and got an new training data. I trained two model from new training data and training data provided by the Organizer. The model trained by new training data has a much better performance, which maybe means that: the more the data is, the better the performance is. However, in real life, there are thousands of devices with same model. It is impossible to collect the images captured by all devices. If we want to get an effective model to distinguish camera model(for unalt images), how many devices and images for each model should be used?

    3. \n

  3. we have test on it, put the PRNU into CNNs. It is not a good way for individual device identification in the case of lower-resolution. We think that the reason is that a lot of information are droped after filtering. So we just add an convolutional layer without actiation function, BN, pooling to learn better filter. However, the performance for individual device identifaction with same model is still not good enough.

    4. \n
", "rawMarkdown": "Thanks for your answers. About your suggestions, I have some things want to discuss.\n\n1. It should not be a hard thing to separate the pristine and manipulated images. The difficult thing, I think, is to have an Fine classification for manipulated images. The model training from manipulated images with Gamma_0.8 doesnot work for the images with Gamma_1.2. So it is necessary for handcrafted features to have an Fine classification. However, in real life, there are a lot of image processions to generate different manipulated images. Maybe the method based on DL would be deal with it, just put as many manipulated images as possible into DL architectures. However, it is not an clear way for researching.\n\n2. In my experiments, I added some new images into training data, 0-5 devices/each model and 20-50 images/each device and got an new training data. I trained two model from new training data and training data provided by the Organizer. The model trained by new training data has a much better performance, which maybe means that: the more the data is, the better the performance is. However, in real life, there are thousands of devices with same model. It is impossible to collect the images captured by all devices. If we want to get an effective model to distinguish camera model(for unalt images), how many devices and images for each model should be used?\n\n3. we have test on it, put the PRNU into CNNs. It is not a good way for individual device identification in the case of lower-resolution. We think that the reason is that a lot of information are droped after filtering. So we just add an convolutional layer without actiation function, BN, pooling to learn better filter. However, the performance for individual device identifaction with same model is still not good enough.", "votes": null }, { "id": "280847", "postDate": "02/11/2018 07:14:58", "content": "
    \n

  1. I agree with you. What is a fine model for manipulated images if there are infinite ways of manipulating an image? that's why I believe an ensemble solution rather than just one approach must be used for real-life situations.

    2. \n

  2. I don't know. It is a try and error investigation. Maybe some models and brands require more devices, others less, others just one, etc.

    3. \n

  3. It was just an idea. Is the CNN you are using the best for this problem? if you used the same that gave you 112th place I would say it isn't ;-). Do you believe just one CNN is enough? So, you must think about it, there are so many ideas to do that :-)

    4. \n
\n\n

Nice researching!

\n\n

Best regards,

", "rawMarkdown": "1. I agree with you. What is a fine model for manipulated images if there are infinite ways of manipulating an image? that's why I believe an ensemble solution rather than just one approach must be used for real-life situations.\n\n2. I don't know. It is a try and error investigation. Maybe some models and brands require more devices, others less, others just one, etc. \n\n3. It was just an idea. Is the CNN you are using the best for this problem? if you used the same that gave you 112th place I would say it isn't ;-). Do you believe just one CNN is enough? So, you must think about it, there are so many ideas to do that :-)\n\nNice researching!\n\nBest regards,", "votes": null }, { "id": "280850", "postDate": "02/11/2018 07:27:47", "content": "

Thanks for your reply. As mentioned above, we did not use CNN-based method, Just traditional schemes were used for this competition.

\n\n

Thank you very much~

", "rawMarkdown": "Thanks for your reply. As mentioned above, we did not use CNN-based method, Just traditional schemes were used for this competition. \n\nThank you very much~", "votes": null }, { "id": "280851", "postDate": "02/11/2018 07:30:22", "content": "

Thank you too!

\n\n

I would suggest you to try an Inception-based CNN for feeding PRNU as input. I am curious to know if it could help on specific camera identification.

\n\n

Bests,

", "rawMarkdown": "Thank you too!\n\nI would suggest you to try an Inception-based CNN for feeding PRNU as input. I am curious to know if it could help on specific camera identification.\n\nBests,", "votes": null } ], "comments": [ { "id": 280823, "author_name": "anselmoferreira35", "author_url": "", "post_date": "02/11/2018 05:56:53", "content": "

My answers:

\n\n
    \n

  1. Very nice question. I've been researching source attribution (not only for cameras) since 2014 and don't know your answer (actually nobody nows the right answer). The best possible in my opinion would have a classifier separated to classify pristine and manipulated images (Including most combinations of manipulations as possible). Just to have an idea of how difficult is the source attribution problem, for printers, the problem gets really worse, as scanners and different toner levels, paper ages, etc are involved.

    2. \n

  2. If you are saying individual devices of same brand and model of the training data, I would say that the accuracy could get a little similar. As some discussions here on Kaggle have mentioned, there were 'good jpegs' on additional data shared here (e.g., taken in the same conditions of the training data). Photos with flash, of homogeneous areas and even with the focus changed will confuse the classifiers.

    3. \n

  3. If you are meaning 'specific individual device' identification, my first idea would be using the PRNU as input for CNNs. Dont know if it could work.

    4. \n
\n\n

Cheers!

", "votes": null, "replies": [ { "id": 280838, "author_name": "meprobjtu", "author_url": "", "post_date": "02/11/2018 07:02:29", "content": "

Thanks for your answers. About your suggestions, I have some things want to discuss.

\n\n
    \n

  1. It should not be a hard thing to separate the pristine and manipulated images. The difficult thing, I think, is to have an Fine classification for manipulated images. The model training from manipulated images with Gamma_0.8 doesnot work for the images with Gamma_1.2. So it is necessary for handcrafted features to have an Fine classification. However, in real life, there are a lot of image processions to generate different manipulated images. Maybe the method based on DL would be deal with it, just put as many manipulated images as possible into DL architectures. However, it is not an clear way for researching.

    2. \n

  2. In my experiments, I added some new images into training data, 0-5 devices/each model and 20-50 images/each device and got an new training data. I trained two model from new training data and training data provided by the Organizer. The model trained by new training data has a much better performance, which maybe means that: the more the data is, the better the performance is. However, in real life, there are thousands of devices with same model. It is impossible to collect the images captured by all devices. If we want to get an effective model to distinguish camera model(for unalt images), how many devices and images for each model should be used?

    3. \n

  3. we have test on it, put the PRNU into CNNs. It is not a good way for individual device identification in the case of lower-resolution. We think that the reason is that a lot of information are droped after filtering. So we just add an convolutional layer without actiation function, BN, pooling to learn better filter. However, the performance for individual device identifaction with same model is still not good enough.

    4. \n
", "votes": null, "replies": [] }, { "id": 280847, "author_name": "anselmoferreira35", "author_url": "", "post_date": "02/11/2018 07:14:58", "content": "
    \n

  1. I agree with you. What is a fine model for manipulated images if there are infinite ways of manipulating an image? that's why I believe an ensemble solution rather than just one approach must be used for real-life situations.

    2. \n

  2. I don't know. It is a try and error investigation. Maybe some models and brands require more devices, others less, others just one, etc.

    3. \n

  3. It was just an idea. Is the CNN you are using the best for this problem? if you used the same that gave you 112th place I would say it isn't ;-). Do you believe just one CNN is enough? So, you must think about it, there are so many ideas to do that :-)

    4. \n
\n\n

Nice researching!

\n\n

Best regards,

", "votes": null, "replies": [] }, { "id": 280850, "author_name": "meprobjtu", "author_url": "", "post_date": "02/11/2018 07:27:47", "content": "

Thanks for your reply. As mentioned above, we did not use CNN-based method, Just traditional schemes were used for this competition.

\n\n

Thank you very much~

", "votes": null, "replies": [] }, { "id": 280851, "author_name": "anselmoferreira35", "author_url": "", "post_date": "02/11/2018 07:30:22", "content": "

Thank you too!

\n\n

I would suggest you to try an Inception-based CNN for feeding PRNU as input. I am curious to know if it could help on specific camera identification.

\n\n

Bests,

", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "280520": "**Data**\n\n**Org-Data**\nPart1 . Training data provided by the organisers\nPart2. Collected from Flick, about 1000 images\n\n**New Training Data**(275/class) = [130~270/class] Part1 + [0~145/class] Part2.\n\n**Post-Processing for New Training Data:**\n\nJPEG compression with quality factor = [70, 90]\n\nresizing (via bicubic interpolation) by a factor of [0.5, 0.8, 1.5, 2.0]\n\ngamma correction using gamma = [0.8, 1.2]\n\n\n**Features**\n\nSRMQ Features to describe the Adjacent pixel correlation\n[SRQM][1]\nPRNU Features to generate the camera Fingerprint\n[PRNU][2]\n\n**Classifier**\nEnsemble Classifer\n[Clssifer][3]\n\n**Tricks in order**\n\n1. training a multi-classifier with SRMQ features for multi-post-processing, specially, for gamma correlation we use histgram feature to train CNN model(~99%)\n\n2. training 9 classifier with SRMQ featuresfor unalt images and manipulated images (~85.0%)\n\n3. finetune the results for unalt and manipulated images using PRNU feature(~94.8%)\n\n4. using test data(110/class) to train 9 classifier with SRMQ features for unalt and manipulated images(~95.7%/private VS ~96.49%/public)\n\n**Inspiration**\n\nBecause of the limitation of time and hardware, we did not use DL method and high-dimension handcrafted features. We just test the CNN method for New Training Data(unalt data), the validation accuray is same to handcrafted feature(SRMQ). So for small dataset, handcrafted features should be a better choice.\n\n**Note: it is important to classify different manipulations. And gamma correlation should be considering** \n\nFor tranditional methods, \n\n1. high-dimension features would be test, such as SRM, RM, and so on;\n\n2. SVM classifier or other classiers would be used;\n\n\nFor DL methods,\n\n1. different image contents should be seperated. \n\n2. multi-stream networks would be a better choice.\n\n3. adding a convolutional layer into first layer of DL architecture.\nmore detail\n[Source camera identification based on content-adaptive fusion residual networks][4]\n\n\n**Questions**\n\n1. In a real-world environment, there are a lot of manipulations for images, even multi-operations for one images. How to deal with it? \n\n2. The device of each models for testing is limitation, is it still work for other devices， not including training data? \n\n3. Device individual identification is a much harder and meanful task than device model identification, How to tranfer DL method for individual identification task?\n\n\n [1]: http://dde.binghamton.edu/download/feature_extractors/\n [2]: http://dde.binghamton.edu/download/camera_fingerprint/\n [3]: http://dde.binghamton.edu/download/ensemble/\n [4]: https://www.sciencedirect.com/science/article/pii/S0167865517303884?via%3Dihub\n\n\nThanks for IEEE's Signal Processing Society; Thanks for Prof. Stamm and Prof. Bestagini; Thanks for everyone~", "280823": "My answers:\n\n1. Very nice question. I've been researching source attribution (not only for cameras) since 2014 and don't know your answer (actually nobody nows the right answer). The best possible in my opinion would have a classifier separated to classify pristine and manipulated images (Including most combinations of manipulations as possible). Just to have an idea of how difficult is the source attribution problem, for printers, the problem gets really worse, as scanners and different toner levels, paper ages, etc are involved.\n\n2. If you are saying individual devices of same brand and model of the training data, I would say that the accuracy could get a little similar. As some discussions here on Kaggle have mentioned, there were 'good jpegs' on additional data shared here (e.g., taken in the same conditions of the training data). Photos with flash, of homogeneous areas and even with the focus changed will confuse the classifiers.\n\n3. If you are meaning 'specific individual device' identification, my first idea would be using the PRNU as input for CNNs. Dont know if it could work.\n\nCheers!", "280838": "Thanks for your answers. About your suggestions, I have some things want to discuss.\n\n1. It should not be a hard thing to separate the pristine and manipulated images. The difficult thing, I think, is to have an Fine classification for manipulated images. The model training from manipulated images with Gamma_0.8 doesnot work for the images with Gamma_1.2. So it is necessary for handcrafted features to have an Fine classification. However, in real life, there are a lot of image processions to generate different manipulated images. Maybe the method based on DL would be deal with it, just put as many manipulated images as possible into DL architectures. However, it is not an clear way for researching.\n\n2. In my experiments, I added some new images into training data, 0-5 devices/each model and 20-50 images/each device and got an new training data. I trained two model from new training data and training data provided by the Organizer. The model trained by new training data has a much better performance, which maybe means that: the more the data is, the better the performance is. However, in real life, there are thousands of devices with same model. It is impossible to collect the images captured by all devices. If we want to get an effective model to distinguish camera model(for unalt images), how many devices and images for each model should be used?\n\n3. we have test on it, put the PRNU into CNNs. It is not a good way for individual device identification in the case of lower-resolution. We think that the reason is that a lot of information are droped after filtering. So we just add an convolutional layer without actiation function, BN, pooling to learn better filter. However, the performance for individual device identifaction with same model is still not good enough.", "280847": "1. I agree with you. What is a fine model for manipulated images if there are infinite ways of manipulating an image? that's why I believe an ensemble solution rather than just one approach must be used for real-life situations.\n\n2. I don't know. It is a try and error investigation. Maybe some models and brands require more devices, others less, others just one, etc. \n\n3. It was just an idea. Is the CNN you are using the best for this problem? if you used the same that gave you 112th place I would say it isn't ;-). Do you believe just one CNN is enough? So, you must think about it, there are so many ideas to do that :-)\n\nNice researching!\n\nBest regards,", "280850": "Thanks for your reply. As mentioned above, we did not use CNN-based method, Just traditional schemes were used for this competition. \n\nThank you very much~", "280851": "Thank you too!\n\nI would suggest you to try an Inception-based CNN for feeding PRNU as input. I am curious to know if it could help on specific camera identification.\n\nBests," }, "source": "meta" }