{ "id": 22011, "title": "Precomputed Wavelet image hashes", "url": "/competitions/avito-duplicate-ads-detection/discussion/22011", "author_name": "dmpetrov", "post_date": "2016-07-02T16:48:34.957000", "votes": 14, "comment_count": 6, "views": 1532, "content": "

I implemented whash() - Wavelet image hash - in the imagehash Python library: https://github.com/JohannesBuchner/imagehash

\n\n

The implementation is presented in the new version of imagehash library - 2.1 https://pypi.python.org/pypi/ImageHash

\n\n

My blog post about whash(): https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/

\n\n

A precomluted whashes are attached as pickle file. The simples Haar hash was calculated with 8x8 hash size (by default parameters in the whash method).

\n\n

For my Kaggle score the whash brought only +0.04% to AUC metric compered to phash, in addition to my current ~92.9% result. You can try another method parameters or use this result in ensemble stage.

\n\n

Usage in Python:

\n\n

whash = pickle.load( open('whash_haar.pkl', 'rb') ).

\n\n

list(tmp.items())[:3]

\n\n
\n

[(2, 8034173447716605444),\n (5, 11459270170091060735),\n (7, 18400019274820157450)]

\n
", "messages": [ { "id": 125788, "postDate": "2016-07-02T16:48:34.957Z", "content": "

I implemented whash() - Wavelet image hash - in the imagehash Python library: https://github.com/JohannesBuchner/imagehash

\n\n

The implementation is presented in the new version of imagehash library - 2.1 https://pypi.python.org/pypi/ImageHash

\n\n

My blog post about whash(): https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/

\n\n

A precomluted whashes are attached as pickle file. The simples Haar hash was calculated with 8x8 hash size (by default parameters in the whash method).

\n\n

For my Kaggle score the whash brought only +0.04% to AUC metric compered to phash, in addition to my current ~92.9% result. You can try another method parameters or use this result in ensemble stage.

\n\n

Usage in Python:

\n\n

whash = pickle.load( open('whash_haar.pkl', 'rb') ).

\n\n

list(tmp.items())[:3]

\n\n
\n

[(2, 8034173447716605444),\n (5, 11459270170091060735),\n (7, 18400019274820157450)]

\n
", "rawMarkdown": "I implemented whash() - Wavelet image hash - in the imagehash Python library: https://github.com/JohannesBuchner/imagehash\r\n\r\nThe implementation is presented in the new version of imagehash library - 2.1 https://pypi.python.org/pypi/ImageHash\r\n\r\nMy blog post about whash(): https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/\r\n\r\nA precomluted whashes are attached as pickle file. The simples Haar hash was calculated with 8x8 hash size (by default parameters in the whash method).\r\n\r\nFor my Kaggle score the whash brought only +0.04% to AUC metric compered to phash, in addition to my current ~92.9% result. You can try another method parameters or use this result in ensemble stage.\r\n\r\nUsage in Python: \r\n\r\nwhash = pickle.load( open('whash_haar.pkl', 'rb') ).\r\n\r\nlist(tmp.items())[:3]\r\n> [(2, 8034173447716605444),\r\n (5, 11459270170091060735),\r\n (7, 18400019274820157450)]", "votes": 14 }, { "id": 126567, "postDate": "2016-07-09T21:36:55.847Z", "content": "

Hey Dmitry, I'm doing a project now for college and based it on your technique and the other hashing, great work ! One thing, on the pypi website whash links to the Kaggle competition. It should link to your tutorial as that has a great explanation.

", "rawMarkdown": "Hey Dmitry, I'm doing a project now for college and based it on your technique and the other hashing, great work ! One thing, on the [pypi website][1] whash links to the Kaggle competition. It should link to your tutorial as that has a great explanation.\r\n\r\n\r\n [1]: https://pypi.python.org/pypi/ImageHash", "votes": 1 }, { "id": 126527, "postDate": "2016-07-09T07:29:41.407Z", "content": "

Hi Lin, you are absolutely right. Thank you for catching that!

\n\n

It is a typo. I was unable to find the original test1.jpg and test2.jpg images that I used. So, I recalculated hashes for images 7993417.jpg and 7803947.jpg from the dataset:

\n\n

hash1 = 354adab5054af0b7

\n\n

hash2 = 5b7724c8bb364551

\n\n

hash1-hash2

\n\n

'> 44

\n\n

The post was updated and I provided a link to your comment: https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/

", "rawMarkdown": "Hi Lin, you are absolutely right. Thank you for catching that!\r\n\r\nIt is a typo. I was unable to find the original test1.jpg and test2.jpg images that I used. So, I recalculated hashes for images 7993417.jpg and 7803947.jpg from the dataset:\r\n\r\nhash1 = 354adab5054af0b7\r\n\r\nhash2 = 5b7724c8bb364551\r\n\r\nhash1-hash2\r\n\r\n'> 44\r\n\r\nThe post was updated and I provided a link to your comment: https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/", "votes": 1 }, { "id": 126571, "postDate": "2016-07-09T23:01:06.347Z", "content": "

Hey Darragh, thank you for paying attention to this...

\n\n

I have already changed the link in the project README.rst file. It should be merged to master branch soon. And, yes, I will ask Johannes (imagehash project author) to change the link in pypi website.

", "rawMarkdown": "Hey Darragh, thank you for paying attention to this...\r\n\r\nI have already changed the link in the project README.rst file. It should be merged to master branch soon. And, yes, I will ask Johannes (imagehash project author) to change the link in pypi website." }, { "id": 126529, "postDate": "2016-07-09T08:30:37.287Z", "rawMarkdown": "", "isDeleted": true }, { "id": 126522, "postDate": "2016-07-09T05:55:01.673Z", "rawMarkdown": "", "votes": 1, "isDeleted": true }, { "id": 126240, "postDate": "2016-07-07T05:00:32.320Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 126567, "author_name": "Darragh", "author_url": "", "post_date": "2016-07-09T21:36:55.847000", "content": "

Hey Dmitry, I'm doing a project now for college and based it on your technique and the other hashing, great work ! One thing, on the pypi website whash links to the Kaggle competition. It should link to your tutorial as that has a great explanation.

", "votes": 1, "replies": [] }, { "id": 126527, "author_name": "dmpetrov", "author_url": "", "post_date": "2016-07-09T07:29:41.407000", "content": "

Hi Lin, you are absolutely right. Thank you for catching that!

\n\n

It is a typo. I was unable to find the original test1.jpg and test2.jpg images that I used. So, I recalculated hashes for images 7993417.jpg and 7803947.jpg from the dataset:

\n\n

hash1 = 354adab5054af0b7

\n\n

hash2 = 5b7724c8bb364551

\n\n

hash1-hash2

\n\n

'> 44

\n\n

The post was updated and I provided a link to your comment: https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/

", "votes": 1, "replies": [] }, { "id": 126571, "author_name": "dmpetrov", "author_url": "", "post_date": "2016-07-09T23:01:06.347000", "content": "

Hey Darragh, thank you for paying attention to this...

\n\n

I have already changed the link in the project README.rst file. It should be merged to master branch soon. And, yes, I will ask Johannes (imagehash project author) to change the link in pypi website.

", "votes": 0, "replies": [] }, { "id": 126529, "author_name": "", "author_url": "", "post_date": "2016-07-09T08:30:37.287000", "content": "", "votes": 0, "replies": [] }, { "id": 126522, "author_name": "", "author_url": "", "post_date": "2016-07-09T05:55:01.673000", "content": "", "votes": 1, "replies": [] }, { "id": 126240, "author_name": "", "author_url": "", "post_date": "2016-07-07T05:00:32.320000", "content": "", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "125788": "I implemented whash() - Wavelet image hash - in the imagehash Python library: https://github.com/JohannesBuchner/imagehash\r\n\r\nThe implementation is presented in the new version of imagehash library - 2.1 https://pypi.python.org/pypi/ImageHash\r\n\r\nMy blog post about whash(): https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/\r\n\r\nA precomluted whashes are attached as pickle file. The simples Haar hash was calculated with 8x8 hash size (by default parameters in the whash method).\r\n\r\nFor my Kaggle score the whash brought only +0.04% to AUC metric compered to phash, in addition to my current ~92.9% result. You can try another method parameters or use this result in ensemble stage.\r\n\r\nUsage in Python: \r\n\r\nwhash = pickle.load( open('whash_haar.pkl', 'rb') ).\r\n\r\nlist(tmp.items())[:3]\r\n> [(2, 8034173447716605444),\r\n (5, 11459270170091060735),\r\n (7, 18400019274820157450)]", "126567": "Hey Dmitry, I'm doing a project now for college and based it on your technique and the other hashing, great work ! One thing, on the [pypi website][1] whash links to the Kaggle competition. It should link to your tutorial as that has a great explanation.\r\n\r\n\r\n [1]: https://pypi.python.org/pypi/ImageHash", "126527": "Hi Lin, you are absolutely right. Thank you for catching that!\r\n\r\nIt is a typo. I was unable to find the original test1.jpg and test2.jpg images that I used. So, I recalculated hashes for images 7993417.jpg and 7803947.jpg from the dataset:\r\n\r\nhash1 = 354adab5054af0b7\r\n\r\nhash2 = 5b7724c8bb364551\r\n\r\nhash1-hash2\r\n\r\n'> 44\r\n\r\nThe post was updated and I provided a link to your comment: https://fullstackml.com/2016/07/02/wavelet-image-hash-in-python/", "126571": "Hey Darragh, thank you for paying attention to this...\r\n\r\nI have already changed the link in the project README.rst file. It should be merged to master branch soon. And, yes, I will ask Johannes (imagehash project author) to change the link in pypi website.", "126529": "", "126522": "", "126240": "" } }