![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/332286/9498/triple_loss.png\")
the idea is to learn transformation functions for func(hit), func(pairs) or func(triplets) etc .... where distance of func(x1,) , func(x2) is small if and only if x1,x2 belong to the same track.
\n\nWe can then apply DBSCAN or other clustering algorithms on func(x), instead of the original x
\n\nBelow shows an example results using triple los. Single hit is used as input.
\n\n
the idea is to learn transformation functions for func(hit), func(pairs) or func(triplets) etc .... where distance of func(x1,) , func(x2) is small if and only if x1,x2 belong to the same track.
\n\nWe can then apply DBSCAN or other clustering algorithms on func(x), instead of the original x
\n\nBelow shows an example results using triple los. Single hit is used as input.
\n\n
Could you explain your idea more?\nI understand for pairs you can call function with vectors and then group similar vectors into clusters.\nBut how it is going to work with \"func(hit)\"? \n\"example results using triple los. Single hit is used as input.\" - does it mean function is \"func(triplets)\" and you color each hit separately? Please explain if possible :)
", "rawMarkdown": "Could you explain your idea more?\nI understand for pairs you can call function with vectors and then group similar vectors into clusters.\nBut how it is going to work with \"func(hit)\"? \n\"example results using triple los. Single hit is used as input.\" - does it mean function is \"func(triplets)\" and you color each hit separately? Please explain if possible :)", "votes": null }, { "id": "332640", "postDate": "05/23/2018 13:50:57", "content": "current dbscan transform (x,y,z) to (x/d,y/d,z/r) where d=(x*2+y*2+z*2)*0.5 and r=(x*2+y*2)**0.5.
\n\nassume that there is a better transform function and we want to find it by metric learning. let the transform function be func (x,y,z) = func (hit).
\n\nto construct triple loss, say:
\n\nselect a hit where its particle_id !=0. call this x.
find another hit with the same particle_id as x above, call this x+.
find another hit with the different particle_id as x above, call this x-.
then you can formulate triple loss with (x,x+,x-)
\n\nNow this results may not be good enough. assume given a hit x, we use kdtree to find some neighbors neigbour, n0 n1 ...nk. Instead of 3 dim (x,y,z) we add another feature called the \"forward motion\", vi = ni-x. now a hit point is described by (x,y,z,v0), (x,y,z,v1), ...(x,y,z,vk). We treat each of these as separate data point. we now want to cluster this. note that v0 has 3 dim and func (x,y,z,v) can be respresented by func (hit,neighbour), aka func (hit pair)
", "rawMarkdown": "current dbscan transform (x,y,z) to (x/d,y/d,z/r) where d=(x**2+y**2+z**2)**0.5 and r=(x**2+y**2)**0.5.\n\nassume that there is a better transform function and we want to find it by metric learning. let the transform function be func (x,y,z) = func (hit).\n\n\nto construct triple loss, say:\n\n 1. select a hit where its particle_id !=0. call this x.\n \n 2. find another hit with the same particle_id as x above, call this x+.\n\n 3. find another hit with the different particle_id as x above, call this x-.\n\nthen you can formulate triple loss with (x,x+,x-)\n\n---\n\nNow this results may not be good enough. assume given a hit x, we use kdtree to find some neighbors neigbour, n0 n1 ...nk. Instead of 3 dim (x,y,z) we add another feature called the \"forward motion\", vi = ni-x. now a hit point is described by (x,y,z,v0), (x,y,z,v1), ...(x,y,z,vk). We treat each of these as separate data point. we now want to cluster this. note that v0 has 3 dim and func (x,y,z,v) can be respresented by func (hit,neighbour), aka func (hit pair)", "votes": null }, { "id": "332842", "postDate": "05/23/2018 21:51:22", "content": "Any success with this so far? I have been trying a similar method to find a transform, but have mostly succeeded in running out of RAM and freezing my machine ;)
", "rawMarkdown": "Any success with this so far? I have been trying a similar method to find a transform, but have mostly succeeded in running out of RAM and freezing my machine ;)", "votes": null }, { "id": "333895", "postDate": "05/26/2018 01:54:33", "content": "Would learning the transformation be similar to how a CNN learns kernel/filter matrices?
", "rawMarkdown": "Would learning the transformation be similar to how a CNN learns kernel/filter matrices?", "votes": null }, { "id": "333972", "postDate": "05/26/2018 07:01:09", "content": "They're of the same spirit, but that's about it. Direct kernel learning is transductive (cannot be used on new 'unobserved' objects). Also, DKL guarantees constraint satisfaction. However, learning a generating function via metric learning is inductive and does not guarantee constraint satisfaction.
", "rawMarkdown": "They're of the same spirit, but that's about it. Direct kernel learning is transductive (cannot be used on new 'unobserved' objects). Also, DKL guarantees constraint satisfaction. However, learning a generating function via metric learning is inductive and does not guarantee constraint satisfaction.", "votes": null }, { "id": "334034", "postDate": "05/26/2018 11:03:11", "content": "Some progress, but too early to call it success.
\n\nClassification rates for pairs of hits with simple Random Forest looks initially promising, at 75-90%. I also found a Random Forest Distance paper with an extended approach that gets excellent results, lending some credibility to this approach.
\n\nI needed a solution to avoid running out of memory, so I wrote a quick-and-dirty function for sparse pairwise distances in that can be used with sklearn DBSCAN with metric='precomputed'. See pairwise_distance_sparse_fast in my notebook.
However runtimes for calculating distances are too long to be productive, estimated 1 hour per event with my metric. Some approaches for fixing this:
\n\nWould this still be the case when the labels are taken into account and the problem is reframed as a semi-supervised learning problem where first you use the labels to somehow train a custom metric which could then be used with DBSCAN on some un labeled data or the test set?
", "rawMarkdown": "Would this still be the case when the labels are taken into account and the problem is reframed as a semi-supervised learning problem where first you use the labels to somehow train a custom metric which could then be used with DBSCAN on some un labeled data or the test set?", "votes": null }, { "id": "335164", "postDate": "05/29/2018 10:20:52", "content": "... sklearn DBSCAN with metric='precomputed' ...
\n\nThere are two ways to use DBSCAN+machine learning
\n\ninput data ---> DBSCAN with learned distance function
input data ---> transformed data ---> DBSCAN with usual L2 distance function
I think method 2 is easier to implement. All \"non linear distance function\" can be re-implemented as \"non linear feature + linear distance\".
\n\nReferring to https://www.kaggle.com/c/trackml-particle-identification/discussion/57643, you many want to split all hits into \"smaller cones\" and perform \"modified DBSCAN\" separately on it.
", "rawMarkdown": "... sklearn DBSCAN with metric='precomputed' ...\n\nThere are two ways to use DBSCAN+machine learning\n\n1. input data ---> DBSCAN with learned distance function\n\n2. input data ---> transformed data ---> DBSCAN with usual L2 distance function\n\nI think method 2 is easier to implement. All \"non linear distance function\" can be re-implemented as \"non linear feature + linear distance\". \n\n\nReferring to https://www.kaggle.com/c/trackml-particle-identification/discussion/57643, you many want to split all hits into \"smaller cones\" and perform \"modified DBSCAN\" separately on it.", "votes": null }, { "id": "335320", "postDate": "05/29/2018 15:22:15", "content": "You mention selecting a hit where particle_id!=0; is it a bad idea to just filter all of those hits out as noise from the get-go? Removes detector noise, and reduces your working dataset by several thousands right out of the gate if you do that.
", "rawMarkdown": "You mention selecting a hit where particle_id!=0; is it a bad idea to just filter all of those hits out as noise from the get-go? Removes detector noise, and reduces your working dataset by several thousands right out of the gate if you do that.", "votes": null }, { "id": "335376", "postDate": "05/29/2018 16:54:44", "content": "For exploratory analysis, you could... But the test set doesn't have particle IDs, so you won't be able to do this for your submission.
\n\nIf you train a model with the cleaned data, the model will probably perform worse on the test set where you won't be able to remove the noise hits.
", "rawMarkdown": "For exploratory analysis, you could... But the test set doesn't have particle IDs, so you won't be able to do this for your submission. \n\nIf you train a model with the cleaned data, the model will probably perform worse on the test set where you won't be able to remove the noise hits.", "votes": null }, { "id": "376746", "postDate": "08/28/2018 02:34:28", "content": "Hi, I am trying metric-learning based clustering on high-dimensional data. Could u pls tell me what metric learning method u use? MMC, ITML, or lmnn? In my practice, it seemed only MMC is suited for semi-supervised clustering. However, it only work when dimension of data is less than 10.
\nAny idea about the metric-learning algorithm? or any suggestion about the clustering algorithm for high-dimension data(several hundred)?\nThanks!
Could you explain your idea more?\nI understand for pairs you can call function with vectors and then group similar vectors into clusters.\nBut how it is going to work with \"func(hit)\"? \n\"example results using triple los. Single hit is used as input.\" - does it mean function is \"func(triplets)\" and you color each hit separately? Please explain if possible :)
", "votes": null, "replies": [ { "id": 332640, "author_name": "hengck23", "author_url": "", "post_date": "05/23/2018 13:50:57", "content": "current dbscan transform (x,y,z) to (x/d,y/d,z/r) where d=(x*2+y*2+z*2)*0.5 and r=(x*2+y*2)**0.5.
\n\nassume that there is a better transform function and we want to find it by metric learning. let the transform function be func (x,y,z) = func (hit).
\n\nto construct triple loss, say:
\n\nselect a hit where its particle_id !=0. call this x.
find another hit with the same particle_id as x above, call this x+.
find another hit with the different particle_id as x above, call this x-.
then you can formulate triple loss with (x,x+,x-)
\n\nNow this results may not be good enough. assume given a hit x, we use kdtree to find some neighbors neigbour, n0 n1 ...nk. Instead of 3 dim (x,y,z) we add another feature called the \"forward motion\", vi = ni-x. now a hit point is described by (x,y,z,v0), (x,y,z,v1), ...(x,y,z,vk). We treat each of these as separate data point. we now want to cluster this. note that v0 has 3 dim and func (x,y,z,v) can be respresented by func (hit,neighbour), aka func (hit pair)
", "votes": null, "replies": [] }, { "id": 335320, "author_name": "ssoder", "author_url": "", "post_date": "05/29/2018 15:22:15", "content": "You mention selecting a hit where particle_id!=0; is it a bad idea to just filter all of those hits out as noise from the get-go? Removes detector noise, and reduces your working dataset by several thousands right out of the gate if you do that.
", "votes": null, "replies": [] }, { "id": 335376, "author_name": "macfarll", "author_url": "", "post_date": "05/29/2018 16:54:44", "content": "For exploratory analysis, you could... But the test set doesn't have particle IDs, so you won't be able to do this for your submission.
\n\nIf you train a model with the cleaned data, the model will probably perform worse on the test set where you won't be able to remove the noise hits.
", "votes": null, "replies": [] } ] }, { "id": 332842, "author_name": "cosmoton", "author_url": "", "post_date": "05/23/2018 21:51:22", "content": "Any success with this so far? I have been trying a similar method to find a transform, but have mostly succeeded in running out of RAM and freezing my machine ;)
", "votes": null, "replies": [ { "id": 334034, "author_name": "jonnor", "author_url": "", "post_date": "05/26/2018 11:03:11", "content": "Some progress, but too early to call it success.
\n\nClassification rates for pairs of hits with simple Random Forest looks initially promising, at 75-90%. I also found a Random Forest Distance paper with an extended approach that gets excellent results, lending some credibility to this approach.
\n\nI needed a solution to avoid running out of memory, so I wrote a quick-and-dirty function for sparse pairwise distances in that can be used with sklearn DBSCAN with metric='precomputed'. See pairwise_distance_sparse_fast in my notebook.
However runtimes for calculating distances are too long to be productive, estimated 1 hour per event with my metric. Some approaches for fixing this:
\n\n... sklearn DBSCAN with metric='precomputed' ...
\n\nThere are two ways to use DBSCAN+machine learning
\n\ninput data ---> DBSCAN with learned distance function
input data ---> transformed data ---> DBSCAN with usual L2 distance function
I think method 2 is easier to implement. All \"non linear distance function\" can be re-implemented as \"non linear feature + linear distance\".
\n\nReferring to https://www.kaggle.com/c/trackml-particle-identification/discussion/57643, you many want to split all hits into \"smaller cones\" and perform \"modified DBSCAN\" separately on it.
", "votes": null, "replies": [] } ] }, { "id": 333895, "author_name": "jackvial", "author_url": "", "post_date": "05/26/2018 01:54:33", "content": "Would learning the transformation be similar to how a CNN learns kernel/filter matrices?
", "votes": null, "replies": [ { "id": 333972, "author_name": "puremath86", "author_url": "", "post_date": "05/26/2018 07:01:09", "content": "They're of the same spirit, but that's about it. Direct kernel learning is transductive (cannot be used on new 'unobserved' objects). Also, DKL guarantees constraint satisfaction. However, learning a generating function via metric learning is inductive and does not guarantee constraint satisfaction.
", "votes": null, "replies": [] }, { "id": 334246, "author_name": "jackvial", "author_url": "", "post_date": "05/26/2018 19:00:55", "content": "Would this still be the case when the labels are taken into account and the problem is reframed as a semi-supervised learning problem where first you use the labels to somehow train a custom metric which could then be used with DBSCAN on some un labeled data or the test set?
", "votes": null, "replies": [] } ] }, { "id": 376746, "author_name": "davidwang1113", "author_url": "", "post_date": "08/28/2018 02:34:28", "content": "Hi, I am trying metric-learning based clustering on high-dimensional data. Could u pls tell me what metric learning method u use? MMC, ITML, or lmnn? In my practice, it seemed only MMC is suited for semi-supervised clustering. However, it only work when dimension of data is less than 10.
\nAny idea about the metric-learning algorithm? or any suggestion about the clustering algorithm for high-dimension data(several hundred)?\nThanks!