![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/321345/9332/3C6D9271-3D89-4DCB-BEC4-B37988608B3D.png\")
https://indico.cern.ch/event/702054/
\n\nThere are several slides that u can read.
", "messages": [ { "id": "321341", "postDate": "05/01/2018 04:24:05", "content": "https://indico.cern.ch/event/702054/
\n\nThere are several slides that u can read.
", "rawMarkdown": "https://indico.cern.ch/event/702054/\n\nThere are several slides that u can read.", "votes": null }, { "id": "321345", "postDate": "05/01/2018 04:33:29", "content": "Fyi
\n\n
![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/321345/9333/906FC58E-7C84-4F94-9C6F-ABB90A37F788.png\")
http://hepsoftwarefoundation.org/activities/gsoc.html
\n\n\n\nThis project contains an experiment-independent set of track reconstruction tools. The main philosophy is to provide high-level track reconstruction modules that can be used for any tracking detector. The description of the tracking detector's geometry is optimized for efficient navigation and quick extrapolation of tracks. Converters for several common geometry description languages exist. Having a highly performant, yet largely customizable implementation of track reconstruction algorithms was a primary objective for the design of this toolset. Additionally, the applicability to real-life HEP experiments plays major role in the development process. Apart from algorithmic code, this project also provides an event data model for the description of track parameters and measurements.
\n\n![\"enter](\"http://hepsoftwarefoundation.org/images/ACTSlogo.gif\")
https://press.cern/backgrounders/12-steps-idea-discovery
\n\n", "rawMarkdown": "https://press.cern/backgrounders/12-steps-idea-discovery\n\nhttps://phys.libretexts.org/TextMaps/University_Physics_TextMaps/Map%3A_University_Physics_(OpenStax)/Map%3A_University_Physics_III_-_Optics_and_Modern_Physics_(OpenStax)/11%3A_Particle_Physics_and_Cosmology/11.4%3A_Particle_Accelerators_and_Detectors", "votes": null }, { "id": "321431", "postDate": "05/01/2018 09:10:26", "content": "https://software.intel.com/en-us/articles/the-modern-code-developer-challenge
\n\n\n\nTrack Reconstruction with Deep Learning at the CERN CMS Experiment\nBy Antonio Carta, published on August 17, 2017
\n\nhttps://github.com/iml-wg/HEP-ML-Resources
", "rawMarkdown": "https://software.intel.com/en-us/articles/the-modern-code-developer-challenge\n\nhttps://software.intel.com/en-us/blogs/2017/08/17/track-reconstruction-with-deep-learning-at-the-cern-cms-experiment\n\nTrack Reconstruction with Deep Learning at the CERN CMS Experiment\nBy Antonio Carta, published on August 17, 2017\n\nhttps://github.com/iml-wg/HEP-ML-Resources", "votes": null }, { "id": "321441", "postDate": "05/01/2018 09:48:18", "content": "These are slides and video of a seminar I gave at CERN in March (note that some details like planning are outdated). This was a \"a particle physicist talk to particle physicists\" seminar. Please find here a seminar I gave to the CS lab Laboratoire de Recherche Informatique Orsay : https://bit.ly/2w1FWqn (\"a particle physicist (tries) to talk to computer scientists\")
", "rawMarkdown": "These are slides and video of a seminar I gave at CERN in March (note that some details like planning are outdated). This was a \"a particle physicist talk to particle physicists\" seminar. Please find here a seminar I gave to the CS lab Laboratoire de Recherche Informatique Orsay : https://bit.ly/2w1FWqn (\"a particle physicist (tries) to talk to computer scientists\")", "votes": null }, { "id": "321464", "postDate": "05/01/2018 11:02:20", "content": "An introduction to the challenge intended to non-physicists is here.
", "rawMarkdown": "An introduction to the challenge intended to **non-physicists** is [here][1]. \n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321278/9331/trackml-participant-document-particle-v1.0.pdf", "votes": null }, { "id": "321524", "postDate": "05/01/2018 13:30:01", "content": "A beta version of the detailed detector description document can be found here:\nhttps://cernbox.cern.ch/index.php/s/tkF9YFCrYu8auWw
\n\nThis will be a amended in the next days with a bit more information,\nand then appear on the TrackML webpage when finalized.
", "rawMarkdown": "A beta version of the detailed detector description document can be found here:\nhttps://cernbox.cern.ch/index.php/s/tkF9YFCrYu8auWw\n\nThis will be a amended in the next days with a bit more information,\nand then appear on the TrackML webpage when finalized.", "votes": null }, { "id": "321657", "postDate": "05/01/2018 17:50:16", "content": "This picture best summarised waht this challenge is about
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/321657/9335/1A674594-7514-4762-9A64-0D667EF579A0.png\")
Indeed, let me just note that we are not aiming for the fitted solution, the finding is sufficient for perfect score.
", "rawMarkdown": "Indeed, let me just note that we are not aiming for the fitted solution, the finding is sufficient for perfect score.", "votes": null }, { "id": "321986", "postDate": "05/02/2018 08:59:08", "content": "Lstm method
\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/321986/9336/5671D372-050B-40D4-B751-C169A5D7FE56.png\")
Thanks for sharing !
", "rawMarkdown": "Thanks for sharing !", "votes": null }, { "id": "322097", "postDate": "05/02/2018 12:13:36", "content": "So this is quite advanced but, if I'm reading this correctly, is it suggesting that we should first sort the training data by layer. Then within each layer we convert to spherical polar coordinates and then we should feed each particle from the training set (with known track) through this LSTM so that it learns the sequence of positions of a genuine particle track as the particle passes from one layer to the next?
\n\nThis trained LSTM can then be applied to our test set data (where we are given just the hit locations) and our goal is to use our LSTM to somehow recognise realistic paths and cluster hits together that could realistically be part of the one track. By doing this, we should be able to assign a unique particle ID to each individual hit?
\n\nI don't really get the application of the trained LSTM to the test set though - can we even use LSTMs for clustering?
\n\nThanks :)
", "rawMarkdown": "So this is quite advanced but, if I'm reading this correctly, is it suggesting that we should first sort the training data by layer. Then within each layer we convert to spherical polar coordinates and then we should feed each particle from the training set (with known track) through this LSTM so that it learns the sequence of positions of a genuine particle track as the particle passes from one layer to the next?\n\nThis trained LSTM can then be applied to our test set data (where we are given just the hit locations) and our goal is to use our LSTM to somehow recognise realistic paths and cluster hits together that could realistically be part of the one track. By doing this, we should be able to assign a unique particle ID to each individual hit?\n\nI don't really get the application of the trained LSTM to the test set though - can we even use LSTMs for clustering?\n\nThanks :)", "votes": null }, { "id": "322101", "postDate": "05/02/2018 12:18:33", "content": "i am still figuring out the details, so far here are the possible methods:
\n\n1) (end-to-end) clustering method : e.g. hough transform or some deep network that does the same
\n\n2) path seed detection and path extension: for the path extension, we can use kalman filtering, or network min-max flow optimization (used for fusing tracklets). LSTM can be considered as a deep network version of kalman filtering.
\n\nfor the path seed detection, i am still thinking
", "rawMarkdown": "i am still figuring out the details, so far here are the possible methods:\n\n1) (end-to-end) clustering method : e.g. hough transform or some deep network that does the same\n\n2) path seed detection and path extension: for the path extension, we can use kalman filtering, or network min-max flow optimization (used for fusing tracklets). LSTM can be considered as a deep network version of kalman filtering.\n\nfor the path seed detection, i am still thinking", "votes": null }, { "id": "322102", "postDate": "05/02/2018 12:19:14", "content": "The method employed and presented in this talk by Dustin, from the HEP.TrkX project functions quite well in low density environment and accuracy degrades with higher density as in this challenge.\nYou understood correctly the approach, and the aim is at having the LSTM cell learn the dynamic of the particle and predict the position of the next hit, based on a series of hits already associated to a track. It is not used for clustering, per say, but to predict the path of the particle.
", "rawMarkdown": "The method employed and presented in this talk by Dustin, from the HEP.TrkX project functions quite well in low density environment and accuracy degrades with higher density as in this challenge.\nYou understood correctly the approach, and the aim is at having the LSTM cell learn the dynamic of the particle and predict the position of the next hit, based on a series of hits already associated to a track. It is not used for clustering, per say, but to predict the path of the particle.", "votes": null }, { "id": "322482", "postDate": "05/03/2018 03:47:21", "content": "hough transform:
\n\n\n\nPATHFINDER\nA track finding package based on Hough transformation
\n\n\n\nhttps://indico.desy.de/indico/event/4421/session/2/contribution/105/material/slides/0.pdf
\n\n", "rawMarkdown": "hough transform:\n\nhttps://indico.jinr.ru/getFile.py/access?contribId=86&sessionId=7&resId=0&materialId=slides&confId=60\n\n\nPATHFINDER\nA track finding package based on Hough transformation\n\nhttps://agenda.linearcollider.org/event/5504/contributions/24537/attachments/20138/31812/LCTPCCollaborationMeeting.pdf\n\nhttps://indico.desy.de/indico/event/4421/session/2/contribution/105/material/slides/0.pdf\n\nhttp://flc.desy.de/lcnotes/notes/localfsExplorer_read?currentPath=/afs/desy.de/group/flc/lcnotes/LC-TOOL-2014-003.pdf", "votes": null }, { "id": "322774", "postDate": "05/03/2018 16:19:54", "content": "trackNet
\n\nhttps://github.com/davidclark1/TrackNet
\n\n\"Feedforward Neural Networks for Track Reconstruction\"
", "rawMarkdown": "trackNet\n\nhttps://github.com/davidclark1/TrackNet\n\n\"Feedforward Neural Networks for Track Reconstruction\"", "votes": null }, { "id": "322843", "postDate": "05/03/2018 19:02:49", "content": "It seems a great challage to work with.
", "rawMarkdown": "It seems a great challage to work with.", "votes": null }, { "id": "322855", "postDate": "05/03/2018 19:27:54", "content": "I am vision gathering a team, can also be a beginner mail me at visioninc12@gmail.com
", "rawMarkdown": "I am vision gathering a team, can also be a beginner mail me at visioninc12@gmail.com", "votes": null }, { "id": "322938", "postDate": "05/04/2018 01:11:29", "content": "Discussion on graph based cnn methods
\n\n", "rawMarkdown": "Discussion on graph based cnn methods\n\nhttps://indico.cern.ch/event/658267/contributions/2881175/attachments/1621912/2581064/Farrell_heptrkx_ctd2018.pdf", "votes": null }, { "id": "322942", "postDate": "05/04/2018 01:29:56", "content": "Hopfield network
\n\nhttp://folk.uio.no/ares/FYS4550/rudiVCI.pdf
", "rawMarkdown": "Hopfield network\n\nhttp://folk.uio.no/ares/FYS4550/rudiVCI.pdf", "votes": null }, { "id": "323064", "postDate": "05/04/2018 09:14:17", "content": "@David, in your slides you mentioned there's no hit merging in this challenge but only in real life, does that mean we don't need to consider the case where two independent hits from two different particles have the same position? (Is it called hit smearing?)
", "rawMarkdown": "David, in your slides you mentioned there's no hit merging in this challenge but only in real life, does that mean we don't need to consider the case where two independent hits from two different particles have the same position? (Is it called hit smearing?)", "votes": null }, { "id": "323235", "postDate": "05/04/2018 17:04:55", "content": "https://www.youtube.com/watch?v=M_kj5NKd5zA
\n\nTrack Reconstruction Algorithms in High Pile Up Environments - ACAT 2017
", "rawMarkdown": "https://www.youtube.com/watch?v=M_kj5NKd5zA\n\nTrack Reconstruction Algorithms in High Pile Up Environments - ACAT 2017", "votes": null }, { "id": "323327", "postDate": "05/04/2018 20:51:17", "content": "@Heng Thanks, this is a very good video that gave us a deep dive of what algorithms CERN scientists have tried to reconstruct the tracks and what they really care about. I recommend others watch this video too.
", "rawMarkdown": "Heng Thanks, this is a very good video that gave us a deep dive of what algorithms CERN scientists have tried to reconstruct the tracks and what they really care about. I recommend others watch this video too.", "votes": null }, { "id": "323334", "postDate": "05/04/2018 21:05:43", "content": "Excellent talk indeed, Heather Gray is one of us in fact...
", "rawMarkdown": "Excellent talk indeed, Heather Gray is one of us in fact...", "votes": null }, { "id": "323338", "postDate": "05/04/2018 21:09:12", "content": "It could happen that two independent hit from two different particles are in the same position, but they are considered independent. We checked it happened at per-mil level, so I would not worry about it.
", "rawMarkdown": "It could happen that two independent hit from two different particles are in the same position, but they are considered independent. We checked it happened at per-mil level, so I would not worry about it.", "votes": null }, { "id": "323501", "postDate": "05/05/2018 09:58:12", "content": "This is a detail introduction for the challenge, not only non-physicists but everyone should read it.
", "rawMarkdown": "This is a detail introduction for the challenge, not only non-physicists but everyone should read it.", "votes": null }, { "id": "323732", "postDate": "05/06/2018 02:53:43", "content": "https://www.physik.uni-heidelberg.de/c/image/exp/f/highrr/Kisel_HD_12.04.2016.pdf
\n\nhttps://www.hephy.at/fileadmin/_HEPHY/user_upload/EricaBrondolin_CTD2015_final.pdf
\n\nMethods for track finding:
\n\nConformal Mapping,
Hough Transformation,
Track Following based on the Kalman Filter,
Cellular Automaton with the Kalman Filter for fitting.
Triplet seeding code
\n\n\n\n![\"enter](\"https://storage.googleapis.com/kaggle-forum-message-attachments/323748/9366/triplets.png\")
https://devblogs.nvidia.com/a-cuda-dynamic-parallelism-case-study-panda/
", "rawMarkdown": "Triplet seeding code\n\nhttps://algo2.iti.kit.edu/download/f-ptfcms-13_-_Parallel_Triplet_Finding_for_CMS_Track_Reconstruction_(MA).pdf\n\n ![enter image description here][1]\n\nhttps://devblogs.nvidia.com/a-cuda-dynamic-parallelism-case-study-panda/\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/323748/9366/triplets.png", "votes": null }, { "id": "323749", "postDate": "05/06/2018 04:28:02", "content": "Cellular automata code
\n\nhttps://github.com/lefreire/icgpu
\n\nhttps://arxiv.org/pdf/1405.5406.pdf\n(Circle detection on images using Learning Automata )
\n\n\"Learning automata select their current action based on past experiences from the environment. It will fall into the range of reinforcement learning if the environment is stochastic and Markov Decision Process (MDP) is used.\"
", "rawMarkdown": "Cellular automata code\n\nhttps://github.com/lefreire/icgpu\n\n\nhttps://arxiv.org/pdf/1405.5406.pdf\n(Circle detection on images using Learning Automata )\n\n\"Learning automata select their current action based on past experiences from the environment. It will fall into the range of reinforcement learning if the environment is stochastic and Markov Decision Process (MDP) is used.\"", "votes": null }, { "id": "324561", "postDate": "05/07/2018 19:45:36", "content": "Thanks.
", "rawMarkdown": "Thanks.", "votes": null }, { "id": "324576", "postDate": "05/07/2018 20:11:32", "content": "Really useful and excellent video. Thanks!!
", "rawMarkdown": "Really useful and excellent video. Thanks!!", "votes": null }, { "id": "324928", "postDate": "05/08/2018 01:43:58", "content": "CNN track seeding
\n\nhttps://indico.cern.ch/event/567550/papers/2638698/files/6044-ACAT2017_CNN_DiFlorio_78_rev1.pdf
", "rawMarkdown": "CNN track seeding\n\nhttps://indico.cern.ch/event/567550/papers/2638698/files/6044-ACAT2017_CNN_DiFlorio_78_rev1.pdf", "votes": null }, { "id": "330540", "postDate": "05/19/2018 04:41:47", "content": "Triplets fitting
\n\nhttps://www.psi.ch/mu3e/TalksEN/ctd2017_AK.pdf
", "rawMarkdown": "Triplets fitting\n\nhttps://www.psi.ch/mu3e/TalksEN/ctd2017_AK.pdf", "votes": null }, { "id": "335835", "postDate": "05/30/2018 13:09:59", "content": "Wow. I am new to Machine Learning and data science, and this amazing repository of research material that you've posted here is fantastic for helping me understand this competition. I've joined in on my own, because why get your feet wet when you can learn by not drowning? :D
\n\nI appreciate the amount of information you're sharing, @hengck23!!!
", "rawMarkdown": "Wow. I am new to Machine Learning and data science, and this amazing repository of research material that you've posted here is fantastic for helping me understand this competition. I've joined in on my own, because why get your feet wet when you can learn by not drowning? :D\n\nI appreciate the amount of information you're sharing, @hengck23!!!", "votes": null }, { "id": "337542", "postDate": "06/03/2018 06:21:15", "content": "That's quite a lot of background info here and it's time to starts reading... Thanks
", "rawMarkdown": "That's quite a lot of background info here and it's time to starts reading... Thanks", "votes": null }, { "id": "339614", "postDate": "06/07/2018 07:56:39", "content": "http://slideplayer.com/slide/10274204/\nhttp://slideplayer.com/slide/11492604/
\n\n![\"enter](\"http://images.slideplayer.com/42/11492604/slides/slide_24.jpg\")
\n ![\"enter](\"http://images.slideplayer.com/42/11492604/slides/slide_29.jpg\")
Neural Combinatorial Optimization with Reinforcement Learning
\n\nhttps://openreview.net/pdf?id=Bk9mxlSFx
", "rawMarkdown": "Neural Combinatorial Optimization with Reinforcement Learning\n\nhttps://openreview.net/pdf?id=Bk9mxlSFx", "votes": null }, { "id": "344326", "postDate": "06/17/2018 16:35:08", "content": "https://www.groundai.com/project/online-multi-target-tracking-using-recurrent-neural-networks/
\n\nOnline Multi-Target Tracking Using Recurrent Neural Networks
\n\n![\"enter](\"https://www.groundai.com/media/arxiv_projects/58650/x8.png\")
LSTM for path and stroke
\n\nhttps://distill.pub/2016/handwriting/
\n\nhttp://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/
\n\n\"We will model the data as a series of vectors containing the step size in x and y directions to the next point, and an end-of-stroke value that is either 0 or 1, denoting whether the next point is still part of the current stroke, or if we need to lift the pen up and start a new stroke.\"
", "rawMarkdown": "LSTM for path and stroke\n\nhttps://distill.pub/2016/handwriting/\n\nhttp://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/\n\n\"We will model the data as a series of vectors containing the step size in x and y directions to the next point, and an end-of-stroke value that is either 0 or 1, denoting whether the next point is still part of the current stroke, or if we need to lift the pen up and start a new stroke.\"", "votes": null }, { "id": "347684", "postDate": "06/25/2018 03:59:57", "content": "Ransac and differentiable deep ransac can be a possible solution
\n\nhttps://datascience.stackexchange.com/questions/12186/fitting-lines-through-large-point-clouds
", "rawMarkdown": "Ransac and differentiable deep ransac can be a possible solution \n\nhttps://datascience.stackexchange.com/questions/12186/fitting-lines-through-large-point-clouds", "votes": null }, { "id": "350595", "postDate": "06/30/2018 03:11:49", "content": "this may be useful (e.g. for merging and extending track):
\n\nhttps://github.com/maikol-solis/trajectory_distance
\n\ntrajectory_distance contains 9 distances between trajectory.
\n\nhttp://chaozhang.org/files/papers/ijcnn17.pdf
\n\nTrajectory Clustering via Deep Representation\nLearning
", "rawMarkdown": "this may be useful (e.g. for merging and extending track):\n\nhttps://github.com/maikol-solis/trajectory_distance\n\ntrajectory_distance contains 9 distances between trajectory.\n\nhttp://chaozhang.org/files/papers/ijcnn17.pdf\n\nTrajectory Clustering via Deep Representation\nLearning", "votes": null }, { "id": "357864", "postDate": "07/17/2018 04:01:47", "content": "this may be useful:
\n\nhttps://ai.googleblog.com/2018/07/improving-connectomics-by-order-of.html
", "rawMarkdown": "this may be useful:\n\nhttps://ai.googleblog.com/2018/07/improving-connectomics-by-order-of.html", "votes": null } ], "comments": [ { "id": 321345, "author_name": "hengck23", "author_url": "", "post_date": "05/01/2018 04:33:29", "content": "Fyi
\n\n
http://hepsoftwarefoundation.org/activities/gsoc.html
\n\n\n\nThis project contains an experiment-independent set of track reconstruction tools. The main philosophy is to provide high-level track reconstruction modules that can be used for any tracking detector. The description of the tracking detector's geometry is optimized for efficient navigation and quick extrapolation of tracks. Converters for several common geometry description languages exist. Having a highly performant, yet largely customizable implementation of track reconstruction algorithms was a primary objective for the design of this toolset. Additionally, the applicability to real-life HEP experiments plays major role in the development process. Apart from algorithmic code, this project also provides an event data model for the description of track parameters and measurements.
\n\n
https://press.cern/backgrounders/12-steps-idea-discovery
\n\n", "votes": null, "replies": [] }, { "id": 321431, "author_name": "hengck23", "author_url": "", "post_date": "05/01/2018 09:10:26", "content": "https://software.intel.com/en-us/articles/the-modern-code-developer-challenge
\n\n\n\nTrack Reconstruction with Deep Learning at the CERN CMS Experiment\nBy Antonio Carta, published on August 17, 2017
\n\nhttps://github.com/iml-wg/HEP-ML-Resources
", "votes": null, "replies": [] }, { "id": 321441, "author_name": "droussea", "author_url": "", "post_date": "05/01/2018 09:48:18", "content": "These are slides and video of a seminar I gave at CERN in March (note that some details like planning are outdated). This was a \"a particle physicist talk to particle physicists\" seminar. Please find here a seminar I gave to the CS lab Laboratoire de Recherche Informatique Orsay : https://bit.ly/2w1FWqn (\"a particle physicist (tries) to talk to computer scientists\")
", "votes": null, "replies": [ { "id": 323064, "author_name": "nicolefinnie", "author_url": "", "post_date": "05/04/2018 09:14:17", "content": "@David, in your slides you mentioned there's no hit merging in this challenge but only in real life, does that mean we don't need to consider the case where two independent hits from two different particles have the same position? (Is it called hit smearing?)
", "votes": null, "replies": [] }, { "id": 323338, "author_name": "droussea", "author_url": "", "post_date": "05/04/2018 21:09:12", "content": "It could happen that two independent hit from two different particles are in the same position, but they are considered independent. We checked it happened at per-mil level, so I would not worry about it.
", "votes": null, "replies": [] } ] }, { "id": 321464, "author_name": "cecilegermain", "author_url": "", "post_date": "05/01/2018 11:02:20", "content": "An introduction to the challenge intended to non-physicists is here.
", "votes": null, "replies": [ { "id": 323501, "author_name": "corochann", "author_url": "", "post_date": "05/05/2018 09:58:12", "content": "This is a detail introduction for the challenge, not only non-physicists but everyone should read it.
", "votes": null, "replies": [] } ] }, { "id": 321524, "author_name": "asalzburger", "author_url": "", "post_date": "05/01/2018 13:30:01", "content": "A beta version of the detailed detector description document can be found here:\nhttps://cernbox.cern.ch/index.php/s/tkF9YFCrYu8auWw
\n\nThis will be a amended in the next days with a bit more information,\nand then appear on the TrackML webpage when finalized.
", "votes": null, "replies": [] }, { "id": 321657, "author_name": "hengck23", "author_url": "", "post_date": "05/01/2018 17:50:16", "content": "This picture best summarised waht this challenge is about
\n\n
Indeed, let me just note that we are not aiming for the fitted solution, the finding is sufficient for perfect score.
", "votes": null, "replies": [] } ] }, { "id": 321986, "author_name": "hengck23", "author_url": "", "post_date": "05/02/2018 08:59:08", "content": "Lstm method
\n\n
So this is quite advanced but, if I'm reading this correctly, is it suggesting that we should first sort the training data by layer. Then within each layer we convert to spherical polar coordinates and then we should feed each particle from the training set (with known track) through this LSTM so that it learns the sequence of positions of a genuine particle track as the particle passes from one layer to the next?
\n\nThis trained LSTM can then be applied to our test set data (where we are given just the hit locations) and our goal is to use our LSTM to somehow recognise realistic paths and cluster hits together that could realistically be part of the one track. By doing this, we should be able to assign a unique particle ID to each individual hit?
\n\nI don't really get the application of the trained LSTM to the test set though - can we even use LSTMs for clustering?
\n\nThanks :)
", "votes": null, "replies": [] }, { "id": 322101, "author_name": "hengck23", "author_url": "", "post_date": "05/02/2018 12:18:33", "content": "i am still figuring out the details, so far here are the possible methods:
\n\n1) (end-to-end) clustering method : e.g. hough transform or some deep network that does the same
\n\n2) path seed detection and path extension: for the path extension, we can use kalman filtering, or network min-max flow optimization (used for fusing tracklets). LSTM can be considered as a deep network version of kalman filtering.
\n\nfor the path seed detection, i am still thinking
", "votes": null, "replies": [] }, { "id": 322102, "author_name": "vlimant", "author_url": "", "post_date": "05/02/2018 12:19:14", "content": "The method employed and presented in this talk by Dustin, from the HEP.TrkX project functions quite well in low density environment and accuracy degrades with higher density as in this challenge.\nYou understood correctly the approach, and the aim is at having the LSTM cell learn the dynamic of the particle and predict the position of the next hit, based on a series of hits already associated to a track. It is not used for clustering, per say, but to predict the path of the particle.
", "votes": null, "replies": [] } ] }, { "id": 322002, "author_name": "nathanlauga", "author_url": "", "post_date": "05/02/2018 09:36:43", "content": "Thanks for sharing !
", "votes": null, "replies": [] }, { "id": 322482, "author_name": "hengck23", "author_url": "", "post_date": "05/03/2018 03:47:21", "content": "hough transform:
\n\n\n\nPATHFINDER\nA track finding package based on Hough transformation
\n\n\n\nhttps://indico.desy.de/indico/event/4421/session/2/contribution/105/material/slides/0.pdf
\n\n", "votes": null, "replies": [] }, { "id": 322774, "author_name": "hengck23", "author_url": "", "post_date": "05/03/2018 16:19:54", "content": "trackNet
\n\nhttps://github.com/davidclark1/TrackNet
\n\n\"Feedforward Neural Networks for Track Reconstruction\"
", "votes": null, "replies": [] }, { "id": 322843, "author_name": "tanmoyie", "author_url": "", "post_date": "05/03/2018 19:02:49", "content": "It seems a great challage to work with.
", "votes": null, "replies": [] }, { "id": 322855, "author_name": "vision12", "author_url": "", "post_date": "05/03/2018 19:27:54", "content": "I am vision gathering a team, can also be a beginner mail me at visioninc12@gmail.com
", "votes": null, "replies": [] }, { "id": 322938, "author_name": "hengck23", "author_url": "", "post_date": "05/04/2018 01:11:29", "content": "Discussion on graph based cnn methods
\n\n", "votes": null, "replies": [] }, { "id": 322942, "author_name": "hengck23", "author_url": "", "post_date": "05/04/2018 01:29:56", "content": "Hopfield network
\n\nhttp://folk.uio.no/ares/FYS4550/rudiVCI.pdf
", "votes": null, "replies": [] }, { "id": 323235, "author_name": "hengck23", "author_url": "", "post_date": "05/04/2018 17:04:55", "content": "https://www.youtube.com/watch?v=M_kj5NKd5zA
\n\nTrack Reconstruction Algorithms in High Pile Up Environments - ACAT 2017
", "votes": null, "replies": [ { "id": 323327, "author_name": "nicolefinnie", "author_url": "", "post_date": "05/04/2018 20:51:17", "content": "@Heng Thanks, this is a very good video that gave us a deep dive of what algorithms CERN scientists have tried to reconstruct the tracks and what they really care about. I recommend others watch this video too.
", "votes": null, "replies": [] }, { "id": 323334, "author_name": "droussea", "author_url": "", "post_date": "05/04/2018 21:05:43", "content": "Excellent talk indeed, Heather Gray is one of us in fact...
", "votes": null, "replies": [] }, { "id": 324576, "author_name": "mamtanig", "author_url": "", "post_date": "05/07/2018 20:11:32", "content": "Really useful and excellent video. Thanks!!
", "votes": null, "replies": [] } ] }, { "id": 323732, "author_name": "hengck23", "author_url": "", "post_date": "05/06/2018 02:53:43", "content": "https://www.physik.uni-heidelberg.de/c/image/exp/f/highrr/Kisel_HD_12.04.2016.pdf
\n\nhttps://www.hephy.at/fileadmin/_HEPHY/user_upload/EricaBrondolin_CTD2015_final.pdf
\n\nMethods for track finding:
\n\nConformal Mapping,
Hough Transformation,
Track Following based on the Kalman Filter,
Cellular Automaton with the Kalman Filter for fitting.
Triplet seeding code
\n\n\n\n
https://devblogs.nvidia.com/a-cuda-dynamic-parallelism-case-study-panda/
", "votes": null, "replies": [ { "id": 324928, "author_name": "hengck23", "author_url": "", "post_date": "05/08/2018 01:43:58", "content": "CNN track seeding
\n\nhttps://indico.cern.ch/event/567550/papers/2638698/files/6044-ACAT2017_CNN_DiFlorio_78_rev1.pdf
", "votes": null, "replies": [] } ] }, { "id": 323749, "author_name": "hengck23", "author_url": "", "post_date": "05/06/2018 04:28:02", "content": "Cellular automata code
\n\nhttps://github.com/lefreire/icgpu
\n\nhttps://arxiv.org/pdf/1405.5406.pdf\n(Circle detection on images using Learning Automata )
\n\n\"Learning automata select their current action based on past experiences from the environment. It will fall into the range of reinforcement learning if the environment is stochastic and Markov Decision Process (MDP) is used.\"
", "votes": null, "replies": [] }, { "id": 324561, "author_name": "avish0694", "author_url": "", "post_date": "05/07/2018 19:45:36", "content": "Thanks.
", "votes": null, "replies": [] }, { "id": 330540, "author_name": "hengck23", "author_url": "", "post_date": "05/19/2018 04:41:47", "content": "Triplets fitting
\n\nhttps://www.psi.ch/mu3e/TalksEN/ctd2017_AK.pdf
", "votes": null, "replies": [] }, { "id": 335835, "author_name": "ssoder", "author_url": "", "post_date": "05/30/2018 13:09:59", "content": "Wow. I am new to Machine Learning and data science, and this amazing repository of research material that you've posted here is fantastic for helping me understand this competition. I've joined in on my own, because why get your feet wet when you can learn by not drowning? :D
\n\nI appreciate the amount of information you're sharing, @hengck23!!!
", "votes": null, "replies": [] }, { "id": 337542, "author_name": "hyyeoh", "author_url": "", "post_date": "06/03/2018 06:21:15", "content": "That's quite a lot of background info here and it's time to starts reading... Thanks
", "votes": null, "replies": [] }, { "id": 339614, "author_name": "hengck23", "author_url": "", "post_date": "06/07/2018 07:56:39", "content": "http://slideplayer.com/slide/10274204/\nhttp://slideplayer.com/slide/11492604/
\n\n\n
Neural Combinatorial Optimization with Reinforcement Learning
\n\nhttps://openreview.net/pdf?id=Bk9mxlSFx
", "votes": null, "replies": [] }, { "id": 344326, "author_name": "hengck23", "author_url": "", "post_date": "06/17/2018 16:35:08", "content": "https://www.groundai.com/project/online-multi-target-tracking-using-recurrent-neural-networks/
\n\nOnline Multi-Target Tracking Using Recurrent Neural Networks
\n\n
LSTM for path and stroke
\n\nhttps://distill.pub/2016/handwriting/
\n\nhttp://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/
\n\n\"We will model the data as a series of vectors containing the step size in x and y directions to the next point, and an end-of-stroke value that is either 0 or 1, denoting whether the next point is still part of the current stroke, or if we need to lift the pen up and start a new stroke.\"
", "votes": null, "replies": [] }, { "id": 347684, "author_name": "hengck23", "author_url": "", "post_date": "06/25/2018 03:59:57", "content": "Ransac and differentiable deep ransac can be a possible solution
\n\nhttps://datascience.stackexchange.com/questions/12186/fitting-lines-through-large-point-clouds
", "votes": null, "replies": [] }, { "id": 350595, "author_name": "hengck23", "author_url": "", "post_date": "06/30/2018 03:11:49", "content": "this may be useful (e.g. for merging and extending track):
\n\nhttps://github.com/maikol-solis/trajectory_distance
\n\ntrajectory_distance contains 9 distances between trajectory.
\n\nhttp://chaozhang.org/files/papers/ijcnn17.pdf
\n\nTrajectory Clustering via Deep Representation\nLearning
", "votes": null, "replies": [] }, { "id": 357864, "author_name": "hengck23", "author_url": "", "post_date": "07/17/2018 04:01:47", "content": "this may be useful:
\n\nhttps://ai.googleblog.com/2018/07/improving-connectomics-by-order-of.html
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "321341": "https://indico.cern.ch/event/702054/\n\nThere are several slides that u can read.", "321345": "Fyi\n\n ![enter image description here][1]\n\n\n ![enter image description here][2]\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321345/9332/3C6D9271-3D89-4DCB-BEC4-B37988608B3D.png\n [2]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321345/9333/906FC58E-7C84-4F94-9C6F-ABB90A37F788.png", "321422": "http://hepsoftwarefoundation.org/activities/gsoc.html\n\nhttp://acts.web.cern.ch/ACTS/\n\nThis project contains an experiment-independent set of track reconstruction tools. The main philosophy is to provide high-level track reconstruction modules that can be used for any tracking detector. The description of the tracking detector's geometry is optimized for efficient navigation and quick extrapolation of tracks. Converters for several common geometry description languages exist. Having a highly performant, yet largely customizable implementation of track reconstruction algorithms was a primary objective for the design of this toolset. Additionally, the applicability to real-life HEP experiments plays major role in the development process. Apart from algorithmic code, this project also provides an event data model for the description of track parameters and measurements.\n\n\n ![enter image description here][1]\n \n\n [1]: http://hepsoftwarefoundation.org/images/ACTSlogo.gif", "321424": "https://press.cern/backgrounders/12-steps-idea-discovery\n\nhttps://phys.libretexts.org/TextMaps/University_Physics_TextMaps/Map%3A_University_Physics_(OpenStax)/Map%3A_University_Physics_III_-_Optics_and_Modern_Physics_(OpenStax)/11%3A_Particle_Physics_and_Cosmology/11.4%3A_Particle_Accelerators_and_Detectors", "321431": "https://software.intel.com/en-us/articles/the-modern-code-developer-challenge\n\nhttps://software.intel.com/en-us/blogs/2017/08/17/track-reconstruction-with-deep-learning-at-the-cern-cms-experiment\n\nTrack Reconstruction with Deep Learning at the CERN CMS Experiment\nBy Antonio Carta, published on August 17, 2017\n\nhttps://github.com/iml-wg/HEP-ML-Resources", "321441": "These are slides and video of a seminar I gave at CERN in March (note that some details like planning are outdated). This was a \"a particle physicist talk to particle physicists\" seminar. Please find here a seminar I gave to the CS lab Laboratoire de Recherche Informatique Orsay : https://bit.ly/2w1FWqn (\"a particle physicist (tries) to talk to computer scientists\")", "321464": "An introduction to the challenge intended to **non-physicists** is [here][1]. \n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321278/9331/trackml-participant-document-particle-v1.0.pdf", "321524": "A beta version of the detailed detector description document can be found here:\nhttps://cernbox.cern.ch/index.php/s/tkF9YFCrYu8auWw\n\nThis will be a amended in the next days with a bit more information,\nand then appear on the TrackML webpage when finalized.", "321657": "This picture best summarised waht this challenge is about\n\n ![enter image description here][1]\n\nhttps://indico.cern.ch/event/451901/contributions/1948142/attachments/1164660/1678519/MerzlayaA_MethodsOfTrackReconstruction_15.09.2015.pdf\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321657/9335/1A674594-7514-4762-9A64-0D667EF579A0.png", "321668": "Indeed, let me just note that we are not aiming for the fitted solution, the finding is sufficient for perfect score.", "321986": "Lstm method\n\n\n![enter image description here][1]\n\n\nhttps://indico.cern.ch/event/595059/contributions/2498118/attachments/1431635/2199380/03222017heptrkx_IML.pdf\n\n\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/321986/9336/5671D372-050B-40D4-B751-C169A5D7FE56.png", "322002": "Thanks for sharing !", "322097": "So this is quite advanced but, if I'm reading this correctly, is it suggesting that we should first sort the training data by layer. Then within each layer we convert to spherical polar coordinates and then we should feed each particle from the training set (with known track) through this LSTM so that it learns the sequence of positions of a genuine particle track as the particle passes from one layer to the next?\n\nThis trained LSTM can then be applied to our test set data (where we are given just the hit locations) and our goal is to use our LSTM to somehow recognise realistic paths and cluster hits together that could realistically be part of the one track. By doing this, we should be able to assign a unique particle ID to each individual hit?\n\nI don't really get the application of the trained LSTM to the test set though - can we even use LSTMs for clustering?\n\nThanks :)", "322101": "i am still figuring out the details, so far here are the possible methods:\n\n1) (end-to-end) clustering method : e.g. hough transform or some deep network that does the same\n\n2) path seed detection and path extension: for the path extension, we can use kalman filtering, or network min-max flow optimization (used for fusing tracklets). LSTM can be considered as a deep network version of kalman filtering.\n\nfor the path seed detection, i am still thinking", "322102": "The method employed and presented in this talk by Dustin, from the HEP.TrkX project functions quite well in low density environment and accuracy degrades with higher density as in this challenge.\nYou understood correctly the approach, and the aim is at having the LSTM cell learn the dynamic of the particle and predict the position of the next hit, based on a series of hits already associated to a track. It is not used for clustering, per say, but to predict the path of the particle.", "322482": "hough transform:\n\nhttps://indico.jinr.ru/getFile.py/access?contribId=86&sessionId=7&resId=0&materialId=slides&confId=60\n\n\nPATHFINDER\nA track finding package based on Hough transformation\n\nhttps://agenda.linearcollider.org/event/5504/contributions/24537/attachments/20138/31812/LCTPCCollaborationMeeting.pdf\n\nhttps://indico.desy.de/indico/event/4421/session/2/contribution/105/material/slides/0.pdf\n\nhttp://flc.desy.de/lcnotes/notes/localfsExplorer_read?currentPath=/afs/desy.de/group/flc/lcnotes/LC-TOOL-2014-003.pdf", "322774": "trackNet\n\nhttps://github.com/davidclark1/TrackNet\n\n\"Feedforward Neural Networks for Track Reconstruction\"", "322843": "It seems a great challage to work with.", "322855": "I am vision gathering a team, can also be a beginner mail me at visioninc12@gmail.com", "322938": "Discussion on graph based cnn methods\n\nhttps://indico.cern.ch/event/658267/contributions/2881175/attachments/1621912/2581064/Farrell_heptrkx_ctd2018.pdf", "322942": "Hopfield network\n\nhttp://folk.uio.no/ares/FYS4550/rudiVCI.pdf", "323064": "David, in your slides you mentioned there's no hit merging in this challenge but only in real life, does that mean we don't need to consider the case where two independent hits from two different particles have the same position? (Is it called hit smearing?)", "323235": "https://www.youtube.com/watch?v=M_kj5NKd5zA\n\nTrack Reconstruction Algorithms in High Pile Up Environments - ACAT 2017", "323327": "Heng Thanks, this is a very good video that gave us a deep dive of what algorithms CERN scientists have tried to reconstruct the tracks and what they really care about. I recommend others watch this video too.", "323334": "Excellent talk indeed, Heather Gray is one of us in fact...", "323338": "It could happen that two independent hit from two different particles are in the same position, but they are considered independent. We checked it happened at per-mil level, so I would not worry about it.", "323501": "This is a detail introduction for the challenge, not only non-physicists but everyone should read it.", "323732": "https://www.physik.uni-heidelberg.de/c/image/exp/f/highrr/Kisel_HD_12.04.2016.pdf\n\nhttps://www.hephy.at/fileadmin/_HEPHY/user_upload/EricaBrondolin_CTD2015_final.pdf\n\nMethods for track finding:\n\n1. Conformal Mapping,\n\n2. Hough Transformation,\n\n3. Track Following based on the Kalman Filter,\n\n4. Cellular Automaton with the Kalman Filter for fitting.", "323748": "Triplet seeding code\n\nhttps://algo2.iti.kit.edu/download/f-ptfcms-13_-_Parallel_Triplet_Finding_for_CMS_Track_Reconstruction_(MA).pdf\n\n ![enter image description here][1]\n\nhttps://devblogs.nvidia.com/a-cuda-dynamic-parallelism-case-study-panda/\n [1]: https://kaggle2.blob.core.windows.net/forum-message-attachments/323748/9366/triplets.png", "323749": "Cellular automata code\n\nhttps://github.com/lefreire/icgpu\n\n\nhttps://arxiv.org/pdf/1405.5406.pdf\n(Circle detection on images using Learning Automata )\n\n\"Learning automata select their current action based on past experiences from the environment. It will fall into the range of reinforcement learning if the environment is stochastic and Markov Decision Process (MDP) is used.\"", "324561": "Thanks.", "324576": "Really useful and excellent video. Thanks!!", "324928": "CNN track seeding\n\nhttps://indico.cern.ch/event/567550/papers/2638698/files/6044-ACAT2017_CNN_DiFlorio_78_rev1.pdf", "330540": "Triplets fitting\n\nhttps://www.psi.ch/mu3e/TalksEN/ctd2017_AK.pdf", "335835": "Wow. I am new to Machine Learning and data science, and this amazing repository of research material that you've posted here is fantastic for helping me understand this competition. I've joined in on my own, because why get your feet wet when you can learn by not drowning? :D\n\nI appreciate the amount of information you're sharing, @hengck23!!!", "337542": "That's quite a lot of background info here and it's time to starts reading... Thanks", "339614": "http://slideplayer.com/slide/10274204/\nhttp://slideplayer.com/slide/11492604/\n\n ![enter image description here][1]\n ![enter image description here][2]\n\n\n [1]: http://images.slideplayer.com/42/11492604/slides/slide_24.jpg\n [2]: http://images.slideplayer.com/42/11492604/slides/slide_29.jpg", "344324": "Neural Combinatorial Optimization with Reinforcement Learning\n\nhttps://openreview.net/pdf?id=Bk9mxlSFx", "344326": "https://www.groundai.com/project/online-multi-target-tracking-using-recurrent-neural-networks/ \n\nOnline Multi-Target Tracking Using Recurrent Neural Networks\n\n![enter image description here][1]\n\n\n [1]: https://www.groundai.com/media/arxiv_projects/58650/x8.png", "344327": "LSTM for path and stroke\n\nhttps://distill.pub/2016/handwriting/\n\nhttp://blog.otoro.net/2015/12/12/handwriting-generation-demo-in-tensorflow/\n\n\"We will model the data as a series of vectors containing the step size in x and y directions to the next point, and an end-of-stroke value that is either 0 or 1, denoting whether the next point is still part of the current stroke, or if we need to lift the pen up and start a new stroke.\"", "347684": "Ransac and differentiable deep ransac can be a possible solution \n\nhttps://datascience.stackexchange.com/questions/12186/fitting-lines-through-large-point-clouds", "350595": "this may be useful (e.g. for merging and extending track):\n\nhttps://github.com/maikol-solis/trajectory_distance\n\ntrajectory_distance contains 9 distances between trajectory.\n\nhttp://chaozhang.org/files/papers/ijcnn17.pdf\n\nTrajectory Clustering via Deep Representation\nLearning", "357864": "this may be useful:\n\nhttps://ai.googleblog.com/2018/07/improving-connectomics-by-order-of.html" }, "source": "meta" }