![\"issue\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/330685/9459/issue.png\")
I am looking at particle 571958733240799233 in event /train_1/event000001000
\n\nI am plotting tx, ty, with colors indicating the rank in the table. The trajectory is supposed to be approximatively on a circle but we clearly have several outliers.
\n\n
Code to generate it:
\n\nhits, cells, particles, truth = load_event('../input/train_1/event000001000')\npart = 571958733240799233\ndata = truth[truth.particle_id == part]\ndata = data.sort_values(by='tz')\ndata['rankc'] = range(data.shape[0])\nfig, ax=plt.subplots(1,1,figsize=(10,10))\nax.scatter(data.tx , data.ty , c=data.rankc)\nax.set_aspect('equal')\nEdited: sort data by tz.
\n\nThe particle does not start near origin, and it moves past the detector boundary before re-entering. That's probably why the trajectory is so weird.
\n\n![\"weird\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/330685/9478/wrong.png\")
I am looking at particle 571958733240799233 in event /train_1/event000001000
\n\nI am plotting tx, ty, with colors indicating the rank in the table. The trajectory is supposed to be approximatively on a circle but we clearly have several outliers.
\n\n
Code to generate it:
\n\nhits, cells, particles, truth = load_event('../input/train_1/event000001000')\npart = 571958733240799233\ndata = truth[truth.particle_id == part]\ndata = data.sort_values(by='tz')\ndata['rankc'] = range(data.shape[0])\nfig, ax=plt.subplots(1,1,figsize=(10,10))\nax.scatter(data.tx , data.ty , c=data.rankc)\nax.set_aspect('equal')\nEdited: sort data by tz.
\n\nThe particle does not start near origin, and it moves past the detector boundary before re-entering. That's probably why the trajectory is so weird.
\n\n
You have to think in 3d.try also tx, tz, and ty, tz plots
", "rawMarkdown": "You have to think in 3d.try also tx, tz, and ty, tz plots", "votes": null }, { "id": "330722", "postDate": "05/19/2018 14:37:11", "content": "Here is my plot for this particle:\n![\"571958733240799233\"](\"https://storage.googleapis.com/kaggle-forum-message-attachments/330722/9458/f6.png\")
\nRight diagram looks more strange.. Here Rxy = sqrt(X^2 + Y^2; X, Y, Z are true.
...\"The rank in the table\" doesn't matter. True order may be recovered via ordering by absolute value of momentum.
", "rawMarkdown": "Here is my plot for this particle:\n![571958733240799233][1]\nRight diagram looks more strange.. Here Rxy = sqrt(X^2 + Y^2; X, Y, Z are true.\n\n...\"The rank in the table\" doesn't matter. True order may be recovered via ordering by absolute value of momentum.\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/330722/9458/f6.png", "votes": null }, { "id": "330734", "postDate": "05/19/2018 15:28:53", "content": "\n\n\nThe rank in the table\" doesn't matter.
\n
You're right. One can order using tz.
", "rawMarkdown": "> The rank in the table\" doesn't matter.\n\nYou're right. One can order using tz.", "votes": null }, { "id": "330736", "postDate": "05/19/2018 15:29:34", "content": "Laws of physics, and also organizers say that in tx ty space the trajectory is on a circle. Here the deviation is rather significant.
", "rawMarkdown": "Laws of physics, and also organizers say that in tx ty space the trajectory is on a circle. Here the deviation is rather significant.", "votes": null }, { "id": "330750", "postDate": "05/19/2018 16:09:18", "content": "\n\n\nCPMP wrote
\n \n... One can order using tz.
\n
I also thought that we can order the hits by tz, but this is not the case. The tz are not ordered by the time (although they should from physics).
", "rawMarkdown": "> **CPMP wrote**\n> \n>... One can order using tz.\n\nI also thought that we can order the hits by tz, but this is not the case. The tz are not ordered by the time (although they should from physics).", "votes": null }, { "id": "330806", "postDate": "05/19/2018 19:02:41", "content": "\n\n\nThe tz are not ordered by the time
\n
How do you know the time of a hit?
\n\nEdit, I guess you assume momentum is decreasing over time. In this case, we see that tz evolution reverts at some point, which is really strange. The last 7 points are not consistent with the previous ones.
", "rawMarkdown": "> The tz are not ordered by the time \n\nHow do you know the time of a hit?\n\nEdit, I guess you assume momentum is decreasing over time. In this case, we see that tz evolution reverts at some point, which is really strange. The last 7 points are not consistent with the previous ones.", "votes": null }, { "id": "330856", "postDate": "05/19/2018 21:41:25", "content": "Laws of physics, and also the organizers say that if the direction of magnetic field is not perfectly parallel to tz, it's value is not constant and there are changes of momentum because of multiple scattering, then the trajectory in the tx ty space is NOT on a circle (not talking about the detectors accuracy).
", "rawMarkdown": "Laws of physics, and also the organizers say that if the direction of magnetic field is not perfectly parallel to tz, it's value is not constant and there are changes of momentum because of multiple scattering, then the trajectory in the tx ty space is NOT on a circle (not talking about the detectors accuracy).", "votes": null }, { "id": "330883", "postDate": "05/20/2018 00:36:47", "content": "@Victor, IMHO, the problem with this particle is because of its origin and momentum. Talking about X and Y, the origin of majority of particles is in the region 0+/-1mm. Even Z is in most cases much below 0+/-55 mm declared by the organizers. The origin of particle 571958733240799233 is (x,y,z)=(-18,6,397) and at the beginning, it is going towards the center (z=0) instead of outside of the detector. Additionally, momentum pz takes only 3% of the total momentum, so small changes in total momentum (scattering) cause great changes in momentum pz, including the change of the sign (direction of movement in Z axis).
\n\nFinally, maybe the most important thing, the particle made 1.25 rotation while the most of the others ~0.25. It must affect on the shape of the trajectory because of multiple scattering.
", "rawMarkdown": "Victor, IMHO, the problem with this particle is because of its origin and momentum. Talking about X and Y, the origin of majority of particles is in the region 0+/-1mm. Even Z is in most cases much below 0+/-55 mm declared by the organizers. The origin of particle 571958733240799233 is (x,y,z)=(-18,6,397) and at the beginning, it is going towards the center (z=0) instead of outside of the detector. Additionally, momentum pz takes only 3% of the total momentum, so small changes in total momentum (scattering) cause great changes in momentum pz, including the change of the sign (direction of movement in Z axis).\n\nFinally, maybe the most important thing, the particle made 1.25 rotation while the most of the others ~0.25. It must affect on the shape of the trajectory because of multiple scattering.", "votes": null }, { "id": "330960", "postDate": "05/20/2018 04:14:29", "content": "Should we expect that the scattering here is like in this experiment ?
\n\nParticles colliding with sensors may (rarely) be deflected by any angle?
", "rawMarkdown": "Should we expect that the scattering here is like in [this experiment][1] ?\n\nParticles colliding with sensors may (rarely) be deflected by **any** angle?\n\n [1]: https://en.wikipedia.org/wiki/Geiger%E2%80%93Marsden_experiment", "votes": null }, { "id": "331012", "postDate": "05/20/2018 07:06:51", "content": "Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. Moreover, momentum on z axis cannot change sign by physics laws.
", "rawMarkdown": "Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. Moreover, momentum on z axis cannot change sign by physics laws.", "votes": null }, { "id": "331126", "postDate": "05/20/2018 12:17:39", "content": "\n\n\nGrzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center.
\n
@CPMP, Using decreasing momentum as a time sequence I have obtained the same result as at the picture by Victor - the particle goes left then right on z axis.
\n\n\n\n\nMoreover, momentum on z axis cannot change sign by physics laws.
\n
I do not know such law.
", "rawMarkdown": ">Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. \n\n@CPMP, Using decreasing momentum as a time sequence I have obtained the same result as at the picture by Victor - the particle goes left then right on z axis.\n\n> Moreover, momentum on z axis cannot change sign by physics laws.\n\nI do not know such law.", "votes": null }, { "id": "331137", "postDate": "05/20/2018 12:29:21", "content": "The deflection in angle that charged particles undergo through the passage of thin silicon detectors can be described by the following formula: ![\"deflection](\"https://i.imgur.com/v3zJvzT.png\")
\nTaken from source.
> I do not know such law.
\n\nIt is called conservation of momentum.
\n\nGiven the magnetic field is aligned with z axis, it cannot modify the momentum in that direction. The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. I'll wait to see what explanation the organizers provide for this strange trajectory.
", "rawMarkdown": "> I do not know such law.\n\nIt is called [conservation of momentum][1]. \n\nGiven the magnetic field is aligned with z axis, it cannot modify the momentum in that direction. The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. I'll wait to see what explanation the organizers provide for this strange trajectory.\n\n [1]: http://scienceworld.wolfram.com/physics/ConservationofMomentum.html", "votes": null }, { "id": "331157", "postDate": "05/20/2018 13:35:56", "content": "\n\n\nnot talking about the detectors accuracy
\n
Not relevant when using truth values as I did.
", "rawMarkdown": "> not talking about the detectors accuracy\n\nNot relevant when using truth values as I did.", "votes": null }, { "id": "331162", "postDate": "05/20/2018 13:58:25", "content": "\n\n\nIt is called conservation of momentum.
\n
I know that one.
\n\n\n\n\nThe only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction.
\n
@CPMP, The detector is made of several tons of such atoms.
", "rawMarkdown": ">It is called conservation of momentum. \n\nI know that one.\n\n>The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. \n\n@CPMP, The detector is made of several tons of such atoms.", "votes": null }, { "id": "331189", "postDate": "05/20/2018 15:22:53", "content": "\n\n\nI know that one.
\n
I'm sure you do.
", "rawMarkdown": "> I know that one.\n\nI'm sure you do.", "votes": null }, { "id": "332148", "postDate": "05/22/2018 16:15:32", "content": "Can you verify the csv name and the particle id? I cannot reproduce the bug .
", "rawMarkdown": "Can you verify the csv name and the particle id? I cannot reproduce the _bug_ .", "votes": null }, { "id": "332158", "postDate": "05/22/2018 16:27:43", "content": "@agerom, Are you using R? It is strange, but using R and bit64 and data.table packages I can use > relation but no < or ==. I had to convert particle_id into character to separate this particle from the others.
\n\nEDIT:
\n\ndfpXX[,par:=as.character(particle_id)]\ndfp00 <- dfpXX[par=='571958733240799233',] # works\ndfp11 <- dfpXX[particle_id==571958733240799233,] # doesn't work \n@agerom I don't know if it is a bug, but you have the code to reproduce what I shared. I dug further, and found that we could fit the x,y trajectory with two circles pretty well. Seems this particle has a collision or something, and the trajectory before/after the collision are circular. Also, it is moving towards negative z before the 'collision', and then to high positive z after it.
", "rawMarkdown": "agerom I don't know if it is a bug, but you have the code to reproduce what I shared. I dug further, and found that we could fit the x,y trajectory with two circles pretty well. Seems this particle has a collision or something, and the trajectory before/after the collision are circular. Also, it is moving towards negative z before the 'collision', and then to high positive z after it.", "votes": null }, { "id": "332180", "postDate": "05/22/2018 17:38:52", "content": "@Grzegorz Sionkowski I am using R and data.table. I can't find the particle in the specific event. By the way, *particle_id* is integer64.
", "rawMarkdown": "Grzegorz Sionkowski I am using R and data.table. I can't find the particle in the specific event. By the way, *particle_id* is _integer64_.", "votes": null }, { "id": "332649", "postDate": "05/23/2018 14:16:30", "content": "yup you're right concerning the bug in R . Nice catch! By the way I plotted true momentum vs true distance. Seems legit ![\"true](\"https://i.imgur.com/8N8fqOZ.png\")
@agerom, I do not think distance from (0,0,0) may be called here \"true distance\". If you are looking at the changes of momentum, true distance (i.e. distance connected with the changes) is IMHO the distance made by the particle along its trajectory, finally about 6 meters, not 2.
", "rawMarkdown": "agerom, I do not think distance from (0,0,0) may be called here \"true distance\". If you are looking at the changes of momentum, true distance (i.e. distance connected with the changes) is IMHO the distance made by the particle along its trajectory, finally about 6 meters, not 2.", "votes": null }, { "id": "335358", "postDate": "05/29/2018 16:31:07", "content": "Hi, many thanks for looking into that.
\n\nI re-simulated some of those awkward particles, and I can confirm that in very rare cases the navigation in the simulation goes wrong. I am currently quantifying the effect (it's at O(1) in 10000), and prepare a notebook with explanations and potentially a helper function to flag those.
", "rawMarkdown": "Hi, many thanks for looking into that.\n \nI re-simulated some of those awkward particles, and I can confirm that in very rare cases the navigation in the simulation goes wrong. I am currently quantifying the effect (it's at O(1) in 10000), and prepare a notebook with explanations and potentially a helper function to flag those.", "votes": null }, { "id": "335381", "postDate": "05/29/2018 17:01:17", "content": "Thanks for the transparency on this, although I'm sure some people are frustrated with these kind of issues, I'd imagine similar issues occur due to hardware limitations much more often then the data issues we are seeing.
", "rawMarkdown": "Thanks for the transparency on this, although I'm sure some people are frustrated with these kind of issues, I'd imagine similar issues occur due to hardware limitations much more often then the data issues we are seeing.", "votes": null }, { "id": "335433", "postDate": "05/29/2018 18:05:03", "content": "Thanks for confirming this was a pathological case. As long as this is a rare event I guess we'll be fine in this competition.
", "rawMarkdown": "Thanks for confirming this was a pathological case. As long as this is a rare event I guess we'll be fine in this competition.", "votes": null }, { "id": "348191", "postDate": "06/26/2018 08:20:50", "content": "Hi, unless mistaken we have not seen the notebook you promised about these wrong tracks.
", "rawMarkdown": "Hi, unless mistaken we have not seen the notebook you promised about these wrong tracks.", "votes": null }, { "id": "348883", "postDate": "06/27/2018 13:06:45", "content": "Apologies, due to other commitments within my experiment I could not finish this yet.
", "rawMarkdown": "Apologies, due to other commitments within my experiment I could not finish this yet.", "votes": null }, { "id": "348906", "postDate": "06/27/2018 14:00:52", "content": "Thanks for the update. Do you think you can complete it before end of competition, possibly few weeks before?
", "rawMarkdown": "Thanks for the update. Do you think you can complete it before end of competition, possibly few weeks before?", "votes": null }, { "id": "352046", "postDate": "07/03/2018 14:49:59", "content": "I have not implemented a particle gun to isolate these cases and re-simulate with same kinematics, I hope I am not 'hit' too much by random number seeds, will keep posted.
", "rawMarkdown": "I have not implemented a particle gun to isolate these cases and re-simulate with same kinematics, I hope I am not 'hit' too much by random number seeds, will keep posted.", "votes": null }, { "id": "352048", "postDate": "07/03/2018 15:00:37", "content": "I have a detector of weird trajectories, I posted few of them in a recent post. I can run it to spot more trajectories if it helps.
", "rawMarkdown": "I have a detector of weird trajectories, I posted few of them in a [recent post][1]. I can run it to spot more trajectories if it helps.\n\n\n [1]: https://www.kaggle.com/c/trackml-particle-identification/discussion/60182#latest-351434", "votes": null }, { "id": "359157", "postDate": "07/19/2018 14:43:08", "content": "Hi,
\n\napologies for the belated reply - the particle in question here is actually a looper, it has just the right properties to not leave the 'virtual' detector volume and thus loops many times before creating some rather random hits on the discs.
\n\n\n\nI have used the start parameters of this particle, smeared them slightly and wrote the full particle path, see
\n\nThanks. For this one I did notice we can fit its trajectory using two circles. This is consistent with what you say, the second circle corresponding to the loop that is detected at a much higher z level.
", "rawMarkdown": "Thanks. For this one I did notice we can fit its trajectory using two circles. This is consistent with what you say, the second circle corresponding to the loop that is detected at a much higher z level.", "votes": null } ], "comments": [ { "id": 330687, "author_name": "artemiosgeromitsos", "author_url": "", "post_date": "05/19/2018 12:43:22", "content": "You have to think in 3d.try also tx, tz, and ty, tz plots
", "votes": null, "replies": [ { "id": 330736, "author_name": "cpmpml", "author_url": "", "post_date": "05/19/2018 15:29:34", "content": "Laws of physics, and also organizers say that in tx ty space the trajectory is on a circle. Here the deviation is rather significant.
", "votes": null, "replies": [] }, { "id": 330856, "author_name": "sionek", "author_url": "", "post_date": "05/19/2018 21:41:25", "content": "Laws of physics, and also the organizers say that if the direction of magnetic field is not perfectly parallel to tz, it's value is not constant and there are changes of momentum because of multiple scattering, then the trajectory in the tx ty space is NOT on a circle (not talking about the detectors accuracy).
", "votes": null, "replies": [] }, { "id": 330960, "author_name": "nedelko", "author_url": "", "post_date": "05/20/2018 04:14:29", "content": "Should we expect that the scattering here is like in this experiment ?
\n\nParticles colliding with sensors may (rarely) be deflected by any angle?
", "votes": null, "replies": [] }, { "id": 331137, "author_name": "artemiosgeromitsos", "author_url": "", "post_date": "05/20/2018 12:29:21", "content": "The deflection in angle that charged particles undergo through the passage of thin silicon detectors can be described by the following formula: \nTaken from source.
\n\n\nnot talking about the detectors accuracy
\n
Not relevant when using truth values as I did.
", "votes": null, "replies": [] } ] }, { "id": 330722, "author_name": "nedelko", "author_url": "", "post_date": "05/19/2018 14:37:11", "content": "Here is my plot for this particle:\n\nRight diagram looks more strange.. Here Rxy = sqrt(X^2 + Y^2; X, Y, Z are true.
...\"The rank in the table\" doesn't matter. True order may be recovered via ordering by absolute value of momentum.
", "votes": null, "replies": [ { "id": 330734, "author_name": "cpmpml", "author_url": "", "post_date": "05/19/2018 15:28:53", "content": "\n\n\nThe rank in the table\" doesn't matter.
\n
You're right. One can order using tz.
", "votes": null, "replies": [] }, { "id": 330750, "author_name": "nedelko", "author_url": "", "post_date": "05/19/2018 16:09:18", "content": "\n\n\nCPMP wrote
\n \n... One can order using tz.
\n
I also thought that we can order the hits by tz, but this is not the case. The tz are not ordered by the time (although they should from physics).
", "votes": null, "replies": [] }, { "id": 330806, "author_name": "cpmpml", "author_url": "", "post_date": "05/19/2018 19:02:41", "content": "\n\n\nThe tz are not ordered by the time
\n
How do you know the time of a hit?
\n\nEdit, I guess you assume momentum is decreasing over time. In this case, we see that tz evolution reverts at some point, which is really strange. The last 7 points are not consistent with the previous ones.
", "votes": null, "replies": [] }, { "id": 330883, "author_name": "sionek", "author_url": "", "post_date": "05/20/2018 00:36:47", "content": "@Victor, IMHO, the problem with this particle is because of its origin and momentum. Talking about X and Y, the origin of majority of particles is in the region 0+/-1mm. Even Z is in most cases much below 0+/-55 mm declared by the organizers. The origin of particle 571958733240799233 is (x,y,z)=(-18,6,397) and at the beginning, it is going towards the center (z=0) instead of outside of the detector. Additionally, momentum pz takes only 3% of the total momentum, so small changes in total momentum (scattering) cause great changes in momentum pz, including the change of the sign (direction of movement in Z axis).
\n\nFinally, maybe the most important thing, the particle made 1.25 rotation while the most of the others ~0.25. It must affect on the shape of the trajectory because of multiple scattering.
", "votes": null, "replies": [] }, { "id": 331012, "author_name": "cpmpml", "author_url": "", "post_date": "05/20/2018 07:06:51", "content": "Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. Moreover, momentum on z axis cannot change sign by physics laws.
", "votes": null, "replies": [] }, { "id": 331126, "author_name": "sionek", "author_url": "", "post_date": "05/20/2018 12:17:39", "content": "\n\n\nGrzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center.
\n
@CPMP, Using decreasing momentum as a time sequence I have obtained the same result as at the picture by Victor - the particle goes left then right on z axis.
\n\n\n\n\nMoreover, momentum on z axis cannot change sign by physics laws.
\n
I do not know such law.
", "votes": null, "replies": [] }, { "id": 331152, "author_name": "cpmpml", "author_url": "", "post_date": "05/20/2018 13:14:04", "content": "> I do not know such law.
\n\nIt is called conservation of momentum.
\n\nGiven the magnetic field is aligned with z axis, it cannot modify the momentum in that direction. The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. I'll wait to see what explanation the organizers provide for this strange trajectory.
", "votes": null, "replies": [] }, { "id": 331162, "author_name": "sionek", "author_url": "", "post_date": "05/20/2018 13:58:25", "content": "\n\n\nIt is called conservation of momentum.
\n
I know that one.
\n\n\n\n\nThe only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction.
\n
@CPMP, The detector is made of several tons of such atoms.
", "votes": null, "replies": [] }, { "id": 331189, "author_name": "cpmpml", "author_url": "", "post_date": "05/20/2018 15:22:53", "content": "\n\n\nI know that one.
\n
I'm sure you do.
", "votes": null, "replies": [] } ] }, { "id": 332148, "author_name": "artemiosgeromitsos", "author_url": "", "post_date": "05/22/2018 16:15:32", "content": "Can you verify the csv name and the particle id? I cannot reproduce the bug .
", "votes": null, "replies": [ { "id": 332158, "author_name": "sionek", "author_url": "", "post_date": "05/22/2018 16:27:43", "content": "@agerom, Are you using R? It is strange, but using R and bit64 and data.table packages I can use > relation but no < or ==. I had to convert particle_id into character to separate this particle from the others.
\n\nEDIT:
\n\ndfpXX[,par:=as.character(particle_id)]\ndfp00 <- dfpXX[par=='571958733240799233',] # works\ndfp11 <- dfpXX[particle_id==571958733240799233,] # doesn't work \n@agerom I don't know if it is a bug, but you have the code to reproduce what I shared. I dug further, and found that we could fit the x,y trajectory with two circles pretty well. Seems this particle has a collision or something, and the trajectory before/after the collision are circular. Also, it is moving towards negative z before the 'collision', and then to high positive z after it.
", "votes": null, "replies": [] }, { "id": 332180, "author_name": "artemiosgeromitsos", "author_url": "", "post_date": "05/22/2018 17:38:52", "content": "@Grzegorz Sionkowski I am using R and data.table. I can't find the particle in the specific event. By the way, *particle_id* is integer64.
", "votes": null, "replies": [] }, { "id": 332649, "author_name": "artemiosgeromitsos", "author_url": "", "post_date": "05/23/2018 14:16:30", "content": "yup you're right concerning the bug in R . Nice catch! By the way I plotted true momentum vs true distance. Seems legit
@agerom, I do not think distance from (0,0,0) may be called here \"true distance\". If you are looking at the changes of momentum, true distance (i.e. distance connected with the changes) is IMHO the distance made by the particle along its trajectory, finally about 6 meters, not 2.
", "votes": null, "replies": [] } ] }, { "id": 335358, "author_name": "asalzburger", "author_url": "", "post_date": "05/29/2018 16:31:07", "content": "Hi, many thanks for looking into that.
\n\nI re-simulated some of those awkward particles, and I can confirm that in very rare cases the navigation in the simulation goes wrong. I am currently quantifying the effect (it's at O(1) in 10000), and prepare a notebook with explanations and potentially a helper function to flag those.
", "votes": null, "replies": [ { "id": 335381, "author_name": "macfarll", "author_url": "", "post_date": "05/29/2018 17:01:17", "content": "Thanks for the transparency on this, although I'm sure some people are frustrated with these kind of issues, I'd imagine similar issues occur due to hardware limitations much more often then the data issues we are seeing.
", "votes": null, "replies": [] }, { "id": 335433, "author_name": "cpmpml", "author_url": "", "post_date": "05/29/2018 18:05:03", "content": "Thanks for confirming this was a pathological case. As long as this is a rare event I guess we'll be fine in this competition.
", "votes": null, "replies": [] }, { "id": 348191, "author_name": "cpmpml", "author_url": "", "post_date": "06/26/2018 08:20:50", "content": "Hi, unless mistaken we have not seen the notebook you promised about these wrong tracks.
", "votes": null, "replies": [] }, { "id": 348883, "author_name": "asalzburger", "author_url": "", "post_date": "06/27/2018 13:06:45", "content": "Apologies, due to other commitments within my experiment I could not finish this yet.
", "votes": null, "replies": [] }, { "id": 348906, "author_name": "cpmpml", "author_url": "", "post_date": "06/27/2018 14:00:52", "content": "Thanks for the update. Do you think you can complete it before end of competition, possibly few weeks before?
", "votes": null, "replies": [] }, { "id": 352046, "author_name": "asalzburger", "author_url": "", "post_date": "07/03/2018 14:49:59", "content": "I have not implemented a particle gun to isolate these cases and re-simulate with same kinematics, I hope I am not 'hit' too much by random number seeds, will keep posted.
", "votes": null, "replies": [] }, { "id": 352048, "author_name": "cpmpml", "author_url": "", "post_date": "07/03/2018 15:00:37", "content": "I have a detector of weird trajectories, I posted few of them in a recent post. I can run it to spot more trajectories if it helps.
", "votes": null, "replies": [] } ] }, { "id": 359157, "author_name": "asalzburger", "author_url": "", "post_date": "07/19/2018 14:43:08", "content": "Hi,
\n\napologies for the belated reply - the particle in question here is actually a looper, it has just the right properties to not leave the 'virtual' detector volume and thus loops many times before creating some rather random hits on the discs.
\n\n\n\nI have used the start parameters of this particle, smeared them slightly and wrote the full particle path, see
\n\nThanks. For this one I did notice we can fit its trajectory using two circles. This is consistent with what you say, the second circle corresponding to the loop that is detected at a much higher z level.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "330685": "I am looking at particle 571958733240799233 in event /train_1/event000001000\n\nI am plotting tx, ty, with colors indicating the rank in the table. The trajectory is supposed to be approximatively on a circle but we clearly have several outliers.\n\n![issue][1]\n\nCode to generate it:\n\n\n hits, cells, particles, truth = load_event('../input/train_1/event000001000')\n part = 571958733240799233\n data = truth[truth.particle_id == part]\n data = data.sort_values(by='tz')\n data['rankc'] = range(data.shape[0])\n fig, ax=plt.subplots(1,1,figsize=(10,10))\n ax.scatter(data.tx , data.ty , c=data.rankc)\n ax.set_aspect('equal')\n\n\n \nEdited: sort data by tz. \n\nThe particle does not start near origin, and it moves past the detector boundary before re-entering. That's probably why the trajectory is so weird. \n\n![weird][2]\n\n \n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/330685/9459/issue.png\n [2]: https://storage.googleapis.com/kaggle-forum-message-attachments/330685/9478/wrong.png", "330687": "You have to think in 3d.try also tx, tz, and ty, tz plots", "330722": "Here is my plot for this particle:\n![571958733240799233][1]\nRight diagram looks more strange.. Here Rxy = sqrt(X^2 + Y^2; X, Y, Z are true.\n\n...\"The rank in the table\" doesn't matter. True order may be recovered via ordering by absolute value of momentum.\n [1]: https://storage.googleapis.com/kaggle-forum-message-attachments/330722/9458/f6.png", "330734": "> The rank in the table\" doesn't matter.\n\nYou're right. One can order using tz.", "330736": "Laws of physics, and also organizers say that in tx ty space the trajectory is on a circle. Here the deviation is rather significant.", "330750": "> **CPMP wrote**\n> \n>... One can order using tz.\n\nI also thought that we can order the hits by tz, but this is not the case. The tz are not ordered by the time (although they should from physics).", "330806": "> The tz are not ordered by the time \n\nHow do you know the time of a hit?\n\nEdit, I guess you assume momentum is decreasing over time. In this case, we see that tz evolution reverts at some point, which is really strange. The last 7 points are not consistent with the previous ones.", "330856": "Laws of physics, and also the organizers say that if the direction of magnetic field is not perfectly parallel to tz, it's value is not constant and there are changes of momentum because of multiple scattering, then the trajectory in the tx ty space is NOT on a circle (not talking about the detectors accuracy).", "330883": "Victor, IMHO, the problem with this particle is because of its origin and momentum. Talking about X and Y, the origin of majority of particles is in the region 0+/-1mm. Even Z is in most cases much below 0+/-55 mm declared by the organizers. The origin of particle 571958733240799233 is (x,y,z)=(-18,6,397) and at the beginning, it is going towards the center (z=0) instead of outside of the detector. Additionally, momentum pz takes only 3% of the total momentum, so small changes in total momentum (scattering) cause great changes in momentum pz, including the change of the sign (direction of movement in Z axis).\n\nFinally, maybe the most important thing, the particle made 1.25 rotation while the most of the others ~0.25. It must affect on the shape of the trajectory because of multiple scattering.", "330960": "Should we expect that the scattering here is like in [this experiment][1] ?\n\nParticles colliding with sensors may (rarely) be deflected by **any** angle?\n\n [1]: https://en.wikipedia.org/wiki/Geiger%E2%80%93Marsden_experiment", "331012": "Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. Moreover, momentum on z axis cannot change sign by physics laws.", "331126": ">Grzegorz, how do you get time? If you order the hits by decreasing momentum then this particle moves away from the detector center. \n\n@CPMP, Using decreasing momentum as a time sequence I have obtained the same result as at the picture by Victor - the particle goes left then right on z axis.\n\n> Moreover, momentum on z axis cannot change sign by physics laws.\n\nI do not know such law.", "331137": "The deflection in angle that charged particles undergo through the passage of _thin_ silicon detectors can be described by the following formula: ![deflection MCS][1]\nTaken from [source][2].\n\n\n [1]: https://i.imgur.com/v3zJvzT.png\n [2]: https://arxiv.org/pdf/1405.2759.pdf", "331152": "> I do not know such law.\n\nIt is called [conservation of momentum][1]. \n\nGiven the magnetic field is aligned with z axis, it cannot modify the momentum in that direction. The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. I'll wait to see what explanation the organizers provide for this strange trajectory.\n\n [1]: http://scienceworld.wolfram.com/physics/ConservationofMomentum.html", "331157": "> not talking about the detectors accuracy\n\nNot relevant when using truth values as I did.", "331162": ">It is called conservation of momentum. \n\nI know that one.\n\n>The only possible explanation is a fairly large deflection or even a collision of the particle with some atom in the detector that makes it rebound in a very different direction. \n\n@CPMP, The detector is made of several tons of such atoms.", "331189": "> I know that one.\n\nI'm sure you do.", "332148": "Can you verify the csv name and the particle id? I cannot reproduce the _bug_ .", "332158": "agerom, Are you using R? It is strange, but using R and bit64 and data.table packages I can use > relation but no < or ==. I had to convert particle_id into character to separate this particle from the others.\n\nEDIT:\n\n dfpXX[,par:=as.character(particle_id)]\n dfp00 <- dfpXX[par=='571958733240799233',] # works\n dfp11 <- dfpXX[particle_id==571958733240799233,] # doesn't work", "332162": "agerom I don't know if it is a bug, but you have the code to reproduce what I shared. I dug further, and found that we could fit the x,y trajectory with two circles pretty well. Seems this particle has a collision or something, and the trajectory before/after the collision are circular. Also, it is moving towards negative z before the 'collision', and then to high positive z after it.", "332180": "Grzegorz Sionkowski I am using R and data.table. I can't find the particle in the specific event. By the way, *particle_id* is _integer64_.", "332649": "yup you're right concerning the bug in R . Nice catch! By the way I plotted true momentum vs true distance. Seems legit ![true ptot vs distance][1]\n\n\n [1]: https://i.imgur.com/8N8fqOZ.png", "332743": "agerom, I do not think distance from (0,0,0) may be called here \"true distance\". If you are looking at the changes of momentum, true distance (i.e. distance connected with the changes) is IMHO the distance made by the particle along its trajectory, finally about 6 meters, not 2.", "335358": "Hi, many thanks for looking into that.\n \nI re-simulated some of those awkward particles, and I can confirm that in very rare cases the navigation in the simulation goes wrong. I am currently quantifying the effect (it's at O(1) in 10000), and prepare a notebook with explanations and potentially a helper function to flag those.", "335381": "Thanks for the transparency on this, although I'm sure some people are frustrated with these kind of issues, I'd imagine similar issues occur due to hardware limitations much more often then the data issues we are seeing.", "335433": "Thanks for confirming this was a pathological case. As long as this is a rare event I guess we'll be fine in this competition.", "348191": "Hi, unless mistaken we have not seen the notebook you promised about these wrong tracks.", "348883": "Apologies, due to other commitments within my experiment I could not finish this yet.", "348906": "Thanks for the update. Do you think you can complete it before end of competition, possibly few weeks before?", "352046": "I have not implemented a particle gun to isolate these cases and re-simulate with same kinematics, I hope I am not 'hit' too much by random number seeds, will keep posted.", "352048": "I have a detector of weird trajectories, I posted few of them in a [recent post][1]. I can run it to spot more trajectories if it helps.\n\n\n [1]: https://www.kaggle.com/c/trackml-particle-identification/discussion/60182#latest-351434", "359157": "Hi,\n\napologies for the belated reply - the particle in question here is actually a looper, it has just the right properties to not leave the 'virtual' detector volume and thus loops many times before creating some rather random hits on the discs.\n\n[See more in this post][1]\n\nI have used the start parameters of this particle, smeared them slightly and wrote the full particle path, see \n\n![Loopers with similar particle properties as particle 571958733240799233 in event /train_1/event000001000 ][2]\n\n [1]: https://www.kaggle.com/c/trackml-particle-identification/discussion/61455\n [2]: https://cernbox.cern.ch/index.php/s/4t2j1EsQufny7lS/download", "359211": "Thanks. For this one I did notice we can fit its trajectory using two circles. This is consistent with what you say, the second circle corresponding to the loop that is detected at a much higher z level." }, "source": "meta" }