{"cells":[{"metadata":{"_uuid":"5ee713e94d176d18b9106a3e9b15f46f655c60db","_cell_guid":"8dc0c4a4-fe7d-4759-80aa-62756b3fd802"},"cell_type":"markdown","source":"**TrackML Particle Tracking**:\n\nTo explore what our universe is made of, scientists at CERN are colliding protons, essentially recreating mini big bangs, and meticulously observing these collisions with intricate silicon detectors. These mini big bangs are created in the world's biggest and most powerful particle accelerator Large Hadron Collider(LHC) built by CERN. The LHC involves collision of Hadron(consisting of quarks which is made up of protons and neutorns). \n\nFrom our Physics class, we know that atom is made up of proton(positively charged), neutrons(They have zero charge) and electrons(negatively charged). Protons alongwith neutrons are found in the nucleus of an atom.  \n\nIt was once thought that neutrons, protons and electrons were fundamental particles. Fundamental particles cannot be broken up into anything smaller. After the invention of the particle accelerator, it was discovered that electrons are fundamental particles, but neutrons and protons are not. Neutrons and protons are made up of quarks, which are held together by Gluons.[source: https://simple.wikipedia.org/wiki/Quark]\n\n\nSo in the LHC, Particles collide at high energies , creating new particles that decay in complex ways as they move through layers of subdetectors. The subdetectors register each particle's passage and microprocessors convert the particles' paths and energies into electrical signals, combining the information to create a digital summary of the \"collision event\". [source: https://home.cern/about/computing/processing-what-record]\n\nThe dataset provided, comprises of multiple independent events, where each event contains simulated measurements (essentially 3D points) of particles generated in a collision between proton bunches. \n\nEach event has four associated files that contain hits, hit cells, particles, and the ground truth association between them.\n"},{"metadata":{"_uuid":"e70637235ae3e9816e2b2861f8d22d7527a65eab","_cell_guid":"431697b3-e9a4-4bca-8582-59c174538344"},"cell_type":"markdown","source":"**Reading the Libraries and Loading the data:**\n\n"},{"metadata":{"_kg_hide-output":true,"_uuid":"7731a18fe7c6c032599831cddf743d91dca21fca","_kg_hide-input":true,"trusted":true,"_cell_guid":"9eeaf5dc-084e-413e-a532-739fcca96ded"},"cell_type":"code","source":"library('readr')\nlibrary('data.table')\nlibrary('tidyr') \nlibrary('dplyr') \nlibrary('ggplot2') \nlibrary('ggthemes')\nlibrary('corrplot') \nlibrary('lubridate') \nlibrary('purrr') \nlibrary('cowplot')\nlibrary('IRdisplay')\nlibrary('viridis')\nlibrary('plotly')\noptions(warn = -1)","execution_count":1,"outputs":[]},{"metadata":{"_kg_hide-output":true,"_uuid":"1c68f98b8d08c093baf4df906a5ef81fa89f02ed","_kg_hide-input":true,"trusted":true,"_cell_guid":"63a6f610-1cce-4c93-b9b8-5d29dfbf08e4"},"cell_type":"code","source":"train_1 <-lapply(Sys.glob(\"../input/train_1/event000001000*.csv\"), fread)\n\n#Note: I have used the code provided in the notebook \n#\"https://www.kaggle.com/pranav84/beginner-s-guide-to-cern-s-particle-tracking-data\" for reading the zip file data. It was very handy. ","execution_count":2,"outputs":[]},{"metadata":{"_uuid":"28a4461babd56883398c54ed2995ccba5810c422","_cell_guid":"a512332e-9866-490d-941e-6d83393efb2c"},"cell_type":"markdown","source":"Each event has four associated files that contain Event hits, Event truth, Event particle and Event hit cells."},{"metadata":{"_kg_hide-output":true,"_uuid":"cae035c374acb16cd6cde19e5a57cb47b6b1d2bd","_kg_hide-input":false,"trusted":true,"_cell_guid":"4bc386ab-7fd8-4a43-8ea0-a026b8efc9f0"},"cell_type":"code","source":"head(train_1)","execution_count":3,"outputs":[]},{"metadata":{"_uuid":"8fdaccf2fd0513366520c451e81f3db4a2064bbe","_cell_guid":"02e2fedb-2b84-4ecd-8080-38eacd76f742"},"cell_type":"markdown","source":"Dealing with one file at a  time for one event.\n\n**Details of Event hits:**\n\nThe hits file for the event000001000,contains 120939 observations with  7 variables.\n\nhit_id: numerical identifier of the hit inside the event(same as number of observations which is 120939 hit_ids)\n\nx, y, z: measured x, y, z position (in millimeter) of the hit in global coordinates.\n\nvolume_id: numerical identifier of the detector group.(9 unique volume_ids)\n\nlayer_id: numerical identifier of the detector layer inside the group.(7 unique layer_ids)\n\nmodule_id: numerical identifier of the detector module inside the layer.\n\n"},{"metadata":{"_kg_hide-input":true,"_uuid":"1c75673dc9d92db10bd345276bad20f5638a2afd","trusted":true,"_cell_guid":"6b5d52aa-653d-4cdf-b371-b00019ce3972"},"cell_type":"code","source":"event_hit <- data.frame(train_1[2])\nstr(event_hit)","execution_count":4,"outputs":[]},{"metadata":{"_uuid":"f93d98444862ec68a31304ddd0ae26846465a221","_cell_guid":"02470832-9a6c-41b6-9cae-e6276c72ede2"},"cell_type":"markdown","source":"Unique values in each column of the file event_hit"},{"metadata":{"_uuid":"84b69bf698e9de1070711001c288a135e6b91421","trusted":true,"_cell_guid":"db37c3f9-802b-4208-adfc-2e0e26132f2c"},"cell_type":"code","source":"event_hit %>% summarise_all(funs(n_distinct))","execution_count":5,"outputs":[]},{"metadata":{"_uuid":"e93b9ce87864b691b4921891a4e02489c0da6155","_cell_guid":"524e6955-3550-4b95-9780-d6d9b3fc3073"},"cell_type":"markdown","source":"**Understanding the layers of detectors:**\n\nLets understand the hits of event000001000, with respect to layer_id and volume_id.\n\nThe experiments at the Large Hadron Collider (LHC) use detectors to analyse the huge number of particles produced by collisions. The detectors gather clues about the particles – including their speed, mass and charge.  Modern particle detectors consist of layers of subdetectors, each designed to look for particular properties, or specific types of particle. "},{"metadata":{"_kg_hide-input":true,"_uuid":"de6844ef194a81c7d730f541c8070bb05a4d10cc","trusted":true,"_cell_guid":"d4e9b65c-d98c-4b01-8f66-5f1a0e56d4dc"},"cell_type":"code","source":"options(repr.plot.width=12, repr.plot.height=12)\np1 <- plot_ly(event_hit, x = event_hit$x, y = event_hit$y, z = event_hit$z, \n             marker = list(color = ~event_hit$layer_id, colorscale = c('#FFE1A1', '#683531'), showscale = TRUE))  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\nhtmlwidgets::saveWidget(p1, \"p1.html\")\ndisplay_html('<iframe src=\"p1.html\" width=100% height=450></iframe>')","execution_count":6,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"06d146152693006ca7709e196c2da0a9dcd93c67","trusted":true,"_cell_guid":"73486c66-1eaa-4e52-a3c2-5da49b70e25b"},"cell_type":"code","source":"p2 <- plot_ly(event_hit, x = event_hit$x, y = event_hit$y, z = event_hit$z, \n             marker = list(color = ~event_hit$volume_id, colorscale = c('#BF382A', '#0C4B8E'), showscale = TRUE))  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\n\nhtmlwidgets::saveWidget(p2, \"p2.html\")\ndisplay_html('<iframe src=\"p2.html\" width=100% height=450></iframe>')","execution_count":7,"outputs":[]},{"metadata":{"_uuid":"4a6f996b4eca2dc8a3b0a9f66a3c26d5a12639cb","_cell_guid":"45bf0b30-a853-4e0b-b8ab-d8f685fa43d7"},"cell_type":"markdown","source":"**Understanding the event_truth file for event0001000:**\n\nThe truth file contains the mapping between hits and generating particles and the true particle state at each measured hit. Each entry maps one hit to one particle.\n\nhit_id: numerical identifier of the hit as defined in the hits file.\n\nparticle_id: numerical identifier of the generating particle as defined in the particles file. A value of 0 means that the hit did not originate from a reconstructible particle, but e.g. from detector noise.\n\ntx, ty, tz true intersection point in global coordinates (in millimeters) between the particle trajectory and the sensitive surface.\n\ntpx, tpy, tpz true particle momentum (in GeV/c) in the global coordinate system at the intersection point. The corresponding vector is tangent to the particle trajectory at the intersection point.\n\nweight per-hit weight used for the scoring metric; total sum of weights within one event equals to one."},{"metadata":{"_uuid":"43f313c120965b51ee842892f1bdb7bb742f5601","trusted":true,"_cell_guid":"e7cf6dc9-b221-4f57-9723-0566844a082b"},"cell_type":"code","source":"event_truth <- data.frame(train_1[4])\nstr(event_truth)\n\n","execution_count":8,"outputs":[]},{"metadata":{"_uuid":"27892fb9875e567c749e5ce3b8296cc1a9297110","_cell_guid":"30e5ecef-fcad-486b-9a0b-ceec5fd5180f"},"cell_type":"markdown","source":"Unique values in each column of file event_truth"},{"metadata":{"_uuid":"75d97d710e923df0e520ba64339b691c90829f9e","trusted":true,"_cell_guid":"26bce470-e838-473a-bafb-5f774f86647c"},"cell_type":"code","source":"event_truth %>% summarise_all(funs(n_distinct))","execution_count":9,"outputs":[]},{"metadata":{"_uuid":"ceecb5d8755a43024eab8984616e3477aa391c14","_cell_guid":"00cb453d-86d9-4927-8ec6-f1dd6b343480"},"cell_type":"markdown","source":"So in the event_truth file, there are a total of 120939 hit_ids and out of which only 10566 unique particles generated. "},{"metadata":{"_kg_hide-input":true,"_uuid":"960d59031177e82f971124ceceb7fbf8aa1273a5","trusted":true,"_cell_guid":"d76f25bb-cf3a-4f5e-9438-b02f168ffefe"},"cell_type":"code","source":"ET <- filter(event_truth, particle_id !=0)\nstr(ET)\n","execution_count":10,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"5cb71b309f31db2417b08c56cf71972a86fea2ea","trusted":true,"_cell_guid":"fa5c5483-6ca4-40fd-8a17-a80139d0fb3a"},"cell_type":"code","source":"p3 <- plot_ly(ET, x = ET$tx, y = ET$ty, z = ET$tz, size = ~ET$weight)  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\n\nhtmlwidgets::saveWidget(p3, \"p3.html\")\ndisplay_html('<iframe src=\"p3.html\" width=100% height=450></iframe>')","execution_count":11,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"fb0ff48fd058ffb6b702a348207ae8e42a96d43c","trusted":true,"_cell_guid":"1c00e311-6515-4dee-94ba-ff7508736b62"},"cell_type":"code","source":"p4 <- plot_ly(ET, x = ET$tpx, y = ET$tpy, z = ET$tpz, size = ~ET$weight)  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\n\nhtmlwidgets::saveWidget(p4, \"p4.html\")\ndisplay_html('<iframe src=\"p4.html\" width=100% height=450></iframe>')","execution_count":12,"outputs":[]},{"metadata":{"_uuid":"33745e4f034c67f63031e741a02137ba4f89bbd5","_cell_guid":"fb3436e2-6043-4b5f-b906-5d69b61a5b5d"},"cell_type":"markdown","source":"**Understanding Event particles:**\n\nThe particles files contains the following values for each particle/entry:\n\nparticle_id: numerical identifier of the particle inside the event.\n\nvx, vy, vz: initial position or vertex (in millimeters) in global coordinates.\n\npx, py, pz: initial momentum (in GeV/c) along each global axis.\n\nq: particle charge (as multiple of the absolute electron charge).\n\nnhits: number of hits generated by this particle.\n\nAll entries contain the generated information or ground truth."},{"metadata":{"_kg_hide-input":true,"_uuid":"cd7be787b54ca4ebaea57baadc548e15d6fdfa39","trusted":true,"_cell_guid":"7111857b-a189-4f3a-9928-cc2de86ae335"},"cell_type":"code","source":"event_particle <- data.frame(train_1[3])\nstr(event_particle)\n\n","execution_count":13,"outputs":[]},{"metadata":{"_uuid":"e4fa2757b15b5e0878130b0c8c86e4e420d08aed","_cell_guid":"28bbdd72-a7a1-481d-9a08-51059584d43d"},"cell_type":"markdown","source":"Unique values in each column of the file event_particle"},{"metadata":{"_uuid":"c01eb5683a32894023ddaf1ef9aa450b40beb8cc","trusted":true,"_cell_guid":"e5dcbcad-8e77-4456-ad45-993772794bbe"},"cell_type":"code","source":"event_particle %>% summarise_all(funs(n_distinct))","execution_count":14,"outputs":[]},{"metadata":{"_uuid":"38c099e0d33de7c1f1f5f4eaafbc29d7d114854e","_cell_guid":"cc8621c9-a709-4c46-9cef-36bc8abf5980"},"cell_type":"markdown","source":"Particles that are located on the outer edge in their intial position generate less number of hits."},{"metadata":{"_kg_hide-input":true,"_uuid":"fabdd033079143711bed57582c3994c9e91c273d","trusted":true,"_cell_guid":"cae4836a-4f03-4e56-b89d-ce123533e1a9"},"cell_type":"code","source":"p5 <- plot_ly(event_particle, x = event_particle$vx, y = event_particle$vy, z = event_particle$vz ,  \n              marker = list(color = ~event_particle$nhits, colorscale = c('#BF382A', '#0C4B8E'), showscale = TRUE), \n               text = ~paste(\"Particle_id:\", event_particle$particle_id, \"<br>\",\n                           \"hits:\", event_particle$nhits, \"<br>\"))  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\n\nhtmlwidgets::saveWidget(p5, \"p5.html\")\ndisplay_html('<iframe src=\"p5.html\" width=100% height=450></iframe>')","execution_count":15,"outputs":[]},{"metadata":{"_uuid":"df62a5fc3754a8644defa735485b25f2d671ba3a","_cell_guid":"41ce4b5c-a4ba-46f8-8189-00f90dac2c8d"},"cell_type":"markdown","source":"Initial momentum of the particles"},{"metadata":{"_kg_hide-input":true,"_uuid":"a8cd68871be93734bbeac5e7bddc77eaf7d7e7fc","trusted":true,"_cell_guid":"4f465122-b192-403f-826b-3fa0ab71037a"},"cell_type":"code","source":"p6 <- plot_ly(event_particle, x = event_particle$px, y = event_particle$py, z = event_particle$pz ,  \n              marker = list(color = ~event_particle$nhits, colorscale = c('#BF382A', '#0C4B8E'), showscale = TRUE), \n               text = ~paste(\"Particle_id:\", event_particle$particle_id, \"<br>\",\n                           \"hits:\", event_particle$nhits, \"<br>\"))  %>%\n  add_markers() %>%\n  layout(scene = list(xaxis = list(title = 'x-axis'),\n                     yaxis = list(title = 'y-axis'),\n                     zaxis = list(title = 'z-axis')))\n\nhtmlwidgets::saveWidget(p6, \"p6.html\")\ndisplay_html('<iframe src=\"p6.html\" width=100% height=450></iframe>')","execution_count":16,"outputs":[]},{"metadata":{"_uuid":"66c7852f3bc69c9396b16d2cbe982290cfc8113a","_cell_guid":"6616efb2-d170-4153-8646-192965f6f170"},"cell_type":"markdown","source":"A cell provides signal information that the detector module has recorded in addition to the position.  It is identified by two channel identifiers that are unique within each detector module and encode the position.\n\nhit_id: numerical identifier of the hit as defined in the hits file.\n\nch0, ch1: channel identifier/coordinates unique within one module.\n\nvalue: signal value information, e.g. how much charge a particle has deposited.\n"},{"metadata":{"_kg_hide-input":false,"_uuid":"813d799c36bc28012823dc479e712dc2aef6bffc","trusted":true,"_cell_guid":"5306cbd6-ba81-4ae0-942b-72895b728a6a"},"cell_type":"code","source":"event_cells <- data.frame(train_1[1])\nstr(event_cells)\n","execution_count":17,"outputs":[]},{"metadata":{"_uuid":"eb0fc9d38c90ff5accc72c74e8dfa4aa65dfe9f1","_cell_guid":"0370eb7f-6fe0-432e-8a20-5d54b55badc6"},"cell_type":"markdown","source":"Unique values in each column of the file event_cells"},{"metadata":{"_uuid":"0cc01f7d66cd1a33f2d099e6ffc9218484c97b0a","trusted":true,"_cell_guid":"f913e068-73c9-4385-9086-6a6c4e9ecd07"},"cell_type":"code","source":"event_cells %>% summarise_all(funs(n_distinct))","execution_count":18,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"R","language":"R","name":"ir"},"language_info":{"mimetype":"text/x-r-source","name":"R","pygments_lexer":"r","version":"3.4.2","file_extension":".r","codemirror_mode":"r"}},"nbformat":4,"nbformat_minor":1}