{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":38257,"databundleVersionId":4319132,"sourceType":"competition"}],"dockerImageVersionId":30497,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#

Libraries

","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport gc\nimport os\nimport re\nimport networkx as nx\n\n\nimport plotly.express as px\nfrom plotly import graph_objects as go\nfrom sklearn.decomposition import PCA","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:26:17.219533Z","iopub.execute_input":"2024-06-23T23:26:17.220428Z","iopub.status.idle":"2024-06-23T23:26:20.632643Z","shell.execute_reply.started":"2024-06-23T23:26:17.220388Z","shell.execute_reply":"2024-06-23T23:26:20.630950Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

💡 Intro:

\n\n \nNeutrinos are electrically neutral and extremely lightweight subatomic particles that interact very weakly with matter. They are one of the fundamental particles of nature and are classified into three different types: electron neutrino, muon neutrino, and tau neutrino. Neutrinos are produced in various sources such as the Sun, supernovae, and nuclear reactors, and they travel at speeds close to the speed of light through space and matter.\n \n \n \n \n\nIceCube is a neutrino observatory located at the South Pole. It consists of a set of detectors buried in the Antarctic ice. Neutrino detection is carried out through a phenomenon called Cherenkov radiation. When a neutrino interacts with an atomic nucleus in the ice, it generates a charged particle that moves faster than the speed of light in the ice. This results in the emission of electromagnetic radiation known as Cherenkov radiation, which can be detected by the IceCube sensors.\n \n\n \n# Why is important the Neutrinos detection?\n\nExploration of the Universe: Neutrinos are particles that travel virtually without interactions through space and matter. This makes them cosmic messengers that can carry valuable information about extreme astrophysical events such as supernova explosions, black holes, neutron stars, and high-energy regions of the universe. Neutrino detection allows us to investigate and better understand these phenomena, expanding our knowledge of the cosmos.\n \nParticle Physics and Understanding the Standard Model: Neutrinos are fundamental to our understanding of particle physics and the standard model. By studying their properties, such as mass, oscillations, and interactions, we can test and improve our fundamental theories of how elementary particles work and how the universe behaves at deeper levels.\n\nHigh-Energy Astrophysics: The detection of high-energy neutrinos from astrophysical sources allows us to explore and understand the most extreme processes in the universe. High-energy neutrinos can be produced in violent cosmic events such as supermassive black hole jets, neutron star collisions, or particle interactions in natural accelerators. These events can shed light on high-energy physics and open new perspectives in our understanding of the universe.\n\n\nSearch for Exotic Particles: In addition to their intrinsic importance, the detection of neutrinos can also provide indirect information about exotic particles such as supersymmetric particles, dark matter, or violations of charge-parity symmetry. By observing the properties and oscillation patterns of neutrinos, we can gather hints and constraints on the existence of particles and phenomena beyond the standard model of particle physics.\n \nSearch for Exotic Particles: In addition to their intrinsic importance, the detection of neutrinos can also provide indirect information about exotic particles such as supersymmetric particles, dark matter, or violations of charge-parity symmetry. By observing the properties and oscillation patterns of neutrinos, we can gather hints and constraints on the existence of particles and phenomena beyond the standard model of particle physics. \n \nIn summary, the detection of neutrinos is important because it allows us to explore the universe, investigate extreme astrophysical events, better understand particle physics, and search for exotic particles and phenomena. Studying neutrinos provides valuable information to expand our knowledge of the fundamental nature of the universe and the processes that occur within it.\n \n \n \n \n \n \n# Info reference:\n \n \nTheoretical Prediction of Presupernova Neutrinos and Their Detection <---\n\nNovel Quantum Sensors for Light Dark Matter and Neutrino Detection <---\n\nObservation of coherent elastic neutrino-nucleus scattering <---\n\n ","metadata":{}},{"cell_type":"markdown","source":"#

Get Paths

","metadata":{}},{"cell_type":"code","source":"train_batchs_names = os.listdir(\"/kaggle/input/icecube-neutrinos-in-deep-ice/train\")[: 10]\nprint(f\"Sample de Batches Seleccionados: \")\nprint(train_batchs_names)","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:26:20.636420Z","iopub.execute_input":"2024-06-23T23:26:20.638217Z","iopub.status.idle":"2024-06-23T23:26:20.928565Z","shell.execute_reply.started":"2024-06-23T23:26:20.638165Z","shell.execute_reply":"2024-06-23T23:26:20.927184Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_batch_names = os.listdir(\"/kaggle/input/icecube-neutrinos-in-deep-ice/test\")\nprint(test_batch_names)","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:26:20.930299Z","iopub.execute_input":"2024-06-23T23:26:20.930619Z","iopub.status.idle":"2024-06-23T23:26:20.943819Z","shell.execute_reply.started":"2024-06-23T23:26:20.930592Z","shell.execute_reply":"2024-06-23T23:26:20.942587Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def GetBatchId(nombre_archivo):\n numero = re.findall(r'\\d+', nombre_archivo)\n return int(numero[0]) if numero else None\n\ndef Batch2Path(batch_id_name, is_train = True):\n path = \"/kaggle/input/icecube-neutrinos-in-deep-ice/\"\n \n if is_train:\n \n path += \"train/\" + batch_id_name\n else:\n \n path += \"test/\" + batch_id_name\n \n return path","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:26:20.947462Z","iopub.execute_input":"2024-06-23T23:26:20.947969Z","iopub.status.idle":"2024-06-23T23:26:20.956396Z","shell.execute_reply.started":"2024-06-23T23:26:20.947922Z","shell.execute_reply":"2024-06-23T23:26:20.955077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

Read Events

","metadata":{}},{"cell_type":"code","source":"meta_train = pd.read_parquet(\"/kaggle/input/icecube-neutrinos-in-deep-ice/train_meta.parquet\")\nmeta_test = pd.read_parquet(\"/kaggle/input/icecube-neutrinos-in-deep-ice/test_meta.parquet\")\n\nprint(f\"Cantidad de eventos de entrenamiento: {len(meta_train)}\")\nprint(f\"Cantidad de eventos a Predecir: {len(meta_test)}\")","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:26:20.958277Z","iopub.execute_input":"2024-06-23T23:26:20.959276Z","iopub.status.idle":"2024-06-23T23:27:07.130462Z","shell.execute_reply.started":"2024-06-23T23:26:20.959223Z","shell.execute_reply":"2024-06-23T23:27:07.129301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sensor_geometry = pd.read_csv(\"/kaggle/input/icecube-neutrinos-in-deep-ice/sensor_geometry.csv\")\n\nx = sensor_geometry.x\ny = sensor_geometry.y\nz = sensor_geometry.z\n\nd = np.sqrt((x.max() - x.min())**2 + (y.max() - y.min())**2 + (z.max() - z.min())**2)\n\nc = (299792.458 * 1000)/10**9\n\ntime_valid = d/c\n\nprint(f\"time valid: {time_valid}m/ns\")","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:27:07.132253Z","iopub.execute_input":"2024-06-23T23:27:07.132603Z","iopub.status.idle":"2024-06-23T23:27:07.163364Z","shell.execute_reply.started":"2024-06-23T23:27:07.132570Z","shell.execute_reply":"2024-06-23T23:27:07.162265Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def angular_dist_score(az_true, zen_true, az_pred, zen_pred):\n '''\n calculate the MAE of the angular distance between two directions.\n The two vectors are first converted to cartesian unit vectors,\n and then their scalar product is computed, which is equal to\n the cosine of the angle between the two vectors. The inverse \n cosine (arccos) thereof is then the angle between the two input vectors\n \n Parameters:\n -----------\n \n az_true : float (or array thereof)\n true azimuth value(s) in radian\n zen_true : float (or array thereof)\n true zenith value(s) in radian\n az_pred : float (or array thereof)\n predicted azimuth value(s) in radian\n zen_pred : float (or array thereof)\n predicted zenith value(s) in radian\n \n Returns:\n --------\n \n dist : float\n mean over the angular distance(s) in radian\n '''\n \n if not (np.all(np.isfinite(az_true)) and\n np.all(np.isfinite(zen_true)) and\n np.all(np.isfinite(az_pred)) and\n np.all(np.isfinite(zen_pred))):\n raise ValueError(\"All arguments must be finite\")\n \n # pre-compute all sine and cosine values\n sa1 = np.sin(az_true)\n ca1 = np.cos(az_true)\n sz1 = np.sin(zen_true)\n cz1 = np.cos(zen_true)\n \n sa2 = np.sin(az_pred)\n ca2 = np.cos(az_pred)\n sz2 = np.sin(zen_pred)\n cz2 = np.cos(zen_pred)\n \n # scalar product of the two cartesian vectors (x = sz*ca, y = sz*sa, z = cz)\n scalar_prod = sz1*sz2*(ca1*ca2 + sa1*sa2) + (cz1*cz2)\n \n # scalar product of two unit vectors is always between -1 and 1, this is against nummerical instability\n # that might otherwise occure from the finite precision of the sine and cosine functions\n scalar_prod = np.clip(scalar_prod, -1, 1)\n \n # convert back to an angle (in radian)\n return np.average(np.abs(np.arccos(scalar_prod)))","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:27:07.165115Z","iopub.execute_input":"2024-06-23T23:27:07.165545Z","iopub.status.idle":"2024-06-23T23:27:07.175728Z","shell.execute_reply.started":"2024-06-23T23:27:07.165506Z","shell.execute_reply":"2024-06-23T23:27:07.174502Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\n

💡 Objective:

\n\nThe detection of pulses generated by Cherenkov radiation is carried out using sensors in the IceCube detector. When a neutrino interacts with an atomic nucleus in the ice, it produces charged particles such as electrons, positrons, or muons, which move at speeds faster than the speed of light in that medium. This high speed generates an electromagnetic shockwave known as Cherenkov radiation. The sensors in IceCube are designed to detect and record the light emitted by this radiation as it passes through the ice.\n\nPrincipal Component Analysis (PCA) is a statistical technique used to reduce dimensionality and extract important information from a dataset. In the context of IceCube, PCA is applied to the pulses of Cherenkov radiation detected by the sensors. PCA identifies the principal components that capture the most variability in the data and ranks them based on their importance.\n\nTo reconstruct the direction of a neutrino in a vector space using PC1, the goal is to explain the variance of the data. PC1 represents the principal direction of variability in the Cherenkov radiation pulses. By analyzing the data and calculating PC1, the direction where the most information is concentrated can be determined. This direction corresponds to the incident neutrino's trajectory.\n\nIn a vector space, each pulse of Cherenkov radiation is considered as a vector with components representing different characteristics of the pulse. By using PCA and finding PC1, the vector pointing in the principal direction of pulse variance is obtained. This direction is used to reconstruct the arrival direction of the neutrino since it is related to the propagation direction of the Cherenkov radiation, which in turn is associated with the neutrino's trajectory.\n\nIn summary, the process involves detecting the pulses generated by Cherenkov radiation, applying PCA to find PC1, and using this principal direction to reconstruct the neutrino's direction in a vector space. By analyzing the data variance, we can reconstruct an approximate direction close to that of the neutrino.\n\n# Notebooks references\n \n Averkova Notebook <---\n \n Graysnow Notebook.<---\n\n","metadata":{}},{"cell_type":"code","source":"def Load_event(idx, batch_names, meta, is_train = True):\n # Read batch Data\n batch_df = pd.read_parquet(Batch2Path(batch_names, is_train)).reset_index()\n batch_id = GetBatchId(batch_names)\n \n if is_train:\n\n #Features: Charge, Time, auxiliary, x, y, z\n relevant_meta = meta[meta[\"batch_id\"] == batch_id].reset_index(drop=True)\n event_count = len(relevant_meta)\n else:\n relevant_meta = meta\n \n\n #Get Event Features (Time, charge, aux, x, y, z)\n first_pulse_index, last_pulse_index, event_id = relevant_meta.iloc[idx][[\"first_pulse_index\", \"last_pulse_index\", \"event_id\"]].astype(int)\n event_feature = batch_df[first_pulse_index: last_pulse_index + 1]\n event_feature = event_feature.merge(sensor_geometry, on=\"sensor_id\", how=\"left\")[[\"time\", \"charge\", \"auxiliary\", \"x\", \"y\", \"z\", \"sensor_id\"]]\n event_feature[\"time\"] -= event_feature[\"time\"].min()\n \n event_feature.x = event_feature.x - event_feature.x.mean()\n event_feature.y = event_feature.y - event_feature.y.mean()\n event_feature.z = event_feature.z - event_feature.z.mean()\n \n if is_train:\n \n #Get angles and Directon Vector\n azimuth, zenith = relevant_meta.iloc[idx][[\"azimuth\", \"zenith\"]].values.T.astype('float16')\n true_angles = np.array([zenith, azimuth])\n vector = np.array([\n np.sin(zenith) * np.cos(azimuth),\n np.sin(zenith) * np.sin(azimuth),\n np.cos(azimuth)\n ])\n \n vector_escalar = np.array([-500, 500])\n x = vector_escalar * vector[0]\n y = vector_escalar * vector[1]\n z = vector_escalar * vector[2]\n true_direction = pd.DataFrame({'x': x, 'y': y, 'z': z})\n \n #Get Prediction whit PC1\n vector_escalar = np.array([-500, 500])\n pca = PCA(n_components = 1).fit(event_feature.loc[~event_feature.auxiliary][[\"x\", \"y\", \"z\"]])\n vector_pca = pca.components_[0]\n x_pred = vector_escalar *vector_pca[0]\n y_pred = vector_escalar *vector_pca[1]\n z_pred = vector_escalar *vector_pca[2]\n pca_direction = pd.DataFrame({'x_pred': x_pred, 'y_pred': y_pred, 'z_pred': z_pred})\n \n zenith_pca = np.arccos(vector_pca[2])\n azimuth_pca = np.arctan2(vector_pca[1], vector_pca[0])\n if azimuth_pca < 0:\n azimuth_pca = 2*np.pi + azimuth_pca\n \n pca_angles = np.array([zenith_pca, azimuth_pca])\n \n #Prepare Output\n if is_train:\n true_angles_direction = [true_angles, true_direction]\n pca_angles_direction = [pca_angles, pca_direction]\n \n \n \n #output info\n print(\"\")\n print(\"==\" * 20)\n if is_train:\n print(f\"Event info.\\n1.Batch Number: {batch_id}\\n2.Event Index of batch_{batch_id}: {idx}\\n3. Event_id: {event_id}\\n4.Zenith_true: {zenith} - Zenith_pca: {zenith_pca}\\n5.Azimuth: {azimuth} - Azimuth_pca: {azimuth_pca}\")\n else:\n print(f\"Event info.\\n1.Batch Number: {batch_id}\\n2.Event Index of batch_{batch_id}: {idx}\\n3. Event_id: {event_id}\\n4.Zenith_pca: {zenith_pca}\\n5.Azimuth_pca: {azimuth_pca}\")\n \n print(\"\")\n print(\"==\" * 20) \n del batch_df\n gc.collect()\n \n #visualize event\n pulses = px.scatter_3d(event_feature, x=\"x\", y=\"y\", z=\"z\", opacity = .5 ,color=\"auxiliary\",\n color_discrete_map={True: \"steelblue\", False: \"aquamarine\"})\n \n if is_train:\n pulses.update_traces(marker_size = event_feature.charge * 10)\n true_direction = px.line_3d(true_direction, x = \"x\", y = \"y\", z = \"z\", color_discrete_sequence=['aquamarine'])\n \n pc1_direction = px.line_3d(pca_direction, x = \"x_pred\", y = \"y_pred\", z = \"z_pred\", color_discrete_sequence=['white'])\n\n fig = go.Figure(data = pulses.data + true_direction.data + pc1_direction.data)\n fig.update_layout(template='plotly_dark')\n fig.show()\n \n else:\n pulses.update_traces(marker_size = event_feature.charge * 10)\n \n pc1_direction = px.line_3d(pca_direction, x = \"x_pred\", y = \"y_pred\", z = \"z_pred\", color_discrete_sequence=['white'])\n\n fig = go.Figure(data = pulses.data + pc1_direction.data)\n fig.update_layout(template='plotly_dark')\n fig.show()\n \n display(event_feature)\n \n # Value of error\n print(\"\")\n print(\"==\" * 20)\n print(f\"the Varianza explained by PC1 is: {pca.explained_variance_ratio_[0]:.2%}\")\n if is_train:\n print(f'The value of error: {angular_dist_score(azimuth, zenith,azimuth_pca, zenith_pca ):.2f}')\n return event_feature, true_angles_direction, pca_angles_direction\n else:\n return azimuth_pca, zenith_pca","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:32:43.498396Z","iopub.execute_input":"2024-06-23T23:32:43.499444Z","iopub.status.idle":"2024-06-23T23:32:43.524839Z","shell.execute_reply.started":"2024-06-23T23:32:43.499391Z","shell.execute_reply":"2024-06-23T23:32:43.523528Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Normal Event","metadata":{}},{"cell_type":"code","source":"event_df, true_dir, pred_dir = Load_event(10, \"batch_1.parquet\", meta_train, is_train = True)","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:46:18.024425Z","iopub.execute_input":"2024-06-23T23:46:18.024959Z","iopub.status.idle":"2024-06-23T23:46:22.213472Z","shell.execute_reply.started":"2024-06-23T23:46:18.024919Z","shell.execute_reply":"2024-06-23T23:46:22.212155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"true_dir","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:51:27.402807Z","iopub.execute_input":"2024-06-23T23:51:27.404266Z","iopub.status.idle":"2024-06-23T23:51:27.415125Z","shell.execute_reply.started":"2024-06-23T23:51:27.404213Z","shell.execute_reply":"2024-06-23T23:51:27.413945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_dir","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:51:33.405089Z","iopub.execute_input":"2024-06-23T23:51:33.405598Z","iopub.status.idle":"2024-06-23T23:51:33.416275Z","shell.execute_reply.started":"2024-06-23T23:51:33.405554Z","shell.execute_reply":"2024-06-23T23:51:33.414929Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objs as go\nfrom plotly.subplots import make_subplots\n\na = event_df[event_df.auxiliary == False].reset_index(drop=True)\n\ncharge_column = a['charge']\n\nfig = make_subplots(rows=1, cols=3, subplot_titles=('Diagrama de Caja', 'Diagrama de Violín', 'Histograma'),\n column_widths=[0.35, 0.35, 0.3])\n\n# Agregar un diagrama de caja\nfig.add_trace(go.Box(y=a['charge'], name='Caja', marker_color='aquamarine'), row=1, col=1)\n\n# Agregar un diagrama de violín\nfig.add_trace(go.Violin(y=a['charge'], name='Violín', line_color='steelblue'), row=1, col=2)\n\n# Agregar un histograma\nfig.add_trace(go.Histogram(x=a['charge'], marker_color='indianred', opacity=0.7, name='Histograma'), row=1, col=3)\n\n# Actualizar el diseño (layout) para tener un fondo negro\nfig.update_layout({\n 'plot_bgcolor': 'black',\n 'paper_bgcolor': 'black',\n 'font': {'color': 'white'},\n 'title': {'text': 'Distribución de Cargas (P.E)', 'font': {'color': 'white'}},\n})\n\n# Actualizar los títulos de los subplots para que sean blancos\nfig.update_annotations(font=dict(color='white'))\n\n# Ajustar la separación entre gráficos\nfig.update_layout(height=700, width=1400, showlegend=False, title_text='Carga', title_font=dict(size=25))\n\n# Mostrar el gráfico\nfig.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:47:01.731787Z","iopub.execute_input":"2024-06-23T23:47:01.733045Z","iopub.status.idle":"2024-06-23T23:47:01.799686Z","shell.execute_reply.started":"2024-06-23T23:47:01.732994Z","shell.execute_reply":"2024-06-23T23:47:01.798207Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# High Energy Event","metadata":{}},{"cell_type":"code","source":"event_df, true_dir, pred_dir = Load_event(69325, \"batch_1.parquet\", meta_train, is_train = True)\n","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:47:09.712421Z","iopub.execute_input":"2024-06-23T23:47:09.712922Z","iopub.status.idle":"2024-06-23T23:47:12.458368Z","shell.execute_reply.started":"2024-06-23T23:47:09.712877Z","shell.execute_reply":"2024-06-23T23:47:12.457057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import plotly.graph_objs as go\nfrom plotly.subplots import make_subplots\n\na = event_df[event_df.auxiliary == False].reset_index(drop=True)\n\ncharge_column = a['charge']\n\nfig = make_subplots(rows=1, cols=3, subplot_titles=('Diagrama de Caja', 'Diagrama de Violín', 'Histograma'),\n column_widths=[0.35, 0.35, 0.3])\n\n# Agregar un diagrama de caja\nfig.add_trace(go.Box(y=a['charge'], name='Caja', marker_color='aquamarine'), row=1, col=1)\n\n# Agregar un diagrama de violín\nfig.add_trace(go.Violin(y=a['charge'], name='Violín', line_color='steelblue'), row=1, col=2)\n\n# Agregar un histograma\nfig.add_trace(go.Histogram(x=a['charge'], marker_color='indianred', opacity=0.7, name='Histograma'), row=1, col=3)\n\n# Actualizar el diseño (layout) para tener un fondo negro\nfig.update_layout({\n 'plot_bgcolor': 'black',\n 'paper_bgcolor': 'black',\n 'font': {'color': 'white'},\n 'title': {'text': 'Distribución de Cargas (P.E)', 'font': {'color': 'white'}},\n})\n\n# Actualizar los títulos de los subplots para que sean blancos\nfig.update_annotations(font=dict(color='white'))\n\n# Ajustar la separación entre gráficos\nfig.update_layout(height=700, width=1400, showlegend=False, title_text='Carga', title_font=dict(size=25))\n\n# Mostrar el gráfico\nfig.show()\n","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:47:13.530584Z","iopub.execute_input":"2024-06-23T23:47:13.531025Z","iopub.status.idle":"2024-06-23T23:47:13.591277Z","shell.execute_reply.started":"2024-06-23T23:47:13.530988Z","shell.execute_reply":"2024-06-23T23:47:13.590039Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Experimental Graph Operations","metadata":{}},{"cell_type":"markdown","source":"Ignore this part, im preparing something to the future with these matrixs","metadata":{}},{"cell_type":"code","source":"event_df = event_df[event_df[\"auxiliary\"] == True]","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:27:18.968116Z","iopub.execute_input":"2024-06-23T23:27:18.968549Z","iopub.status.idle":"2024-06-23T23:27:18.975281Z","shell.execute_reply.started":"2024-06-23T23:27:18.968510Z","shell.execute_reply":"2024-06-23T23:27:18.974202Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"event_df.drop(\"auxiliary\", axis = 1, inplace = True)","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:27:18.976711Z","iopub.execute_input":"2024-06-23T23:27:18.977197Z","iopub.status.idle":"2024-06-23T23:27:18.988250Z","shell.execute_reply.started":"2024-06-23T23:27:18.977154Z","shell.execute_reply":"2024-06-23T23:27:18.986831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%time\nG = nx.Graph()\nG.add_nodes_from(event_df.sensor_id)\nnum_nodes = len(G.nodes)\ndist_matrix = np.zeros((num_nodes, num_nodes))\nedges = []\n\nfor s_id_i in event_df.sensor_id:\n for s_id_j in event_df.sensor_id:\n edges.append((s_id_i, s_id_j))\n \nG.add_edges_from(edges)\n\nfor i, s_id_i in enumerate(G.nodes):\n for j, s_id_j in enumerate(G.nodes):\n if i != j:\n dist = np.linalg.norm(event_df[event_df[\"sensor_id\"] == s_id_i][[\"x\", \"y\", \"z\"]].values[0] - event_df[event_df[\"sensor_id\"] == s_id_j][[\"x\", \"y\", \"z\"]].values[0])\n dist_matrix[i, j] = dist\n dist_matrix[j, i] = dist \n\n\n\nprint(\"Matriz de adyacencia de distancias euclidianas:\")\nprint(dist_matrix.shape)\n","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:27:18.990120Z","iopub.execute_input":"2024-06-23T23:27:18.990453Z","iopub.status.idle":"2024-06-23T23:28:00.200177Z","shell.execute_reply.started":"2024-06-23T23:27:18.990424Z","shell.execute_reply":"2024-06-23T23:28:00.198736Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt \n\n# Dibujamos el grafo\nplt.figure(figsize=(10, 6))\nnx.draw(G, with_labels=True, node_color='skyblue', node_size=500, font_size=10)\nplt.title('Grafo de distancias euclidianas entre sensores')\nplt.show()\n\nprint(\"Matriz de adyacencia de distancias euclidianas:\")\nprint(dist_matrix.shape)","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:00.201745Z","iopub.execute_input":"2024-06-23T23:28:00.202165Z","iopub.status.idle":"2024-06-23T23:28:01.911592Z","shell.execute_reply.started":"2024-06-23T23:28:00.202132Z","shell.execute_reply":"2024-06-23T23:28:01.910488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"event_df = event_df.groupby(\"sensor_id\").agg({\"charge\" : \"mean\", \"time\" : \"mean\"}).reset_index()","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:01.913114Z","iopub.execute_input":"2024-06-23T23:28:01.913541Z","iopub.status.idle":"2024-06-23T23:28:01.927896Z","shell.execute_reply.started":"2024-06-23T23:28:01.913502Z","shell.execute_reply":"2024-06-23T23:28:01.926722Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = []\ntarget = []\ndata.append((event_df, dist_matrix))\ntarget.append(true_dir[0])","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:01.929298Z","iopub.execute_input":"2024-06-23T23:28:01.929649Z","iopub.status.idle":"2024-06-23T23:28:01.935598Z","shell.execute_reply.started":"2024-06-23T23:28:01.929609Z","shell.execute_reply":"2024-06-23T23:28:01.934430Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n\n\nclass GCNLayer(nn.Module):\n def __init__(self, in_features, out_features):\n super(GCNLayer, self).__init__()\n self.linear = nn.Linear(in_features, out_features)\n\n def forward(self, x, adjacency_matrix):\n # Realiza la multiplicación matricial entre las características de los nodos y la matriz de adyacencia\n x = torch.matmul(adjacency_matrix, self.linear(x))\n return F.relu(x)\n\nclass GCN(nn.Module):\n def __init__(self, in_features, hidden_features, out_features):\n super(GCN, self).__init__()\n self.layer1 = GCNLayer(in_features, hidden_features)\n self.layer2 = GCNLayer(hidden_features, out_features)\n\n def forward(self, data):\n outputs = []\n for event_df, dist_matrix in data:\n # Obtener las características y la matriz de adyacencia de cada evento\n x = event_df\n adjacency_matrix = dist_matrix\n\n # Pasar los datos a través de las capas GCN\n output = self.layer1(x, adjacency_matrix)\n output = self.layer2(output, adjacency_matrix)\n outputs.append(output)\n return torch.stack(outputs)\n\n# Ejemplo de uso\n# Definición de las dimensiones de entrada, ocultas y de salida\nin_features = 3 # Carga, tiempo, posición (x, y, z)\nhidden_features = 64\nout_features = 2 # Azimuth, Zenith\n\n# Creación de una instancia de la GCN\ngcn = GCN(in_features, hidden_features, out_features)\n\n\nevent_df_tensor = torch.tensor(event_df.values, dtype=torch.float32)\ndist_matrix_tensor = torch.tensor(dist_matrix, dtype=torch.float32)\ntarget_tensor = torch.tensor(target, dtype=torch.float32)\ndata = [(event_df_tensor, dist_matrix_tensor)]\n\n\n# Pasar los datos por la red GCN\noutput = gcn(data)\n\nprint(\"Output shape:\", output.shape) # Salida de la GCN\n","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:01.937336Z","iopub.execute_input":"2024-06-23T23:28:01.937661Z","iopub.status.idle":"2024-06-23T23:28:04.999745Z","shell.execute_reply.started":"2024-06-23T23:28:01.937634Z","shell.execute_reply":"2024-06-23T23:28:04.998533Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#

Sample Submission...

","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"meta_test = pd.read_parquet(\"/kaggle/input/icecube-neutrinos-in-deep-ice/test_meta.parquet\")\ntest = pd.read_parquet(\"/kaggle/input/icecube-neutrinos-in-deep-ice/test/batch_661.parquet\")","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:05.001798Z","iopub.execute_input":"2024-06-23T23:28:05.002905Z","iopub.status.idle":"2024-06-23T23:28:05.027131Z","shell.execute_reply.started":"2024-06-23T23:28:05.002840Z","shell.execute_reply":"2024-06-23T23:28:05.025909Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.read_parquet(\"/kaggle/input/icecube-neutrinos-in-deep-ice/sample_submission.parquet\")\nfor i in range(3):\n azimuth, zenith = Load_event(i, \"batch_661.parquet\", meta_test, is_train = False)\n submission.iloc[i, 1] = azimuth\n submission.iloc[i, 2] = zenith","metadata":{"execution":{"iopub.status.busy":"2024-06-23T23:28:05.028729Z","iopub.execute_input":"2024-06-23T23:28:05.029265Z","iopub.status.idle":"2024-06-23T23:28:06.180473Z","shell.execute_reply.started":"2024-06-23T23:28:05.029221Z","shell.execute_reply":"2024-06-23T23:28:06.179300Z"},"trusted":true},"execution_count":null,"outputs":[]}]}