{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":39763,"databundleVersionId":11756775,"sourceType":"competition"}],"dockerImageVersionId":30918,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Published on April 08, 2023. By Prata, Marília, (mpwolke)","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\n#Two lines Required to Plot Plotly\nimport plotly.io as pio\npio.renderers.default = 'iframe'\n\nimport plotly.graph_objs as go\nimport plotly.offline as py\nimport plotly.express as px\n\n#Ignore warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\nimport tensorflow as tf\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:00:52.696631Z","iopub.execute_input":"2025-04-09T00:00:52.697036Z","iopub.status.idle":"2025-04-09T00:01:24.362840Z","shell.execute_reply.started":"2025-04-09T00:00:52.696999Z","shell.execute_reply":"2025-04-09T00:01:24.360611Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"![](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQOJvo6kr5e81-_k8G5SUYdAePMZ84kdqAcOA&s)Instagram\n\nFull Waveform Inversion (FWI) is a revolutionary method in seismology that uses the entire wavefield to create high-resolution subsurface models","metadata":{}},{"cell_type":"markdown","source":"## Competition Citation\n\n@misc{waveform-inversion,\n\n    author = {Youzuo Lin and Lu Lu and Walter Reade and Addison Howard and Maria Cruz and Ashley Chow and Yale-UNC Data-Driven Full Waveform Inversion},\n    \n    title = { Yale/UNC-CH - Geophysical Waveform Inversion},\n    \n    year = {2025},\n    \n    howpublished = {\\url{https://kaggle.com/competitions/waveform-inversion}},\n    \n    note = {Kaggle}\n}","metadata":{}},{"cell_type":"markdown","source":"### Competition Overview\n\n\"In this competition, you’ll estimate subsurface properties—like velocity maps—from on seismic waveform data. Known as Full Waveform Inversion (FWI), this process could lead to more accurate and efficient seismic analysis, making it more useful in a variety of fields.\"\n\nhttps://www.kaggle.com/competitions/waveform-inversion","metadata":{}},{"cell_type":"code","source":"sub = pd.read_csv('/kaggle/input/waveform-inversion/sample_submission.csv')\nsub.tail()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T22:56:48.512081Z","iopub.execute_input":"2025-04-08T22:56:48.512396Z","iopub.status.idle":"2025-04-08T22:57:07.705425Z","shell.execute_reply.started":"2025-04-08T22:56:48.512372Z","shell.execute_reply":"2025-04-08T22:57:07.704534Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Load one npy file","metadata":{}},{"cell_type":"code","source":"img = np.load('../input/waveform-inversion/train_samples/FlatFault_A/seis4_1_0.npy')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:04:31.373148Z","iopub.execute_input":"2025-04-09T00:04:31.373571Z","iopub.status.idle":"2025-04-09T00:04:36.041560Z","shell.execute_reply.started":"2025-04-09T00:04:31.373541Z","shell.execute_reply":"2025-04-09T00:04:36.040500Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"img.dtype","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:05:40.899870Z","iopub.execute_input":"2025-04-08T23:05:40.900243Z","iopub.status.idle":"2025-04-08T23:05:40.905097Z","shell.execute_reply.started":"2025-04-08T23:05:40.900138Z","shell.execute_reply":"2025-04-08T23:05:40.904457Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(img.shape)\nprint(3 * 4096)\nnpshape = 3 * 4096","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:05:55.641455Z","iopub.execute_input":"2025-04-08T23:05:55.641717Z","iopub.status.idle":"2025-04-08T23:05:55.645844Z","shell.execute_reply.started":"2025-04-08T23:05:55.641696Z","shell.execute_reply":"2025-04-08T23:05:55.645057Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"img = tf.io.read_file('../input/waveform-inversion/train_samples/FlatFault_A/seis4_1_0.npy')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:04:50.784761Z","iopub.execute_input":"2025-04-09T00:04:50.785204Z","iopub.status.idle":"2025-04-09T00:04:51.967966Z","shell.execute_reply.started":"2025-04-09T00:04:50.785172Z","shell.execute_reply":"2025-04-09T00:04:51.966905Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"img = tf.io.decode_raw(img, tf.float64)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:04:56.420786Z","iopub.execute_input":"2025-04-09T00:04:56.421214Z","iopub.status.idle":"2025-04-09T00:04:57.250794Z","shell.execute_reply.started":"2025-04-09T00:04:56.421181Z","shell.execute_reply":"2025-04-09T00:04:57.249907Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"tensorshape = img.shape[0]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:07:29.631146Z","iopub.execute_input":"2025-04-08T23:07:29.631403Z","iopub.status.idle":"2025-04-08T23:07:29.634536Z","shell.execute_reply.started":"2025-04-08T23:07:29.631383Z","shell.execute_reply":"2025-04-08T23:07:29.633818Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"tensorshape - npshape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:07:43.845829Z","iopub.execute_input":"2025-04-08T23:07:43.846181Z","iopub.status.idle":"2025-04-08T23:07:43.851194Z","shell.execute_reply.started":"2025-04-08T23:07:43.846156Z","shell.execute_reply":"2025-04-08T23:07:43.850356Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(img[16:]) # npy value","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:08:03.859403Z","iopub.execute_input":"2025-04-08T23:08:03.859662Z","iopub.status.idle":"2025-04-08T23:08:03.869014Z","shell.execute_reply.started":"2025-04-08T23:08:03.859643Z","shell.execute_reply":"2025-04-08T23:08:03.868323Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dtype=32\nremove_len = 1024//dtype\nimg = img[remove_len:]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-08T23:08:22.910827Z","iopub.execute_input":"2025-04-08T23:08:22.911131Z","iopub.status.idle":"2025-04-08T23:08:22.915604Z","shell.execute_reply.started":"2025-04-08T23:08:22.911111Z","shell.execute_reply":"2025-04-08T23:08:22.914778Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Failing to plot npy file","metadata":{}},{"cell_type":"markdown","source":"#### Pycbc is used to explore astrophysical sources of gravitational waves. I was desperate.\n\nIn this competition, you’ll estimate subsurface properties—like velocity maps—from on seismic waveform data. \n\nFirst error: metadata-generation-failed","metadata":{}},{"cell_type":"code","source":"!pip install pycbc","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:56:17.183039Z","iopub.execute_input":"2025-04-09T00:56:17.183714Z","iopub.status.idle":"2025-04-09T00:56:26.336692Z","shell.execute_reply.started":"2025-04-09T00:56:17.183666Z","shell.execute_reply":"2025-04-09T00:56:26.335194Z"},"_kg_hide-output":true,"collapsed":true,"jupyter":{"outputs_hidden":true}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#Code by Nayu. T.S.https://www.kaggle.com/nayuts/let-s-visualize-data-to-understand\n\ntrain_id = \"seis4_1_0\"\nsignal_array = np.load(f\"../input/waveform-inversion/train_samples/FlatFault_A/{train_id}.npy\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:18:03.580264Z","iopub.execute_input":"2025-04-09T00:18:03.580668Z","iopub.status.idle":"2025-04-09T00:18:03.964916Z","shell.execute_reply.started":"2025-04-09T00:18:03.580637Z","shell.execute_reply":"2025-04-09T00:18:03.963766Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from IPython import display\nimport matplotlib.pyplot as plt\nfrom scipy.ndimage import distance_transform_edt","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:24:23.971210Z","iopub.execute_input":"2025-04-09T00:24:23.971627Z","iopub.status.idle":"2025-04-09T00:24:23.976933Z","shell.execute_reply.started":"2025-04-09T00:24:23.971598Z","shell.execute_reply":"2025-04-09T00:24:23.975520Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#By Lupin11 https://www.kaggle.com/code/lupin11/weight-maps-for-better-boundaries\n\ndef get_ids(tar_path):\n    ids = []\n    for img_id in os.listdir(tar_path):\n        ids.append(img_id)\n    print(f\"{len(ids)} samples in {tar_path}\")\n    return ids\n\ntar_path = \"/kaggle/input/waveform-inversion/train_samples\"\nids = get_ids(tar_path)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:25:32.810135Z","iopub.execute_input":"2025-04-09T00:25:32.810619Z","iopub.status.idle":"2025-04-09T00:25:32.822289Z","shell.execute_reply.started":"2025-04-09T00:25:32.810586Z","shell.execute_reply":"2025-04-09T00:25:32.820858Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#By Lupin11 https://www.kaggle.com/code/lupin11/weight-maps-for-better-boundaries\n\ndef gen_weight_map(image, sigma=0.2):\n    \"\"\"\n    Generates a weight map based on the ground truth.\n    \n    Args:\n        image (numpy.ndarray): Ground truth, binary image of shape (height, width).\n        sigma (float): Controls the range of boundaries.\n        \n    Returns:\n        weight_map (numpy.ndarray): Weight map of the same shape as the ground truth.\n    \"\"\"\n    distance = distance_transform_edt(1 - image)\n    distance = distance / np.max(distance)\n    weight_map = np.exp(-0.5 * (distance / sigma) ** 2)\n    weight_map[image == 1] = 0\n    return weight_map","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:25:57.855091Z","iopub.execute_input":"2025-04-09T00:25:57.855675Z","iopub.status.idle":"2025-04-09T00:25:57.861756Z","shell.execute_reply.started":"2025-04-09T00:25:57.855636Z","shell.execute_reply":"2025-04-09T00:25:57.860447Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### TypeError: Invalid shape (500, 5, 1000, 70) for image data","metadata":{}},{"cell_type":"code","source":"#By Lupin11 https://www.kaggle.com/code/lupin11/weight-maps-for-better-boundaries\n\nfor img_id in ids[:20]:  # visualize 20 images\n    sample_path = f\"{tar_path}/{img_id}\"\n    gt = np.load(f\"{sample_path}//seis4_1_0.npy\")\n    if np.all(gt == 0):  # skip images without contrails\n        continue\n    weight_map = gen_weight_map(gt)  # generate the weight map\n    plt.figure(figsize=(6, 3))\n    ax = plt.subplot(1, 2, 1)\n    ax.imshow(gt, interpolation='none')\n    ax.set_title('GroundTruth')\n    ax = plt.subplot(1, 2, 2)\n    ax.imshow(weight_map, interpolation='none')\n    ax.set_title('WeightMap')\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:28:07.396172Z","iopub.execute_input":"2025-04-09T00:28:07.396583Z","iopub.status.idle":"2025-04-09T00:28:48.551585Z","shell.execute_reply.started":"2025-04-09T00:28:07.396552Z","shell.execute_reply":"2025-04-09T00:28:48.549924Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Trying gwpy and failing","metadata":{}},{"cell_type":"code","source":"!python -m pip install gwpy","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:39:09.195472Z","iopub.execute_input":"2025-04-09T00:39:09.195810Z","iopub.status.idle":"2025-04-09T00:39:23.042264Z","shell.execute_reply.started":"2025-04-09T00:39:09.195787Z","shell.execute_reply":"2025-04-09T00:39:23.040832Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#By Geir Drange https://www.kaggle.com/mistag/data-preprocessing-with-gwpy\n\nfrom gwpy.timeseries import TimeSeries\nfrom gwpy.plot import Plot\n\ndef read_file(fname):\n    data = np.load(fname)\n    d1 = TimeSeries(data[0,:], sample_rate=2048)\n    d2 = TimeSeries(data[1,:], sample_rate=2048)\n    d3 = TimeSeries(data[2,:], sample_rate=2048)\n    return d1, d2, d3","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:39:28.691474Z","iopub.execute_input":"2025-04-09T00:39:28.691921Z","iopub.status.idle":"2025-04-09T00:39:31.166771Z","shell.execute_reply.started":"2025-04-09T00:39:28.691885Z","shell.execute_reply":"2025-04-09T00:39:31.165913Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#By Geir Drange https://www.kaggle.com/mistag/data-preprocessing-with-gwpy\n\ndef create_rgb(fname):\n    r1, r2, r3 = read_file(fname)\n    p1, p2, p3 = preprocess(r1, r2, r3)\n    hq1 = p1.q_transform(outseg=(0, 2))\n    hq2 = p2.q_transform(outseg=(0, 2))\n    hq3 = p3.q_transform(outseg=(0, 2))\n    img = np.zeros([hq1.shape[0], hq1.shape[1], 3], dtype=np.uint8)\n    scaler = MinMaxScaler()\n    img[:,:,0] = 255*scaler.fit_transform(hq1)\n    img[:,:,1] = 255*scaler.fit_transform(hq2)\n    img[:,:,2] = 255*scaler.fit_transform(hq3)\n    return Image.fromarray(img).rotate(90, expand=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:39:37.583023Z","iopub.execute_input":"2025-04-09T00:39:37.584036Z","iopub.status.idle":"2025-04-09T00:39:37.591787Z","shell.execute_reply.started":"2025-04-09T00:39:37.583991Z","shell.execute_reply":"2025-04-09T00:39:37.590261Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#### ValueError: Cannot generate TimeSeries with 3-dimensional data","metadata":{}},{"cell_type":"code","source":"create_rgb('../input/waveform-inversion/train_samples/FlatFault_A/seis4_1_0.npy')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-04-09T00:39:42.179663Z","iopub.execute_input":"2025-04-09T00:39:42.180133Z","iopub.status.idle":"2025-04-09T00:39:42.563883Z","shell.execute_reply.started":"2025-04-09T00:39:42.180095Z","shell.execute_reply":"2025-04-09T00:39:42.562535Z"},"collapsed":true,"jupyter":{"outputs_hidden":true},"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"#Issues, Issues and more Issues:\n\n#### I tried so many things. Unfortunately, I wasn't able to plot a single npy file. \n\nNo module pycbc\n\nValueError: x and y can be no greater than 2D, but have shapes (5,) and (5, 1000, 70)\n\n3D Error\n\nTime Series error\n\nUmap erro\n\nTukey window  error","metadata":{}},{"cell_type":"markdown","source":"#In the real world, Inversion look like this:\n\n#### I couldn't resist not adding that image Reade : )\n\n![](https://i.ytimg.com/vi/yFvsfoXxZoI/hqdefault.jpg)","metadata":{}},{"cell_type":"markdown","source":"#Acknowledgements:\n\nAlien https://www.kaggle.com/h053473666/use-tensorflow-tpu-to-decode-npy-files/notebook\n\nNayu. T.S.https://www.kaggle.com/nayuts/let-s-visualize-data-to-understand\n\nLupin11 https://www.kaggle.com/code/lupin11/weight-maps-for-better-boundaries\n\nAlex Nitz https://www.kaggle.com/alexnitz/pycbc-making-images\n\nGeir Drange https://www.kaggle.com/mistag/data-preprocessing-with-gwpy","metadata":{}}]}