I had a hard time understanding what was going on with the data. Here's a potentially more basic explanation for those new to the domain (like me).
\nWe’re dealing with a seismic imaging problem. In seismic imaging, energy waves are sent into the ground (from sources like vibroseis trucks or air guns), and then the signals that bounce back are recorded by receivers (geophones or hydrophones). We want to figure out how fast those waves travel at different points below the surface—this speed is called the subsurface velocity.
\nNow, imagine we cut the Earth in half so we can look at it from the side. Along that vertical slice, each point in that 2D cross-section has some speed (velocity) with which sound waves (seismic waves) travel. This 2D arrangement of speeds is called a velocity map. Meanwhile, up on the surface, we have instruments recording the wave signals over time, which we call seismic data.
\nSo the fundamental pair is:
\nShort answer: each “family” is a different type of 2D velocity cross-section (and corresponding seismic data) that was artificially generated to represent distinct geological scenarios.
\nVel Family
\nFault Family
\nStyle Family
\nEssentially, these three “families” cover a spectrum of geological complexity:
\nThey’re synthetic datasets. That means people used a physics simulator to generate the seismic data given an artificially created velocity map. This approach ensures we have a “true velocity map” for every sample, which is extremely valuable for training or testing machine learning algorithms, because real field data usually doesn’t have perfect “ground truth.”
\nEach family can have sub-variations:
\nFor example, FlatVel_A means the simplest version of layered velocity with almost no major complexity. CurveVel_B means a more complex version of layered velocity with significant curving/folding. FlatFault_A means a simpler fault scenario; CurveFault_B means a complex fault scenario with curved layers. And so on.
\nIn each subfolder, you see .npy files (NumPy format) containing:
.npy file holds a batch of 500 samples. Concretely, if you open, say, FlatVel_B, you might see something like:
\n/\n \n data1.npy <-- seismic samples\n data2.npy <-- another seismic samples\n model/\n model1.npy <-- velocity maps\n model2.npy <-- another velocity maps\nThe index in the file (like data1.npy[i] and model1.npy[i]) lines up the i-th seismic sample with the i-th velocity map.
For the Fault family, it’s the same pairing but with filenames like seis4_1_0.npy (seismic) and vel4_1_0.npy (velocity). Each still has 500 paired samples.
Seismic data is in 4 dimensions:
\nSo if the shape is (500, 5, 1000, 70), that means:
Velocity map is in 3 dimensions:
\nSo if the shape is (500, 70, 70), that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.
.npy files (like data1.npy/model1.npy pairs). This is what you can use to train or test your approach. Essentially, the competition is: “Given 3D seismic waveforms, can you predict the 2D velocity cross-section for each sample?”
\nIn real life, if you record seismic data in the field, you don’t automatically know the exact velocity structure underground (and that’s often what you want to find). Having synthetic data with known “answers” (velocity maps) helps us benchmark algorithms. The final step is to see who can do the best “inversion” from seismic to velocity.
\n.npy files holding (a) seismic data and (b) velocity maps. Each .npy holds 500 “examples” (pairs). That’s the entire structure in a nutshell. Each folder is basically a chunk of data with 500 training examples. The difference is just how the underground geology was generated (flat layers, curved layers, faults, or random images). The ultimate goal is to see if your method can handle them all!
\n\n\nIf you want a deeper dive into the background of the problem, AI generated podcasts on this topic, and more, check out the literature review and concept background notebook I've made here
\n
\n 👉 https://www.kaggle.com/code/tpmeli/geo-lit-review-winning-strategies-starter
\n", "messages": [ { "id": 3176264, "postDate": "2025-04-11T07:05:34.247Z", "content": "If you're looking for a comprehensive exploration of geographic and WFI data insights, including detailed visualizations, statistical analyses, and key observations, check out the Exploratory Deep Dive notebook I've prepared here:
\n
\n 👉 https://www.kaggle.com/code/tpmeli/exploratory-deep-dive-geo-wfi-data-insights
I had a hard time understanding what was going on with the data. Here's a potentially more basic explanation for those new to the domain (like me).
\nWe’re dealing with a seismic imaging problem. In seismic imaging, energy waves are sent into the ground (from sources like vibroseis trucks or air guns), and then the signals that bounce back are recorded by receivers (geophones or hydrophones). We want to figure out how fast those waves travel at different points below the surface—this speed is called the subsurface velocity.
\nNow, imagine we cut the Earth in half so we can look at it from the side. Along that vertical slice, each point in that 2D cross-section has some speed (velocity) with which sound waves (seismic waves) travel. This 2D arrangement of speeds is called a velocity map. Meanwhile, up on the surface, we have instruments recording the wave signals over time, which we call seismic data.
\nSo the fundamental pair is:
\nShort answer: each “family” is a different type of 2D velocity cross-section (and corresponding seismic data) that was artificially generated to represent distinct geological scenarios.
\nVel Family
\nFault Family
\nStyle Family
\nEssentially, these three “families” cover a spectrum of geological complexity:
\nThey’re synthetic datasets. That means people used a physics simulator to generate the seismic data given an artificially created velocity map. This approach ensures we have a “true velocity map” for every sample, which is extremely valuable for training or testing machine learning algorithms, because real field data usually doesn’t have perfect “ground truth.”
\nEach family can have sub-variations:
\nFor example, FlatVel_A means the simplest version of layered velocity with almost no major complexity. CurveVel_B means a more complex version of layered velocity with significant curving/folding. FlatFault_A means a simpler fault scenario; CurveFault_B means a complex fault scenario with curved layers. And so on.
\nIn each subfolder, you see .npy files (NumPy format) containing:
.npy file holds a batch of 500 samples. Concretely, if you open, say, FlatVel_B, you might see something like:
\n/\n \n data1.npy <-- seismic samples\n data2.npy <-- another seismic samples\n model/\n model1.npy <-- velocity maps\n model2.npy <-- another velocity maps\nThe index in the file (like data1.npy[i] and model1.npy[i]) lines up the i-th seismic sample with the i-th velocity map.
For the Fault family, it’s the same pairing but with filenames like seis4_1_0.npy (seismic) and vel4_1_0.npy (velocity). Each still has 500 paired samples.
Seismic data is in 4 dimensions:
\nSo if the shape is (500, 5, 1000, 70), that means:
Velocity map is in 3 dimensions:
\nSo if the shape is (500, 70, 70), that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.
.npy files (like data1.npy/model1.npy pairs). This is what you can use to train or test your approach. Essentially, the competition is: “Given 3D seismic waveforms, can you predict the 2D velocity cross-section for each sample?”
\nIn real life, if you record seismic data in the field, you don’t automatically know the exact velocity structure underground (and that’s often what you want to find). Having synthetic data with known “answers” (velocity maps) helps us benchmark algorithms. The final step is to see who can do the best “inversion” from seismic to velocity.
\n.npy files holding (a) seismic data and (b) velocity maps. Each .npy holds 500 “examples” (pairs). That’s the entire structure in a nutshell. Each folder is basically a chunk of data with 500 training examples. The difference is just how the underground geology was generated (flat layers, curved layers, faults, or random images). The ultimate goal is to see if your method can handle them all!
\n\n\nIf you want a deeper dive into the background of the problem, AI generated podcasts on this topic, and more, check out the literature review and concept background notebook I've made here
\n
\n 👉 https://www.kaggle.com/code/tpmeli/geo-lit-review-winning-strategies-starter
\n", "rawMarkdown": "I had a hard time understanding what was going on with the data. Here's a potentially more basic explanation for those new to the domain (like me).\n\n---\n\n## 1. What are we even talking about? \nWe’re dealing with a **seismic imaging** problem. In seismic imaging, energy waves are sent into the ground (from sources like vibroseis trucks or air guns), and then the signals that bounce back are recorded by receivers (geophones or hydrophones). We want to figure out how fast those waves travel at different points below the surface—this speed is called the **subsurface velocity**. \n\nNow, imagine we cut the Earth in half so we can look at it from the side. Along that vertical slice, each point in that 2D cross-section has some speed (velocity) with which sound waves (seismic waves) travel. This 2D arrangement of speeds is called a **velocity map**. Meanwhile, up on the surface, we have instruments recording the wave signals over time, which we call **seismic data**. \n\nSo the fundamental pair is: \n1. **Seismic data**: all the waveforms recorded over time. \n2. **Velocity map**: a 2D “picture” of how speed varies with depth and horizontal position.\n\n---\n\n## 2. Why are there three “families”? \n\n**Short answer**: each “family” is a different *type* of 2D velocity cross-section (and corresponding seismic data) that was artificially generated to represent distinct geological scenarios. \n\n1. **Vel Family** \n - **What it looks like:** Think of relatively orderly, layered sediments. “Vel” stands for “velocity,” but effectively this family has simpler layered structures (like a cake with flat or gently curved layers). \n - **Why it’s simpler:** The layers might be horizontal (FlatVel) or smoothly dipping/curving (CurveVel), but no major breaks in the rock. \n - **What the seismic data show:** You typically get clear, continuous reflection events. \n\n2. **Fault Family** \n - **What it looks like:** Same idea of layered geology, *but* now there are faults—places where the layers are cut and shifted relative to each other. \n - **Why it’s more complex:** A fault is basically a break in the Earth where one side of a layer is displaced relative to the other. This adds complexity. \n - **What the seismic data show:** Discontinuous reflections, sudden jumps, or “diffractions” at the fault lines.\n\n3. **Style Family** \n - **What it looks like:** Not your typical “layer cake.” Instead, it’s more chaotic or irregular shapes, because the creators used “style transfer” from arbitrary images to generate velocity patterns. Imagine velocity maps that might have swirls, blobs, or textures—no clean layering. \n - **Why it’s the most unpredictable:** It can mimic random or exotic geologies. \n - **What the seismic data show:** More scattered wave energy, less straightforward layering.\n\nEssentially, these three “families” cover a spectrum of geological complexity: \n- **Vel**: simpler stratified layers \n- **Fault**: layered but with big breaks (faults) \n- **Style**: all sorts of random or organic shapes\n\n---\n\n## 3. So, these are synthetic or real? \nThey’re **synthetic** datasets. That means people used a physics simulator to generate the seismic data *given* an artificially created velocity map. This approach ensures we have a “true velocity map” for every sample, which is extremely valuable for training or testing machine learning algorithms, because real field data usually doesn’t have perfect “ground truth.” \n\n---\n\n## 4. Why do we need so many subfolders? \nEach family can have sub-variations: \n- “A” vs. “B” versions: typically “A” is somewhat simpler (fewer random variations, fewer layers, gentler changes), while “B” is more complex (more layers, more variability). \n- “Flat” vs. “Curve” (for Vel or Fault): indicates whether layers are relatively flat or curved/folded. \n\nFor example, **FlatVel_A** means the simplest version of layered velocity with almost no major complexity. **CurveVel_B** means a more complex version of layered velocity with significant curving/folding. **FlatFault_A** means a simpler fault scenario; **CurveFault_B** means a complex fault scenario with curved layers. And so on.\n\n---\n\n## 5. What’s actually in each folder? \nIn each subfolder, you see `.npy` files (NumPy format) containing:\n- **Seismic data** — the time series recordings at the surface. Each seismic `.npy` file holds a batch of 500 samples. \n- **Velocity maps** — the 2D grid of wave speeds for those same 500 samples.\n\nConcretely, if you open, say, **FlatVel_B**, you might see something like: \n```\nFlatVel_B/\n data/\n data1.npy <-- 500 seismic samples\n data2.npy <-- another 500 seismic samples\n model/\n model1.npy <-- 500 velocity maps\n model2.npy <-- another 500 velocity maps\n```\nThe index in the file (like `data1.npy[i]` and `model1.npy[i]`) lines up the i-th seismic sample with the i-th velocity map.\n\nFor the **Fault** family, it’s the same pairing but with filenames like `seis4_1_0.npy` (seismic) and `vel4_1_0.npy` (velocity). Each still has 500 paired samples.\n\n---\n\n## 6. Shape of the Data: 4D vs. 3D \n- **Seismic data** is in 4 dimensions: \n 1. The “batch” dimension (500) for the number of samples. \n 2. The number of sources (like 5 different shot points). \n 3. The number of time steps (like 1000 samples in time). \n 4. The number of receivers (like 70 recording positions). \n\n So if the shape is `(500, 5, 1000, 70)`, that means: \n - 500 different subsurface scenarios \n - Each scenario has 5 seismic sources \n - Recorded for 1000 time steps \n - At 70 receiver positions.\n\n- **Velocity map** is in 3 dimensions:\n 1. The “batch” dimension (500) for the number of samples. \n 2. The height (70 grid points from top to bottom). \n 3. The width (70 grid points horizontally). \n\n So if the shape is `(500, 70, 70)`, that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.\n\n---\n\n## 7. The Competition Setup \n- You have a **training** folder with many `.npy` files (like `data1.npy`/`model1.npy` pairs). This is what you can use to train or test your approach. \n- You also get a **test** folder that only has the seismic data (no velocity). The competition wants you to predict the velocity maps for those unknown examples. \n- You submit your predictions in a particular CSV or NumPy format so the organizers can compare them to the real velocity maps (which they keep hidden for evaluation).\n\nEssentially, the competition is: “Given 3D seismic waveforms, can you predict the 2D velocity cross-section for each sample?” \n\n---\n\n## 8. Why does it matter? \nIn real life, if you record seismic data in the field, you don’t automatically know the exact velocity structure underground (and that’s often what you want to find). Having synthetic data with known “answers” (velocity maps) helps us benchmark algorithms. The final step is to see who can do the best “inversion” from seismic to velocity.\n\n---\n\n## Recap in Super-Simple Terms\n\n1. **3 Families**: \n - **Vel** = simpler, layered Earth. \n - **Fault** = layered Earth but broken by faults. \n - **Style** = more random, weird patterns. \n2. **Each family** has subfolders (e.g., FlatVel_A, CurveVel_B) indicating more or less complexity. \n3. **Inside each subfolder** are `.npy` files holding (a) seismic data and (b) velocity maps. Each `.npy` holds 500 “examples” (pairs). \n4. **The competition**: train a model on these known pairs to learn to predict velocity from seismic, then apply it to new test seismic data.\n\nThat’s the entire structure in a nutshell. Each folder is basically a chunk of data with 500 training examples. The difference is just how the underground geology was generated (flat layers, curved layers, faults, or random images). The ultimate goal is to see if your method can handle them all!\n\n# Next Steps\n\n## 👉 Literature Review + Papers with Code + Background\n>**If you want a deeper dive into the background of the problem, AI generated podcasts on this topic, and more, check out the literature review and concept background notebook I've made here**\n👉 https://www.kaggle.com/code/tpmeli/geo-lit-review-winning-strategies-starter\n\n## 👉 Exploratory Data Analysis (EDA)\n> **If you're looking for a comprehensive exploration of geographic and WFI data insights, including detailed visualizations, statistical analyses, and key observations, check out the Exploratory Deep Dive notebook I've prepared here:**\n👉 https://www.kaggle.com/code/tpmeli/exploratory-deep-dive-geo-wfi-data-insights\n", "votes": 155 }, { "id": 3187488, "postDate": "2025-04-26T05:14:29.447Z", "content": "If you're looking for a comprehensive exploration of geographic and WFI data insights, including detailed visualizations, statistical analyses, and key observations, check out the Exploratory Deep Dive notebook I've prepared here:
\n
\n 👉 https://www.kaggle.com/code/tpmeli/exploratory-deep-dive-geo-wfi-data-insights
Great insights!! thanks a lot for the spelled out explanation, I needed this. I joined this comp a few days earlier and I was stuck on what those 100 gigs of data were. This discussion really cleared my confusions.
", "rawMarkdown": "Great insights!! thanks a lot for the spelled out explanation, I needed this. I joined this comp a few days earlier and I was stuck on what those 100 gigs of data were. This discussion really cleared my confusions. ", "votes": 3 }, { "id": 3182840, "postDate": "2025-04-19T23:18:54.860Z", "content": "Thank you for providing valuable information. I had difficulty understanding the dimensionality of the seismic data, but your help was appreciated.
", "rawMarkdown": "Thank you for providing valuable information. I had difficulty understanding the dimensionality of the seismic data, but your help was appreciated.", "votes": 3, "replies": [ { "id": 3182867, "postDate": "2025-04-20T01:28:36.253Z", "content": "Happy it helped!
", "rawMarkdown": "Happy it helped! ", "votes": 2 } ] }, { "id": 3176655, "postDate": "2025-04-11T15:06:35.483Z", "content": "Thank you for further elaborating on the dataset! You clearly did a great job explaining these complex concepts 👍, while we might have used a bit too much jargon ourselves. 😅 Please let us know if you have any other questions about the data or the FWI problem itself.
", "rawMarkdown": "Thank you for further elaborating on the dataset! You clearly did a great job explaining these complex concepts 👍, while we might have used a bit too much jargon ourselves. 😅 Please let us know if you have any other questions about the data or the FWI problem itself.", "votes": 3, "replies": [ { "id": 3178002, "postDate": "2025-04-13T16:09:28.427Z", "content": "Hi @youzuolin / @hanchenwang114 is there any significance to the batch dimension? For example, if we take index=0 and index=1 from the same seismic data file, are these more closely related than data points from different files?
\n= np.load()\n= np.load()\n\n\n= arr1[, ...]\n= arr1[, ...]\n\n\n= arr1[, ...]\n= arr2[, ...]\nThere is no specific correlation between nearby indexed samples from the same file, meaning index=0 and index=1 from the same seismic data file are not more similar to each other compared to other far away indexed samples.
\nHowever, I would like to point out that for the \"Fault\" family (like the two files loaded in your code snippet), \"The naming of files can be described as {vel|seis}_{n}_1_{i}.npy, where vel and seis specify if a file includes velocity maps or seismic data, n represents the number of initial flatten layers for velocity maps generation and i is the index of a file (start from 0) among the ones with the same n.\", which can be found in appendix section D.2. in the OpenFWI paper: https://arxiv.org/pdf/2111.02926
\nIn short, there is no similarity difference between nearby or far away sample indices. \"Fault\" family file names indicate the complexity of the data/velocity maps, the larger number, the more complicated.
", "rawMarkdown": "There is no specific correlation between nearby indexed samples from the same file, meaning index=0 and index=1 from the same seismic data file are not more similar to each other compared to other far away indexed samples. \n\nHowever, I would like to point out that for the \"Fault\" family (like the two files loaded in your code snippet), \"The naming of files can be described as {vel\\|seis}\\_{n}\\_1\\_{i}.npy, where vel and seis specify if a file includes velocity maps or seismic data, n represents the number of initial flatten layers for velocity maps generation and i is the index of a file (start from 0) among the ones with the same n.\", which can be found in appendix section D.2. in the OpenFWI paper: https://arxiv.org/pdf/2111.02926\n\nIn short, there is no similarity difference between nearby or far away sample indices. \"Fault\" family file names indicate the complexity of the data/velocity maps, the larger number, the more complicated.", "votes": 5 } ] } ] }, { "id": 3180077, "postDate": "2025-04-16T05:20:50Z", "content": "I appreciate you explaining the dataset so well.🤟
", "rawMarkdown": "I appreciate you explaining the dataset so well.🤟", "votes": 4 }, { "id": 3179710, "postDate": "2025-04-15T16:02:27.473Z", "content": "Thanks for sharing it this I appreciate it.
", "rawMarkdown": "Thanks for sharing it this I appreciate it.", "votes": 2 }, { "id": 3207741, "postDate": "2025-05-23T07:21:17.180Z", "content": "Have a better understanding, appreciate it so much!
", "rawMarkdown": "Have a better understanding, appreciate it so much!", "votes": 2 }, { "id": 3179199, "postDate": "2025-04-15T06:36:51.450Z", "content": "What are the actual np array shapes in practice, if opening all files and just printing the shape?
\nObviously always 500 batch. Is velocity map always 70x70? What's the min/max/avg for seismic sources, time steps, and receivers?
", "rawMarkdown": "What are the actual np array shapes in practice, if opening all files and just printing the shape?\n\nObviously always 500 batch. Is velocity map always 70x70? What's the min/max/avg for seismic sources, time steps, and receivers?", "votes": 1, "replies": [ { "id": 3179571, "postDate": "2025-04-15T13:57:33.943Z", "content": "Update: yes, the shapes are all consistent and exactly as the data description above describes. :)
\nOld Response: This is an EDA question not a data description question. Create a notebook and explore and share your results with us!
", "rawMarkdown": "Update: yes, the shapes are all consistent and exactly as the data description above describes. :)\n\nOld Response: This is an EDA question not a data description question. Create a notebook and explore and share your results with us!", "votes": 3, "replies": [ { "id": 3179605, "postDate": "2025-04-15T14:25:31.363Z", "content": "That's fair, although this is a great place to post the summary results of such EDA. So I'll still ask and see if anyone answers.
\nI think the \"is velocity map always 500x70x70?\" question belongs in this thread though. The data description implies it varies, but the competition format and my preconceived notions implies the opposite. Do you know the answer?
", "rawMarkdown": "That's fair, although this is a great place to post the summary results of such EDA. So I'll still ask and see if anyone answers. \n\nI think the \"is velocity map always 500x70x70?\" question belongs in this thread though. The data description implies it varies, but the competition format and my preconceived notions implies the opposite. Do you know the answer?", "votes": 2 }, { "id": 3179611, "postDate": "2025-04-15T14:32:21.013Z", "content": "Oh right, the question came from this data description. It is ambiguous. First half has an implied \"always\", and states that height and width are 70, but second half has an \"if\".
\nSo it's a data description question.
\n\n", "rawMarkdown": "Oh right, the question came from this data description. It is ambiguous. First half has an implied \"always\", and states that height and width are 70, but second half has an \"if\". \n\nSo it's a data description question. \n\n>Velocity map is in 3 dimensions:\nThe “batch” dimension (500) for the number of samples.\nThe height (70 grid points from top to bottom).\nThe width (70 grid points horizontally).\nSo if the shape is (500, 70, 70), that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.", "votes": 2 }, { "id": 3179638, "postDate": "2025-04-15T14:53:14.447Z", "content": "Velocity map is in 3 dimensions:
\n
\n The “batch” dimension (500) for the number of samples.
\n The height (70 grid points from top to bottom).
\n The width (70 grid points horizontally).
\n So if the shape is (500, 70, 70), that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.
Ah, I see - The \"if\" was only used rhetorically to clarify the meaning of the structure, not to suggest any conditional variability :). And WHOOPS - I accidentally clicked the downvote - I changed it immediately to an upvote, sorry if you got a notification about that!
", "rawMarkdown": "Ah, I see - The \"if\" was only used rhetorically to clarify the meaning of the structure, not to suggest any conditional variability :). And WHOOPS - I accidentally clicked the downvote - I changed it immediately to an upvote, sorry if you got a notification about that!", "votes": 2 } ] } ] }, { "id": 3186568, "postDate": "2025-04-24T21:30:24.687Z", "content": "Thanks for the valuable EDA. Could you please elaborate on the 500 seismic subsurface samples.What parameter is changed in every senario? Wave frequence?
", "rawMarkdown": "Thanks for the valuable EDA. Could you please elaborate on the 500 seismic subsurface samples.What parameter is changed in every senario? Wave frequence?", "votes": 2, "replies": [ { "id": 3186578, "postDate": "2025-04-24T22:04:03.510Z", "content": "Each of the 500 seismic samples is essentially a different \"geological scenario\" -- You can think of them like they are each their own data point on their own. So in CurveFault - they are 500 curve fault scenarios. Think of them like separate data points that are all curve fault scenarios. They've simply \"grouped them\" into families with 500 different scenarios in each.
\nI'm going to update my EDA and background further with visual examples on all this. I'm in the process of making many more notebooks that clarify all these things. Just need a little more time. :)
", "rawMarkdown": "Each of the 500 seismic samples is essentially a different \"geological scenario\" -- You can think of them like they are each their own data point on their own. So in CurveFault - they are 500 curve fault scenarios. Think of them like separate data points that are all curve fault scenarios. They've simply \"grouped them\" into families with 500 different scenarios in each.\n\nI'm going to update my EDA and background further with visual examples on all this. I'm in the process of making many more notebooks that clarify all these things. Just need a little more time. :)", "votes": 4 } ] }, { "id": 3185970, "postDate": "2025-04-24T04:07:30.197Z", "content": "😃🥰6666666
", "rawMarkdown": "😃🥰6666666", "votes": 2 }, { "id": 3188939, "postDate": "2025-04-28T13:54:21.210Z", "content": "Do we know the locations of the sources and receivers?
", "rawMarkdown": "Do we know the locations of the sources and receivers? " }, { "id": 3185609, "postDate": "2025-04-23T15:20:22.087Z", "content": "Thanks for providing valuable and authentic information of the dataset. I was having troubles understanding complex concepts. This explanation gave me a head start. 🚀
", "rawMarkdown": "Thanks for providing valuable and authentic information of the dataset. I was having troubles understanding complex concepts. This explanation gave me a head start. 🚀 ", "votes": 2 }, { "id": 3179646, "postDate": "2025-04-15T14:58:47.687Z", "content": "I‘m wondering how to use physic way to solve it.
", "rawMarkdown": "I‘m wondering how to use physic way to solve it.", "votes": 2, "replies": [ { "id": 3179792, "postDate": "2025-04-15T17:46:16.400Z", "content": "There are too many physics-based FWI papers. I will recommend this paper as a good starting point: Virieux J, Operto S. An overview of full-waveform inversion in exploration geophysics. Geophysics. 2009 Nov;74(6):WCC1-26.
", "rawMarkdown": "There are too many physics-based FWI papers. I will recommend this paper as a good starting point: Virieux J, Operto S. An overview of full-waveform inversion in exploration geophysics. Geophysics. 2009 Nov;74(6):WCC1-26.", "votes": 5 }, { "id": 3188076, "postDate": "2025-04-27T02:38:47.243Z", "content": "If you are talking about the ML approach, you can incorporate physics directly into your neural network in the loss function. Instead of only learning statistical patterns that reduce MAE or MSE without being actually physically plausible structures (they'll produce blurs and strange discontinuities to \"hack\" the metric but won't make velocity maps that could be real), Physics Informed Neural Networks (PINNs) explicitly use the underlying physical equations to determine the physical \"realism\" of the prediction. Here's how it works:
\nPredict the Velocity Map:
\nCompute Physics-Based Loss:
\nEnsure Physical Plausibility:
\nIntegrating physics directly into the training process helps your neural network generate models consistent with real-world physics, enhancing both interpretability and reliability.
\nI'm currently working on a detailed kernel demonstrating this method, which I'll publish soon. Many others in the community are also actively exploring and refining these methods.
", "rawMarkdown": "If you are talking about the ML approach, you can incorporate physics directly into your neural network in the loss function. Instead of only learning statistical patterns that reduce MAE or MSE without being actually physically plausible structures (they'll produce blurs and strange discontinuities to \"hack\" the metric but won't make velocity maps that could be real), Physics Informed Neural Networks (PINNs) explicitly use the underlying physical equations to determine the physical \"realism\" of the prediction. Here's how it works:\n\n1. **Predict the Velocity Map:**\n - Your neural network first predicts a velocity model based on input seismic data.\n - You then apply the real physics-based wave equations to this predicted velocity map to **go back to the seismic data that velocity map would generate**.\n\n2. **Compute Physics-Based Loss:**\n - You calculate the loss by comparing this physics-simulated seismic data to your ground truth seismic data.\n - This loss measures how physically realistic your predicted velocity map is by checking how closely it reproduces the observed seismic data.\n\n3. **Ensure Physical Plausibility:**\n - Minimizing this physics-informed loss helps ensure your velocity maps remain physically realistic and geologically meaningful, rather than just fitting purely statistical metrics like MAE or MSE, which might otherwise produce unrealistic solutions.\n\nIntegrating physics directly into the training process helps your neural network generate models consistent with real-world physics, enhancing both interpretability and reliability.\n\nI'm currently working on a detailed kernel demonstrating this method, which I'll publish soon. Many others in the community are also actively exploring and refining these methods.\n\n", "votes": 4, "replies": [ { "id": 3197356, "postDate": "2025-05-08T05:21:16.467Z", "content": "Have you made advancements in this direction? I tried to create a Torch loss funciona that reproduces the seismic data, but it is too slow to actually be used in training.
", "rawMarkdown": "Have you made advancements in this direction? I tried to create a Torch loss funciona that reproduces the seismic data, but it is too slow to actually be used in training.", "votes": 2 }, { "id": 3197753, "postDate": "2025-05-08T14:29:15.817Z", "content": "I haven't yet. :) Still experimenting with tons of neural network architectures and hyperparameters. I'm planning on using a PINN in an ensemble if it helps but only if it actually helps. The other models I'm using are quite good so I'm not sure how much I'll end up investing in that direction.
\nHave you tried optimizing your function or seeing if you can get 90% of the accuracy you need while making it more efficient?
", "rawMarkdown": "I haven't yet. :) Still experimenting with tons of neural network architectures and hyperparameters. I'm planning on using a PINN in an ensemble if it helps but only if it actually helps. The other models I'm using are quite good so I'm not sure how much I'll end up investing in that direction.\n\nHave you tried optimizing your function or seeing if you can get 90% of the accuracy you need while making it more efficient?" }, { "id": 3197809, "postDate": "2025-05-08T16:02:48.997Z", "content": "I have a torch version of the code presented here, but although it is already vectorized as much as possible (I think), it is still to slow to be actually usable (it takes a couple of minutes to calculate a batch of size 32 on a P100 Kaggle GPU).
\nEDIT: here is my code.
", "rawMarkdown": "I have a torch version of the code presented [here](https://www.kaggle.com/code/jaewook704/waveform-inversion-vel-to-seis), but although it is already vectorized as much as possible (I think), it is still to slow to be actually usable (it takes a couple of minutes to calculate a batch of size 32 on a P100 Kaggle GPU).\n\nEDIT: [here](https://www.kaggle.com/code/fpeccia/pytorch-forward-propagation-loss-function) is my code.", "votes": 1 }, { "id": 3197821, "postDate": "2025-05-08T16:33:45.220Z", "content": "I ran your code through a chatGPT deep research query asking how it could be more efficient. Here is the output:
\n--> https://chatgpt.com/share/681cdc54-eb08-8008-8af9-7bcf7e3ed6c4
\n--> AI Podcast of this output: https://notebooklm.google.com/notebook/ebc4af11-51aa-40aa-a9ff-042abd3db34e/audio
Hopefully this helps get the ideas flowing
", "rawMarkdown": "I ran your code through a chatGPT deep research query asking how it could be more efficient. Here is the output:\n\n--> https://chatgpt.com/share/681cdc54-eb08-8008-8af9-7bcf7e3ed6c4\n--> AI Podcast of this output: https://notebooklm.google.com/notebook/ebc4af11-51aa-40aa-a9ff-042abd3db34e/audio\n\nHopefully this helps get the ideas flowing", "votes": 1 }, { "id": 3197909, "postDate": "2025-05-08T18:57:11.903Z", "content": "Thanks! I had already done something similar, but there are still intriguing paths to follow to try to me it more performant.
", "rawMarkdown": "Thanks! I had already done something similar, but there are still intriguing paths to follow to try to me it more performant.", "votes": 1 } ] } ] }, { "id": 3176707, "postDate": "2025-04-11T15:49:19.473Z", "content": "I can only take credit for my confusion, ignorance, inexperience, and good prompting ;) - AI helped explain the rest.
", "rawMarkdown": "I can only take credit for my confusion, ignorance, inexperience, and good prompting ;) - AI helped explain the rest. ", "votes": 2 }, { "id": 3176574, "postDate": "2025-04-11T13:47:58.400Z", "content": "Thanks for taking the time to explain the concept.
", "rawMarkdown": "Thanks for taking the time to explain the concept. ", "votes": 2 }, { "id": 3230837, "postDate": "2025-06-23T14:08:00.033Z", "content": "Have a better comprehension thank to your amazing explanation. Keep up this great work.
", "rawMarkdown": "Have a better comprehension thank to your amazing explanation. Keep up this great work." }, { "id": 3218714, "postDate": "2025-06-06T15:52:30.437Z", "content": "Thanks for giving me some head-starts! Normally, the problems in machine learning are some types of tabular, image, audio or time-series, but this problem is hard as heck.
", "rawMarkdown": "Thanks for giving me some head-starts! Normally, the problems in machine learning are some types of tabular, image, audio or time-series, but this problem is hard as heck." }, { "id": 3194382, "postDate": "2025-05-05T19:41:26.097Z", "content": "Thank you for this great explanation!
\nThings have become clearer now.
Thank you for taking the time to write this clear explanation.
", "rawMarkdown": "Thank you for taking the time to write this clear explanation.", "votes": 1, "isDeleted": true }, { "id": 3193802, "postDate": "2025-05-05T03:52:06.667Z", "content": "Thank you. This is really helpful!
", "rawMarkdown": "Thank you. This is really helpful!", "votes": 1 }, { "id": 3186651, "postDate": "2025-04-25T02:07:48.897Z", "content": "what a good explanation, thanks!
", "rawMarkdown": "what a good explanation, thanks!", "votes": 1 }, { "id": 3178278, "postDate": "2025-04-14T01:36:51.963Z", "content": "Well explained thanks ****
", "rawMarkdown": "Well explained thanks ****", "votes": 1 } ], "comments": [ { "id": 3187488, "author_name": "", "author_url": "", "post_date": "2025-04-26T05:14:29.447000", "content": "Great insights!! thanks a lot for the spelled out explanation, I needed this. I joined this comp a few days earlier and I was stuck on what those 100 gigs of data were. This discussion really cleared my confusions.
", "votes": 3, "replies": [] }, { "id": 3182840, "author_name": "OV104", "author_url": "", "post_date": "2025-04-19T23:18:54.860000", "content": "Thank you for providing valuable information. I had difficulty understanding the dimensionality of the seismic data, but your help was appreciated.
", "votes": 3, "replies": [ { "id": 3182867, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-20T01:28:36.253000", "content": "Happy it helped!
", "votes": 2, "replies": [] } ] }, { "id": 3176655, "author_name": "Youzuo Lin", "author_url": "", "post_date": "2025-04-11T15:06:35.483000", "content": "Thank you for further elaborating on the dataset! You clearly did a great job explaining these complex concepts 👍, while we might have used a bit too much jargon ourselves. 😅 Please let us know if you have any other questions about the data or the FWI problem itself.
", "votes": 3, "replies": [ { "id": 3178002, "author_name": "Bartley", "author_url": "", "post_date": "2025-04-13T16:09:28.427000", "content": "Hi @youzuolin / @hanchenwang114 is there any significance to the batch dimension? For example, if we take index=0 and index=1 from the same seismic data file, are these more closely related than data points from different files?
\n= np.load()\n= np.load()\n\n\n= arr1[, ...]\n= arr1[, ...]\n\n\n= arr1[, ...]\n= arr2[, ...]\nThere is no specific correlation between nearby indexed samples from the same file, meaning index=0 and index=1 from the same seismic data file are not more similar to each other compared to other far away indexed samples.
\nHowever, I would like to point out that for the \"Fault\" family (like the two files loaded in your code snippet), \"The naming of files can be described as {vel|seis}_{n}_1_{i}.npy, where vel and seis specify if a file includes velocity maps or seismic data, n represents the number of initial flatten layers for velocity maps generation and i is the index of a file (start from 0) among the ones with the same n.\", which can be found in appendix section D.2. in the OpenFWI paper: https://arxiv.org/pdf/2111.02926
\nIn short, there is no similarity difference between nearby or far away sample indices. \"Fault\" family file names indicate the complexity of the data/velocity maps, the larger number, the more complicated.
", "votes": 5, "replies": [] } ] } ] }, { "id": 3180077, "author_name": "Jehad Ur Rahman Khan", "author_url": "", "post_date": "2025-04-16T05:20:50", "content": "I appreciate you explaining the dataset so well.🤟
", "votes": 4, "replies": [] }, { "id": 3179710, "author_name": "", "author_url": "", "post_date": "2025-04-15T16:02:27.473000", "content": "Thanks for sharing it this I appreciate it.
", "votes": 2, "replies": [] }, { "id": 3207741, "author_name": "", "author_url": "", "post_date": "2025-05-23T07:21:17.180000", "content": "Have a better understanding, appreciate it so much!
", "votes": 2, "replies": [] }, { "id": 3179199, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-04-15T06:36:51.450000", "content": "What are the actual np array shapes in practice, if opening all files and just printing the shape?
\nObviously always 500 batch. Is velocity map always 70x70? What's the min/max/avg for seismic sources, time steps, and receivers?
", "votes": 1, "replies": [ { "id": 3179571, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-15T13:57:33.943000", "content": "Update: yes, the shapes are all consistent and exactly as the data description above describes. :)
\nOld Response: This is an EDA question not a data description question. Create a notebook and explore and share your results with us!
", "votes": 3, "replies": [ { "id": 3179605, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-04-15T14:25:31.363000", "content": "That's fair, although this is a great place to post the summary results of such EDA. So I'll still ask and see if anyone answers.
\nI think the \"is velocity map always 500x70x70?\" question belongs in this thread though. The data description implies it varies, but the competition format and my preconceived notions implies the opposite. Do you know the answer?
", "votes": 2, "replies": [] }, { "id": 3179611, "author_name": "Robert Hatch", "author_url": "", "post_date": "2025-04-15T14:32:21.013000", "content": "Oh right, the question came from this data description. It is ambiguous. First half has an implied \"always\", and states that height and width are 70, but second half has an \"if\".
\nSo it's a data description question.
\n\n", "votes": 2, "replies": [] }, { "id": 3179638, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-15T14:53:14.447000", "content": "Velocity map is in 3 dimensions:
\n
\n The “batch” dimension (500) for the number of samples.
\n The height (70 grid points from top to bottom).
\n The width (70 grid points horizontally).
\n So if the shape is (500, 70, 70), that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.
Ah, I see - The \"if\" was only used rhetorically to clarify the meaning of the structure, not to suggest any conditional variability :). And WHOOPS - I accidentally clicked the downvote - I changed it immediately to an upvote, sorry if you got a notification about that!
", "votes": 2, "replies": [] } ] } ] }, { "id": 3186568, "author_name": "Evdilos_Ikaria", "author_url": "", "post_date": "2025-04-24T21:30:24.687000", "content": "Thanks for the valuable EDA. Could you please elaborate on the 500 seismic subsurface samples.What parameter is changed in every senario? Wave frequence?
", "votes": 2, "replies": [ { "id": 3186578, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-24T22:04:03.510000", "content": "Each of the 500 seismic samples is essentially a different \"geological scenario\" -- You can think of them like they are each their own data point on their own. So in CurveFault - they are 500 curve fault scenarios. Think of them like separate data points that are all curve fault scenarios. They've simply \"grouped them\" into families with 500 different scenarios in each.
\nI'm going to update my EDA and background further with visual examples on all this. I'm in the process of making many more notebooks that clarify all these things. Just need a little more time. :)
", "votes": 4, "replies": [] } ] }, { "id": 3185970, "author_name": "大帅哥沈翰", "author_url": "", "post_date": "2025-04-24T04:07:30.197000", "content": "😃🥰6666666
", "votes": 2, "replies": [] }, { "id": 3188939, "author_name": "Shenwen Yu", "author_url": "", "post_date": "2025-04-28T13:54:21.210000", "content": "Do we know the locations of the sources and receivers?
", "votes": 0, "replies": [] }, { "id": 3185609, "author_name": "Hansal Kothari", "author_url": "", "post_date": "2025-04-23T15:20:22.087000", "content": "Thanks for providing valuable and authentic information of the dataset. I was having troubles understanding complex concepts. This explanation gave me a head start. 🚀
", "votes": 2, "replies": [] }, { "id": 3179646, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2025-04-15T14:58:47.687000", "content": "I‘m wondering how to use physic way to solve it.
", "votes": 2, "replies": [ { "id": 3179792, "author_name": "Hanchen Wang", "author_url": "", "post_date": "2025-04-15T17:46:16.400000", "content": "There are too many physics-based FWI papers. I will recommend this paper as a good starting point: Virieux J, Operto S. An overview of full-waveform inversion in exploration geophysics. Geophysics. 2009 Nov;74(6):WCC1-26.
", "votes": 5, "replies": [] }, { "id": 3188076, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-27T02:38:47.243000", "content": "If you are talking about the ML approach, you can incorporate physics directly into your neural network in the loss function. Instead of only learning statistical patterns that reduce MAE or MSE without being actually physically plausible structures (they'll produce blurs and strange discontinuities to \"hack\" the metric but won't make velocity maps that could be real), Physics Informed Neural Networks (PINNs) explicitly use the underlying physical equations to determine the physical \"realism\" of the prediction. Here's how it works:
\nPredict the Velocity Map:
\nCompute Physics-Based Loss:
\nEnsure Physical Plausibility:
\nIntegrating physics directly into the training process helps your neural network generate models consistent with real-world physics, enhancing both interpretability and reliability.
\nI'm currently working on a detailed kernel demonstrating this method, which I'll publish soon. Many others in the community are also actively exploring and refining these methods.
", "votes": 4, "replies": [ { "id": 3197356, "author_name": "Federico Peccia", "author_url": "", "post_date": "2025-05-08T05:21:16.467000", "content": "Have you made advancements in this direction? I tried to create a Torch loss funciona that reproduces the seismic data, but it is too slow to actually be used in training.
", "votes": 2, "replies": [] }, { "id": 3197753, "author_name": "Tom M", "author_url": "", "post_date": "2025-05-08T14:29:15.817000", "content": "I haven't yet. :) Still experimenting with tons of neural network architectures and hyperparameters. I'm planning on using a PINN in an ensemble if it helps but only if it actually helps. The other models I'm using are quite good so I'm not sure how much I'll end up investing in that direction.
\nHave you tried optimizing your function or seeing if you can get 90% of the accuracy you need while making it more efficient?
", "votes": 0, "replies": [] }, { "id": 3197809, "author_name": "Federico Peccia", "author_url": "", "post_date": "2025-05-08T16:02:48.997000", "content": "I have a torch version of the code presented here, but although it is already vectorized as much as possible (I think), it is still to slow to be actually usable (it takes a couple of minutes to calculate a batch of size 32 on a P100 Kaggle GPU).
\nEDIT: here is my code.
", "votes": 1, "replies": [] }, { "id": 3197821, "author_name": "Tom M", "author_url": "", "post_date": "2025-05-08T16:33:45.220000", "content": "I ran your code through a chatGPT deep research query asking how it could be more efficient. Here is the output:
\n--> https://chatgpt.com/share/681cdc54-eb08-8008-8af9-7bcf7e3ed6c4
\n--> AI Podcast of this output: https://notebooklm.google.com/notebook/ebc4af11-51aa-40aa-a9ff-042abd3db34e/audio
Hopefully this helps get the ideas flowing
", "votes": 1, "replies": [] }, { "id": 3197909, "author_name": "Federico Peccia", "author_url": "", "post_date": "2025-05-08T18:57:11.903000", "content": "Thanks! I had already done something similar, but there are still intriguing paths to follow to try to me it more performant.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3176707, "author_name": "Tom M", "author_url": "", "post_date": "2025-04-11T15:49:19.473000", "content": "I can only take credit for my confusion, ignorance, inexperience, and good prompting ;) - AI helped explain the rest.
", "votes": 2, "replies": [] }, { "id": 3176574, "author_name": "Sandy", "author_url": "", "post_date": "2025-04-11T13:47:58.400000", "content": "Thanks for taking the time to explain the concept.
", "votes": 2, "replies": [] }, { "id": 3230837, "author_name": "Toussaint", "author_url": "", "post_date": "2025-06-23T14:08:00.033000", "content": "Have a better comprehension thank to your amazing explanation. Keep up this great work.
", "votes": 0, "replies": [] }, { "id": 3218714, "author_name": "xplore", "author_url": "", "post_date": "2025-06-06T15:52:30.437000", "content": "Thanks for giving me some head-starts! Normally, the problems in machine learning are some types of tabular, image, audio or time-series, but this problem is hard as heck.
", "votes": 0, "replies": [] }, { "id": 3194382, "author_name": "", "author_url": "", "post_date": "2025-05-05T19:41:26.097000", "content": "Thank you for this great explanation!
\nThings have become clearer now.
Thank you for taking the time to write this clear explanation.
", "votes": 1, "replies": [] }, { "id": 3193802, "author_name": "Spectacle", "author_url": "", "post_date": "2025-05-05T03:52:06.667000", "content": "Thank you. This is really helpful!
", "votes": 1, "replies": [] }, { "id": 3186651, "author_name": "kaiseionishi", "author_url": "", "post_date": "2025-04-25T02:07:48.897000", "content": "what a good explanation, thanks!
", "votes": 1, "replies": [] }, { "id": 3178278, "author_name": "Memoona Qaiser", "author_url": "", "post_date": "2025-04-14T01:36:51.963000", "content": "Well explained thanks ****
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "3176264": "I had a hard time understanding what was going on with the data. Here's a potentially more basic explanation for those new to the domain (like me).\n\n---\n\n## 1. What are we even talking about? \nWe’re dealing with a **seismic imaging** problem. In seismic imaging, energy waves are sent into the ground (from sources like vibroseis trucks or air guns), and then the signals that bounce back are recorded by receivers (geophones or hydrophones). We want to figure out how fast those waves travel at different points below the surface—this speed is called the **subsurface velocity**. \n\nNow, imagine we cut the Earth in half so we can look at it from the side. Along that vertical slice, each point in that 2D cross-section has some speed (velocity) with which sound waves (seismic waves) travel. This 2D arrangement of speeds is called a **velocity map**. Meanwhile, up on the surface, we have instruments recording the wave signals over time, which we call **seismic data**. \n\nSo the fundamental pair is: \n1. **Seismic data**: all the waveforms recorded over time. \n2. **Velocity map**: a 2D “picture” of how speed varies with depth and horizontal position.\n\n---\n\n## 2. Why are there three “families”? \n\n**Short answer**: each “family” is a different *type* of 2D velocity cross-section (and corresponding seismic data) that was artificially generated to represent distinct geological scenarios. \n\n1. **Vel Family** \n - **What it looks like:** Think of relatively orderly, layered sediments. “Vel” stands for “velocity,” but effectively this family has simpler layered structures (like a cake with flat or gently curved layers). \n - **Why it’s simpler:** The layers might be horizontal (FlatVel) or smoothly dipping/curving (CurveVel), but no major breaks in the rock. \n - **What the seismic data show:** You typically get clear, continuous reflection events. \n\n2. **Fault Family** \n - **What it looks like:** Same idea of layered geology, *but* now there are faults—places where the layers are cut and shifted relative to each other. \n - **Why it’s more complex:** A fault is basically a break in the Earth where one side of a layer is displaced relative to the other. This adds complexity. \n - **What the seismic data show:** Discontinuous reflections, sudden jumps, or “diffractions” at the fault lines.\n\n3. **Style Family** \n - **What it looks like:** Not your typical “layer cake.” Instead, it’s more chaotic or irregular shapes, because the creators used “style transfer” from arbitrary images to generate velocity patterns. Imagine velocity maps that might have swirls, blobs, or textures—no clean layering. \n - **Why it’s the most unpredictable:** It can mimic random or exotic geologies. \n - **What the seismic data show:** More scattered wave energy, less straightforward layering.\n\nEssentially, these three “families” cover a spectrum of geological complexity: \n- **Vel**: simpler stratified layers \n- **Fault**: layered but with big breaks (faults) \n- **Style**: all sorts of random or organic shapes\n\n---\n\n## 3. So, these are synthetic or real? \nThey’re **synthetic** datasets. That means people used a physics simulator to generate the seismic data *given* an artificially created velocity map. This approach ensures we have a “true velocity map” for every sample, which is extremely valuable for training or testing machine learning algorithms, because real field data usually doesn’t have perfect “ground truth.” \n\n---\n\n## 4. Why do we need so many subfolders? \nEach family can have sub-variations: \n- “A” vs. “B” versions: typically “A” is somewhat simpler (fewer random variations, fewer layers, gentler changes), while “B” is more complex (more layers, more variability). \n- “Flat” vs. “Curve” (for Vel or Fault): indicates whether layers are relatively flat or curved/folded. \n\nFor example, **FlatVel_A** means the simplest version of layered velocity with almost no major complexity. **CurveVel_B** means a more complex version of layered velocity with significant curving/folding. **FlatFault_A** means a simpler fault scenario; **CurveFault_B** means a complex fault scenario with curved layers. And so on.\n\n---\n\n## 5. What’s actually in each folder? \nIn each subfolder, you see `.npy` files (NumPy format) containing:\n- **Seismic data** — the time series recordings at the surface. Each seismic `.npy` file holds a batch of 500 samples. \n- **Velocity maps** — the 2D grid of wave speeds for those same 500 samples.\n\nConcretely, if you open, say, **FlatVel_B**, you might see something like: \n```\nFlatVel_B/\n data/\n data1.npy <-- 500 seismic samples\n data2.npy <-- another 500 seismic samples\n model/\n model1.npy <-- 500 velocity maps\n model2.npy <-- another 500 velocity maps\n```\nThe index in the file (like `data1.npy[i]` and `model1.npy[i]`) lines up the i-th seismic sample with the i-th velocity map.\n\nFor the **Fault** family, it’s the same pairing but with filenames like `seis4_1_0.npy` (seismic) and `vel4_1_0.npy` (velocity). Each still has 500 paired samples.\n\n---\n\n## 6. Shape of the Data: 4D vs. 3D \n- **Seismic data** is in 4 dimensions: \n 1. The “batch” dimension (500) for the number of samples. \n 2. The number of sources (like 5 different shot points). \n 3. The number of time steps (like 1000 samples in time). \n 4. The number of receivers (like 70 recording positions). \n\n So if the shape is `(500, 5, 1000, 70)`, that means: \n - 500 different subsurface scenarios \n - Each scenario has 5 seismic sources \n - Recorded for 1000 time steps \n - At 70 receiver positions.\n\n- **Velocity map** is in 3 dimensions:\n 1. The “batch” dimension (500) for the number of samples. \n 2. The height (70 grid points from top to bottom). \n 3. The width (70 grid points horizontally). \n\n So if the shape is `(500, 70, 70)`, that means each of the 500 scenarios has a 70×70 “image” representing velocity in the ground.\n\n---\n\n## 7. The Competition Setup \n- You have a **training** folder with many `.npy` files (like `data1.npy`/`model1.npy` pairs). This is what you can use to train or test your approach. \n- You also get a **test** folder that only has the seismic data (no velocity). The competition wants you to predict the velocity maps for those unknown examples. \n- You submit your predictions in a particular CSV or NumPy format so the organizers can compare them to the real velocity maps (which they keep hidden for evaluation).\n\nEssentially, the competition is: “Given 3D seismic waveforms, can you predict the 2D velocity cross-section for each sample?” \n\n---\n\n## 8. Why does it matter? \nIn real life, if you record seismic data in the field, you don’t automatically know the exact velocity structure underground (and that’s often what you want to find). Having synthetic data with known “answers” (velocity maps) helps us benchmark algorithms. The final step is to see who can do the best “inversion” from seismic to velocity.\n\n---\n\n## Recap in Super-Simple Terms\n\n1. **3 Families**: \n - **Vel** = simpler, layered Earth. \n - **Fault** = layered Earth but broken by faults. \n - **Style** = more random, weird patterns. \n2. **Each family** has subfolders (e.g., FlatVel_A, CurveVel_B) indicating more or less complexity. \n3. **Inside each subfolder** are `.npy` files holding (a) seismic data and (b) velocity maps. Each `.npy` holds 500 “examples” (pairs). \n4. **The competition**: train a model on these known pairs to learn to predict velocity from seismic, then apply it to new test seismic data.\n\nThat’s the entire structure in a nutshell. Each folder is basically a chunk of data with 500 training examples. The difference is just how the underground geology was generated (flat layers, curved layers, faults, or random images). The ultimate goal is to see if your method can handle them all!\n\n# Next Steps\n\n## 👉 Literature Review + Papers with Code + Background\n>**If you want a deeper dive into the background of the problem, AI generated podcasts on this topic, and more, check out the literature review and concept background notebook I've made here**\n👉 https://www.kaggle.com/code/tpmeli/geo-lit-review-winning-strategies-starter\n\n## 👉 Exploratory Data Analysis (EDA)\n> **If you're looking for a comprehensive exploration of geographic and WFI data insights, including detailed visualizations, statistical analyses, and key observations, check out the Exploratory Deep Dive notebook I've prepared here:**\n👉 https://www.kaggle.com/code/tpmeli/exploratory-deep-dive-geo-wfi-data-insights\n", "3187488": "Great insights!! thanks a lot for the spelled out explanation, I needed this. I joined this comp a few days earlier and I was stuck on what those 100 gigs of data were. This discussion really cleared my confusions. ", "3182840": "Thank you for providing valuable information. I had difficulty understanding the dimensionality of the seismic data, but your help was appreciated.", "3176655": "Thank you for further elaborating on the dataset! You clearly did a great job explaining these complex concepts 👍, while we might have used a bit too much jargon ourselves. 😅 Please let us know if you have any other questions about the data or the FWI problem itself.", "3180077": "I appreciate you explaining the dataset so well.🤟", "3179710": "Thanks for sharing it this I appreciate it.", "3207741": "Have a better understanding, appreciate it so much!", "3179199": "What are the actual np array shapes in practice, if opening all files and just printing the shape?\n\nObviously always 500 batch. Is velocity map always 70x70? What's the min/max/avg for seismic sources, time steps, and receivers?", "3186568": "Thanks for the valuable EDA. Could you please elaborate on the 500 seismic subsurface samples.What parameter is changed in every senario? Wave frequence?", "3185970": "😃🥰6666666", "3188939": "Do we know the locations of the sources and receivers? ", "3185609": "Thanks for providing valuable and authentic information of the dataset. I was having troubles understanding complex concepts. This explanation gave me a head start. 🚀 ", "3179646": "I‘m wondering how to use physic way to solve it.", "3176707": "I can only take credit for my confusion, ignorance, inexperience, and good prompting ;) - AI helped explain the rest. ", "3176574": "Thanks for taking the time to explain the concept. ", "3230837": "Have a better comprehension thank to your amazing explanation. Keep up this great work.", "3218714": "Thanks for giving me some head-starts! Normally, the problems in machine learning are some types of tabular, image, audio or time-series, but this problem is hard as heck.", "3194382": "Thank you for this great explanation! \nThings have become clearer now.", "3187994": "Thank you for taking the time to write this clear explanation.", "3193802": "Thank you. This is really helpful!", "3186651": "what a good explanation, thanks!", "3178278": "Well explained thanks ****" } }