{
  "id": 587506,
  "title": "Tips to make forward model efficient",
  "url": "/competitions/waveform-inversion/discussion/587506",
  "author_name": "Bilzard",
  "post_date": "2025-07-01T09:41:58.585000",
  "votes": 6,
  "comment_count": 25,
  "views": 0,
  "content": "<p>I tried much effort to make forward model as efficient as possible.<br>\nHowever, it seemed I cannot drastic computation improvement comparing to O(n_t * n_z * nx).<br>\nSo let's share the useful tips to make forward model efficient.</p>\n<p>For example, I used these tips:</p>\n<ul>\n<li>convert rolling to conv5x5</li>\n<li>convert conv 5x5 to: FFT-&gt;element-wise multiplication-&gt;inverse FFT</li>\n</ul>\n<p><strong>implementation detail</strong>:</p>\n<p>This approach uses the fact that <strong>convolution with circular padding is equivalent to item-wise multiplication in the frequency domain</strong>.<br>\nTheoretically,</p>\n<p>$$<br>\n\\text{FFT}(\\text{Conv}(w(x, y), f(x, y))) = \\text{FFT}(w(x, y)) * \\text{FFT}(f(x, y))<br>\n$$</p>\n<pre><code>     ():\n            ....\n            \n            kernel5 = .build_kernel()\n            full = torch.zeros(nzbc, nxbc, dtype=dtype)\n            full[:, :] = kernel5\n            full = torch.roll(full, shifts=(-, -), dims=(, ))\n            K_freq = torch.fft.rfft2(full)\n\n     ():\n         torch.tensor(\n            [\n                [, , c3, , ],\n                [, , c2, , ],\n                [c3, c2, , c2, c3],\n                [, , c2, , ],\n                [, , c3, , ],\n            ],\n            dtype=.dtype,\n        )\n\n     () -&gt; torch.Tensor:\n        B, S, _, _ = p1.shape\n        \n        p_flat = rearrange(p1, )\n        P_freq = torch.fft.rfft2(p_flat)\n        L_freq = P_freq * .K_freq\n        lap = torch.fft.irfft2(L_freq, s=(.nzbc, .nxbc))\n         rearrange(lap, , b=B, s=S)\n</code></pre>\n<p><strong>Note:</strong></p>\n<ul>\n<li>Though FFT equivalent is 30-50% faster than conv5x5 implementation, it slightly increased error:<ul>\n<li>e.g. MAE=3.56e-06 -&gt; 2.50e-05 for FlatVel_A, 3.45e-06 -&gt; 1.07e-04 for FlatVel_B etc.</li></ul></li>\n</ul>\n<p><strong>The performance</strong>:</p>\n<ul>\n<li>6-7 hours to generate full seismic data of OpenWFI (470K)<ul>\n<li><strong>0.046~0.053 sec/sample</strong> (1x RTX4090)</li></ul></li>\n</ul>\n<p><strong>What I didn't tried</strong>:</p>\n<ul>\n<li>write custom cuda kernel</li>\n</ul>",
  "messages": [
    {
      "id": 3237767,
      "postDate": "2025-07-01T09:41:58.587Z",
      "content": "<p>I tried much effort to make forward model as efficient as possible.<br>\nHowever, it seemed I cannot drastic computation improvement comparing to O(n_t * n_z * nx).<br>\nSo let's share the useful tips to make forward model efficient.</p>\n<p>For example, I used these tips:</p>\n<ul>\n<li>convert rolling to conv5x5</li>\n<li>convert conv 5x5 to: FFT-&gt;element-wise multiplication-&gt;inverse FFT</li>\n</ul>\n<p><strong>implementation detail</strong>:</p>\n<p>This approach uses the fact that <strong>convolution with circular padding is equivalent to item-wise multiplication in the frequency domain</strong>.<br>\nTheoretically,</p>\n<p>$$<br>\n\\text{FFT}(\\text{Conv}(w(x, y), f(x, y))) = \\text{FFT}(w(x, y)) * \\text{FFT}(f(x, y))<br>\n$$</p>\n<pre><code>     ():\n            ....\n            \n            kernel5 = .build_kernel()\n            full = torch.zeros(nzbc, nxbc, dtype=dtype)\n            full[:, :] = kernel5\n            full = torch.roll(full, shifts=(-, -), dims=(, ))\n            K_freq = torch.fft.rfft2(full)\n\n     ():\n         torch.tensor(\n            [\n                [, , c3, , ],\n                [, , c2, , ],\n                [c3, c2, , c2, c3],\n                [, , c2, , ],\n                [, , c3, , ],\n            ],\n            dtype=.dtype,\n        )\n\n     () -&gt; torch.Tensor:\n        B, S, _, _ = p1.shape\n        \n        p_flat = rearrange(p1, )\n        P_freq = torch.fft.rfft2(p_flat)\n        L_freq = P_freq * .K_freq\n        lap = torch.fft.irfft2(L_freq, s=(.nzbc, .nxbc))\n         rearrange(lap, , b=B, s=S)\n</code></pre>\n<p><strong>Note:</strong></p>\n<ul>\n<li>Though FFT equivalent is 30-50% faster than conv5x5 implementation, it slightly increased error:<ul>\n<li>e.g. MAE=3.56e-06 -&gt; 2.50e-05 for FlatVel_A, 3.45e-06 -&gt; 1.07e-04 for FlatVel_B etc.</li></ul></li>\n</ul>\n<p><strong>The performance</strong>:</p>\n<ul>\n<li>6-7 hours to generate full seismic data of OpenWFI (470K)<ul>\n<li><strong>0.046~0.053 sec/sample</strong> (1x RTX4090)</li></ul></li>\n</ul>\n<p><strong>What I didn't tried</strong>:</p>\n<ul>\n<li>write custom cuda kernel</li>\n</ul>",
      "rawMarkdown": "I tried much effort to make forward model as efficient as possible.\nHowever, it seemed I cannot drastic computation improvement comparing to O(n_t * n_z * nx).\nSo let's share the useful tips to make forward model efficient.\n\nFor example, I used these tips:\n\n- convert rolling to conv5x5\n- convert conv 5x5 to: FFT->element-wise multiplication->inverse FFT\n\n**implementation detail**:\n\nThis approach uses the fact that **convolution with circular padding is equivalent to item-wise multiplication in the frequency domain**.\nTheoretically,\n\n$$\n\\text{FFT}(\\text{Conv}(w(x, y), f(x, y))) = \\text{FFT}(w(x, y)) * \\text{FFT}(f(x, y))\n$$\n\n\n```python\n    def __init__(self, ...):\n            ....\n            # build full padded kernel\n            kernel5 = self.build_kernel()\n            full = torch.zeros(nzbc, nxbc, dtype=dtype)\n            full[:5, :5] = kernel5\n            full = torch.roll(full, shifts=(-2, -2), dims=(0, 1))\n            K_freq = torch.fft.rfft2(full)\n\n    def build_kernel(self, c2=4 / 3, c3=-1 / 12):\n        return torch.tensor(\n            [\n                [0, 0, c3, 0, 0],\n                [0, 0, c2, 0, 0],\n                [c3, c2, 0, c2, c3],\n                [0, 0, c2, 0, 0],\n                [0, 0, c3, 0, 0],\n            ],\n            dtype=self.dtype,\n        )\n\n    def fft_laplacian(self, p1: torch.Tensor) -> torch.Tensor:\n        B, S, _, _ = p1.shape\n        # flatten channels for fft\n        p_flat = rearrange(p1, \"b s nzbc nxbc -> (b s) 1 nzbc nxbc\")\n        P_freq = torch.fft.rfft2(p_flat)\n        L_freq = P_freq * self.K_freq\n        lap = torch.fft.irfft2(L_freq, s=(self.nzbc, self.nxbc))\n        return rearrange(lap, \"(b s) 1 nzbc nxbc -> b s nzbc nxbc\", b=B, s=S)\n```\n**Note:**\n\n- Though FFT equivalent is 30-50% faster than conv5x5 implementation, it slightly increased error:\n    - e.g. MAE=3.56e-06 -> 2.50e-05 for FlatVel_A, 3.45e-06 -> 1.07e-04 for FlatVel_B etc.\n\n**The performance**:\n\n- 6-7 hours to generate full seismic data of OpenWFI (470K)\n    - **0.046~0.053 sec/sample** (1x RTX4090)\n\n**What I didn't tried**:\n\n- write custom cuda kernel",
      "votes": 6
    },
    {
      "id": 3239808,
      "postDate": "2025-07-03T07:07:35.377Z",
      "content": "<p><a href=\"https://www.kaggle.com/cnumber\" target=\"_blank\">@cnumber</a> <a href=\"https://www.kaggle.com/junkoda\" target=\"_blank\">@junkoda</a> </p>\n<p><strong>BONUS:</strong></p>\n<p>Well, considering the efficiency of <code>torch.compile</code> is mainly come from fusing multiple graph computation, why don't we fuse multiple time steps?</p>\n<p><strong>The result:</strong></p>\n<p>Yes, fusing multiple time step using <code>torch.compile</code> gives us further scale up in efficiency.<br>\nFusing 50~100 time step calculation with <code>torch.compile</code> result in <strong>7.3 times faster</strong> result. (<strong>9.77 times faster</strong> comparing to the Conv2d implementation without toch.compile.)</p>\n<p>With this trick, calculating PINN reconstruction loss in real time becomes much realistic.</p>\n<p>(Note that <code>nk</code> means number of time step computation using <code>torch.compile</code>)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff0103642564c9032046c326932e404f3%2FScreenshot%202025-07-03%20at%2016.04.17.png?generation=1751526276272992&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "@cnumber @junkoda \n\n**BONUS:**\n\nWell, considering the efficiency of `torch.compile` is mainly come from fusing multiple graph computation, why don't we fuse multiple time steps?\n\n**The result:**\n\nYes, fusing multiple time step using `torch.compile` gives us further scale up in efficiency.\nFusing 50~100 time step calculation with `torch.compile` result in **7.3 times faster** result. (**9.77 times faster** comparing to the Conv2d implementation without toch.compile.)\n\nWith this trick, calculating PINN reconstruction loss in real time becomes much realistic.\n\n(Note that `nk` means number of time step computation using `torch.compile`)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff0103642564c9032046c326932e404f3%2FScreenshot%202025-07-03%20at%2016.04.17.png?generation=1751526276272992&alt=media)",
      "votes": 3,
      "replies": [
        {
          "id": 3239812,
          "postDate": "2025-07-03T07:10:33.573Z",
          "content": "<p>Thanks for the experiment. I am going to repeat it. Do you think u can release your code for me to reference?</p>",
          "rawMarkdown": "Thanks for the experiment. I am going to repeat it. Do you think u can release your code for me to reference?",
          "replies": [
            {
              "id": 3239854,
              "postDate": "2025-07-03T08:07:35.880Z",
              "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I attached a2d_mod_abc24_torch_v10_4.py in this topic. Maybe it will help.</p>\n<p>LICENSE: Apache 2.0</p>",
              "rawMarkdown": "@hengck23 I attached a2d_mod_abc24_torch_v10_4.py in this topic. Maybe it will help.\n\nLICENSE: Apache 2.0",
              "votes": 1
            }
          ]
        },
        {
          "id": 3240237,
          "postDate": "2025-07-03T15:54:39.140Z",
          "content": "<p><strong>Generation speed of Forward Process (forward only)</strong>:</p>\n<p>Maybe the speedup for only forward process (without <code>autograd()</code>) is around <strong>3~4x times</strong>.</p>\n<p>Note that with this condition, I observed that the generation speed is almost the same with nk=10 and nk=100.</p>\n<p><strong>hypothesis:</strong></p>\n<ul>\n<li>Speedup by <code>torch.compile()</code> is mainly on backward computation (i.e. <code>autograd()</code>)?</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff6f6a9264abff2be05c30a61772e8b74%2FScreenshot%202025-07-04%20at%200.51.28.png?generation=1751558034936343&amp;alt=media\" alt=\"\"></p>",
          "rawMarkdown": "**Generation speed of Forward Process (forward only)**:\n\nMaybe the speedup for only forward process (without `autograd()`) is around **3~4x times**.\n\nNote that with this condition, I observed that the generation speed is almost the same with nk=10 and nk=100.\n\n**hypothesis:**\n\n- Speedup by `torch.compile()` is mainly on backward computation (i.e. `autograd()`)?\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff6f6a9264abff2be05c30a61772e8b74%2FScreenshot%202025-07-04%20at%200.51.28.png?generation=1751558034936343&alt=media)"
        },
        {
          "id": 3240578,
          "postDate": "2025-07-04T02:29:50.190Z",
          "content": "<p>We achieved x10 speed with torch.compile for forward process, so maybe our conditions could be a different different</p>",
          "rawMarkdown": "We achieved x10 speed with torch.compile for forward process, so maybe our conditions could be a different different",
          "replies": [
            {
              "id": 3240960,
              "postDate": "2025-07-04T11:41:13.350Z",
              "content": "<p>Hmm.. Or, maybe the baseline is different. My baseline is conv2d implementation which is already faster than roll without compile.<br>\nHowever, I'm interested if compiling only single step makes 10x speed up.<br>\nI think hardware (GPU) architecture also affect the result.</p>",
              "rawMarkdown": "Hmm.. Or, maybe the baseline is different. My baseline is conv2d implementation which is already faster than roll without compile.\nHowever, I'm interested if compiling only single step makes 10x speed up.\nI think hardware (GPU) architecture also affect the result."
            }
          ]
        }
      ]
    },
    {
      "id": 3237769,
      "postDate": "2025-07-01T09:43:21.580Z",
      "content": "<p>Using the original code with torch.compile boosted the speed to around x10</p>",
      "rawMarkdown": "Using the original code with torch.compile boosted the speed to around x10",
      "votes": 3,
      "replies": [
        {
          "id": 3237776,
          "postDate": "2025-07-01T09:51:53.267Z",
          "content": "<p>Wow, it seems cheapest approach considering the change of lines it requires.</p>\n<p>I tried to use <code>torch.compile</code> but my rewritten torch code outputs hundreds of compile error so I forgiven this.</p>",
          "rawMarkdown": "Wow, it seems cheapest approach considering the change of lines it requires.\n\nI tried to use `torch.compile` but my rewritten torch code outputs hundreds of compile error so I forgiven this."
        },
        {
          "id": 3239339,
          "postDate": "2025-07-02T17:44:31.830Z",
          "content": "<p>How long did the compilation (first function call) take? I used on a different code, but it took about 30 minutes for T=500 and started complaining that it failed to optimize and I gave up for T=1000.</p>",
          "rawMarkdown": "How long did the compilation (first function call) take? I used on a different code, but it took about 30 minutes for T=500 and started complaining that it failed to optimize and I gave up for T=1000."
        },
        {
          "id": 3239560,
          "postDate": "2025-07-03T00:19:52.040Z",
          "content": "<p>It only took around 3 seconds.<br>\nI used mode=\"default\" without any other arguments.</p>",
          "rawMarkdown": "It only took around 3 seconds.\nI used mode=\"default\" without any other arguments.",
          "votes": 1
        },
        {
          "id": 3239567,
          "postDate": "2025-07-03T00:46:59.467Z",
          "content": "<p>The trick was to compile only the function used in the internal loop<br>\nThe simulation code should look like this</p>\n<pre><code> ()\n    initialize_parameters()\n     i  ():\n        p0, p1 = calc_one_step(p0, p1)\n     p0\n</code></pre>\n<p>Compiling this entire function takes hous, but one can achieve acceleration even by compiling ohly the <code>calc_one_step</code> function.</p>",
          "rawMarkdown": "The trick was to compile only the function used in the internal loop\nThe simulation code should look like this\n\n```python\n\ndef vel_to_seis()\n    initialize_parameters()\n    for i in range(1000):\n        p0, p1 = calc_one_step(p0, p1)\n    return p0\n```\n\nCompiling this entire function takes hous, but one can achieve acceleration even by compiling ohly the `calc_one_step` function.",
          "votes": 3
        }
      ]
    },
    {
      "id": 3237852,
      "postDate": "2025-07-01T10:48:43.243Z",
      "content": "<p>in my experiment i am surprised that roll is faster then conv2d.<br>\nit is a pity that the conv2d cannot be converted to 2x conv1d</p>",
      "rawMarkdown": "in my experiment i am surprised that roll is faster then conv2d.\nit is a pity that the conv2d cannot be converted to 2x conv1d\n",
      "votes": 1,
      "replies": [
        {
          "id": 3237859,
          "postDate": "2025-07-01T10:53:41.023Z",
          "content": "<p>Why it can't?</p>",
          "rawMarkdown": "Why it can't?",
          "replies": [
            {
              "id": 3237870,
              "postDate": "2025-07-01T10:58:41.490Z",
              "content": "<p>\"Why it can't?\"<br>\nthe kernel is symmetrical but not separable.</p>",
              "rawMarkdown": "\"Why it can't?\"\nthe kernel is symmetrical but not separable."
            },
            {
              "id": 3237888,
              "postDate": "2025-07-01T11:09:37.083Z",
              "content": "<p>Why not?</p>\n<p>horizontal_part = ndimage.convolve1d(p1, kernel, axis=2, mode='wrap')<br>\n            vertical_part = ndimage.convolve1d(p1, kernel, axis=1, mode='wrap')</p>\n<pre><code>        p = temp1 * p1 - temp2 * p0 + alpha * (horizontal_part + vertical_part)\n</code></pre>\n<p>This is what I used</p>",
              "rawMarkdown": "Why not?\n\n\nhorizontal_part = ndimage.convolve1d(p1, kernel, axis=2, mode='wrap')\n            vertical_part = ndimage.convolve1d(p1, kernel, axis=1, mode='wrap')\n            \n            p = temp1 * p1 - temp2 * p0 + alpha * (horizontal_part + vertical_part)\n\nThis is what I used",
              "votes": 1
            },
            {
              "id": 3237901,
              "postDate": "2025-07-01T11:17:36.623Z",
              "content": "<p>Well, the kernel is not separable, but we can add the result of two 1d kernels element-wise. However, this implementation was slower than 5x5 kernel in my case.</p>",
              "rawMarkdown": "Well, the kernel is not separable, but we can add the result of two 1d kernels element-wise. However, this implementation was slower than 5x5 kernel in my case.",
              "votes": 1
            },
            {
              "id": 3237910,
              "postDate": "2025-07-01T11:22:50.933Z",
              "content": "<p>here is it<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5d223bd212858c7931609fd443b64cc3%2FSelection_999(8315).png?generation=1751368951635577&amp;alt=media\" alt=\"\"></p>\n<p>so it is not 'conv1d(conv1d(x))' but 'conv1d(x) + conv1d(x)'</p>",
              "rawMarkdown": "here is it\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5d223bd212858c7931609fd443b64cc3%2FSelection_999(8315).png?generation=1751368951635577&alt=media)\n\nso it is not 'conv1d(conv1d(x))' but 'conv1d(x) + conv1d(x)'",
              "votes": 1
            }
          ]
        },
        {
          "id": 3237860,
          "postDate": "2025-07-01T10:53:53.243Z",
          "content": "<blockquote>\n  <p>in my experiment i am surprised that roll is faster then conv2d.</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> <br>\nInteresting. In my case, I remembered conv5x5 was much faster than rolling operations. (FYI: 1x5 &amp;5x1 1d conv were slower than 5x5 conv in my case.)</p>\n<p>Did you use torch.compile() on your roll implementation?</p>",
          "rawMarkdown": "> in my experiment i am surprised that roll is faster then conv2d.\n\n@hengck23 \nInteresting. In my case, I remembered conv5x5 was much faster than rolling operations. (FYI: 1x5 &5x1 1d conv were slower than 5x5 conv in my case.)\n\nDid you use torch.compile() on your roll implementation?",
          "votes": 1,
          "replies": [
            {
              "id": 3237865,
              "postDate": "2025-07-01T10:56:23.157Z",
              "content": "<p>no. just run pytorch code.</p>",
              "rawMarkdown": "no. just run pytorch code.",
              "votes": 1
            }
          ]
        },
        {
          "id": 3239466,
          "postDate": "2025-07-02T20:56:28.303Z",
          "content": "<p>I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️<br>\nIt must depend on low-level optimization like data fetching strategy. I just tried. </p>\n<pre><code>roll    33.62 sec\nconv1  136.22 sec\nconv2   23.68 sec\n</code></pre>\n<p>PyTorch and RTX4090<br>\n500 data with batch_size=5 </p>\n<p>So I agree with 🦖.<br>\nBy the way, we are all reptiles 😄</p>",
          "rawMarkdown": "I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️\nIt must depend on low-level optimization like data fetching strategy. I just tried. \n\n```text\nroll    33.62 sec\nconv1  136.22 sec\nconv2   23.68 sec\n```\n\nPyTorch and RTX4090\n500 data with batch_size=5 \n\nSo I agree with 🦖.\nBy the way, we are all reptiles 😄\n                ",
          "votes": 1,
          "replies": [
            {
              "id": 3239931,
              "postDate": "2025-07-03T09:41:55.140Z",
              "content": "<blockquote>\n  <p>I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/junkoda\" target=\"_blank\">@junkoda</a> Maybe replacing Conv1d into conv2d(1x5) and conv2d(5x1) will get closer result. This trick is a bit niche, I believe this difference indicates pytorch's Cuda kernel for 1d conv has more room to optimize.</p>",
              "rawMarkdown": "> I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️\n\n@junkoda Maybe replacing Conv1d into conv2d(1x5) and conv2d(5x1) will get closer result. This trick is a bit niche, I believe this difference indicates pytorch's Cuda kernel for 1d conv has more room to optimize."
            }
          ]
        },
        {
          "id": 3239572,
          "postDate": "2025-07-03T00:53:39.220Z",
          "content": "<p>Does this hold even with torch.compile?</p>",
          "rawMarkdown": "Does this hold even with torch.compile?",
          "replies": [
            {
              "id": 3239725,
              "postDate": "2025-07-03T04:26:19.580Z",
              "content": "<blockquote>\n  <p>Does this hold even with torch.compile?</p>\n</blockquote>\n<p>With compile, <strong>roll is the fastest</strong>.<br>\nInterestingly, FFT implementation gets slower with torch.compile.<br>\nlessen: automatic optimization is often better than manual optimization… 😂</p>\n<pre><code>    model.forward_single_step = torch.(\n        model.forward_single_step, fullgraph=, dynamic=, mode=\n    )\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fdf33cc9e90b5ae6d0b9f5002a8b031da%2FScreenshot%202025-07-03%20at%2013.22.37.png?generation=1751516626625096&amp;alt=media\" alt=\"\"></p>",
              "rawMarkdown": "> Does this hold even with torch.compile?\n\nWith compile, **roll is the fastest**.\nInterestingly, FFT implementation gets slower with torch.compile.\nlessen: automatic optimization is often better than manual optimization... 😂\n\n```python\n    model.forward_single_step = torch.compile(\n        model.forward_single_step, fullgraph=False, dynamic=False, mode=\"default\"\n    )\n```\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fdf33cc9e90b5ae6d0b9f5002a8b031da%2FScreenshot%202025-07-03%20at%2013.22.37.png?generation=1751516626625096&alt=media)",
              "votes": 2
            },
            {
              "id": 3239753,
              "postDate": "2025-07-03T05:16:10.853Z",
              "content": "<p>Thank you, seems reasonable</p>",
              "rawMarkdown": "Thank you, seems reasonable"
            }
          ]
        }
      ]
    },
    {
      "id": 3237775,
      "postDate": "2025-07-01T09:50:32.567Z",
      "rawMarkdown": "",
      "votes": -1,
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3239808,
      "author_name": "Bilzard",
      "author_url": "",
      "post_date": "2025-07-03T07:07:35.377000",
      "content": "<p><a href=\"https://www.kaggle.com/cnumber\" target=\"_blank\">@cnumber</a> <a href=\"https://www.kaggle.com/junkoda\" target=\"_blank\">@junkoda</a> </p>\n<p><strong>BONUS:</strong></p>\n<p>Well, considering the efficiency of <code>torch.compile</code> is mainly come from fusing multiple graph computation, why don't we fuse multiple time steps?</p>\n<p><strong>The result:</strong></p>\n<p>Yes, fusing multiple time step using <code>torch.compile</code> gives us further scale up in efficiency.<br>\nFusing 50~100 time step calculation with <code>torch.compile</code> result in <strong>7.3 times faster</strong> result. (<strong>9.77 times faster</strong> comparing to the Conv2d implementation without toch.compile.)</p>\n<p>With this trick, calculating PINN reconstruction loss in real time becomes much realistic.</p>\n<p>(Note that <code>nk</code> means number of time step computation using <code>torch.compile</code>)</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff0103642564c9032046c326932e404f3%2FScreenshot%202025-07-03%20at%2016.04.17.png?generation=1751526276272992&amp;alt=media\" alt=\"\"></p>",
      "votes": 3,
      "replies": [
        {
          "id": 3239812,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2025-07-03T07:10:33.573000",
          "content": "<p>Thanks for the experiment. I am going to repeat it. Do you think u can release your code for me to reference?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3239854,
              "author_name": "Bilzard",
              "author_url": "",
              "post_date": "2025-07-03T08:07:35.880000",
              "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> I attached a2d_mod_abc24_torch_v10_4.py in this topic. Maybe it will help.</p>\n<p>LICENSE: Apache 2.0</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 3240237,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2025-07-03T15:54:39.140000",
          "content": "<p><strong>Generation speed of Forward Process (forward only)</strong>:</p>\n<p>Maybe the speedup for only forward process (without <code>autograd()</code>) is around <strong>3~4x times</strong>.</p>\n<p>Note that with this condition, I observed that the generation speed is almost the same with nk=10 and nk=100.</p>\n<p><strong>hypothesis:</strong></p>\n<ul>\n<li>Speedup by <code>torch.compile()</code> is mainly on backward computation (i.e. <code>autograd()</code>)?</li>\n</ul>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff6f6a9264abff2be05c30a61772e8b74%2FScreenshot%202025-07-04%20at%200.51.28.png?generation=1751558034936343&amp;alt=media\" alt=\"\"></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3240578,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-07-04T02:29:50.190000",
          "content": "<p>We achieved x10 speed with torch.compile for forward process, so maybe our conditions could be a different different</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3240960,
              "author_name": "Bilzard",
              "author_url": "",
              "post_date": "2025-07-04T11:41:13.350000",
              "content": "<p>Hmm.. Or, maybe the baseline is different. My baseline is conv2d implementation which is already faster than roll without compile.<br>\nHowever, I'm interested if compiling only single step makes 10x speed up.<br>\nI think hardware (GPU) architecture also affect the result.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3237769,
      "author_name": "c-number",
      "author_url": "",
      "post_date": "2025-07-01T09:43:21.580000",
      "content": "<p>Using the original code with torch.compile boosted the speed to around x10</p>",
      "votes": 3,
      "replies": [
        {
          "id": 3237776,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2025-07-01T09:51:53.267000",
          "content": "<p>Wow, it seems cheapest approach considering the change of lines it requires.</p>\n<p>I tried to use <code>torch.compile</code> but my rewritten torch code outputs hundreds of compile error so I forgiven this.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3239339,
          "author_name": "🐢 Jun Koda",
          "author_url": "",
          "post_date": "2025-07-02T17:44:31.830000",
          "content": "<p>How long did the compilation (first function call) take? I used on a different code, but it took about 30 minutes for T=500 and started complaining that it failed to optimize and I gave up for T=1000.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3239560,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-07-03T00:19:52.040000",
          "content": "<p>It only took around 3 seconds.<br>\nI used mode=\"default\" without any other arguments.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 3239567,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-07-03T00:46:59.467000",
          "content": "<p>The trick was to compile only the function used in the internal loop<br>\nThe simulation code should look like this</p>\n<pre><code> ()\n    initialize_parameters()\n     i  ():\n        p0, p1 = calc_one_step(p0, p1)\n     p0\n</code></pre>\n<p>Compiling this entire function takes hous, but one can achieve acceleration even by compiling ohly the <code>calc_one_step</code> function.</p>",
          "votes": 3,
          "replies": []
        }
      ]
    },
    {
      "id": 3237852,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2025-07-01T10:48:43.243000",
      "content": "<p>in my experiment i am surprised that roll is faster then conv2d.<br>\nit is a pity that the conv2d cannot be converted to 2x conv1d</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3237859,
          "author_name": "greySnow",
          "author_url": "",
          "post_date": "2025-07-01T10:53:41.023000",
          "content": "<p>Why it can't?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3237870,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2025-07-01T10:58:41.490000",
              "content": "<p>\"Why it can't?\"<br>\nthe kernel is symmetrical but not separable.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3237888,
              "author_name": "greySnow",
              "author_url": "",
              "post_date": "2025-07-01T11:09:37.083000",
              "content": "<p>Why not?</p>\n<p>horizontal_part = ndimage.convolve1d(p1, kernel, axis=2, mode='wrap')<br>\n            vertical_part = ndimage.convolve1d(p1, kernel, axis=1, mode='wrap')</p>\n<pre><code>        p = temp1 * p1 - temp2 * p0 + alpha * (horizontal_part + vertical_part)\n</code></pre>\n<p>This is what I used</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3237901,
              "author_name": "Bilzard",
              "author_url": "",
              "post_date": "2025-07-01T11:17:36.623000",
              "content": "<p>Well, the kernel is not separable, but we can add the result of two 1d kernels element-wise. However, this implementation was slower than 5x5 kernel in my case.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3237910,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2025-07-01T11:22:50.933000",
              "content": "<p>here is it<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F5d223bd212858c7931609fd443b64cc3%2FSelection_999(8315).png?generation=1751368951635577&amp;alt=media\" alt=\"\"></p>\n<p>so it is not 'conv1d(conv1d(x))' but 'conv1d(x) + conv1d(x)'</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 3237860,
          "author_name": "Bilzard",
          "author_url": "",
          "post_date": "2025-07-01T10:53:53.243000",
          "content": "<blockquote>\n  <p>in my experiment i am surprised that roll is faster then conv2d.</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> <br>\nInteresting. In my case, I remembered conv5x5 was much faster than rolling operations. (FYI: 1x5 &amp;5x1 1d conv were slower than 5x5 conv in my case.)</p>\n<p>Did you use torch.compile() on your roll implementation?</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3237865,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2025-07-01T10:56:23.157000",
              "content": "<p>no. just run pytorch code.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 3239466,
          "author_name": "🐢 Jun Koda",
          "author_url": "",
          "post_date": "2025-07-02T20:56:28.303000",
          "content": "<p>I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️<br>\nIt must depend on low-level optimization like data fetching strategy. I just tried. </p>\n<pre><code>roll    33.62 sec\nconv1  136.22 sec\nconv2   23.68 sec\n</code></pre>\n<p>PyTorch and RTX4090<br>\n500 data with batch_size=5 </p>\n<p>So I agree with 🦖.<br>\nBy the way, we are all reptiles 😄</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3239931,
              "author_name": "Bilzard",
              "author_url": "",
              "post_date": "2025-07-03T09:41:55.140000",
              "content": "<blockquote>\n  <p>I'm surprised that conv2d with 25 operations is much faster than conv1d with 10 operations‼️</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/junkoda\" target=\"_blank\">@junkoda</a> Maybe replacing Conv1d into conv2d(1x5) and conv2d(5x1) will get closer result. This trick is a bit niche, I believe this difference indicates pytorch's Cuda kernel for 1d conv has more room to optimize.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3239572,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-07-03T00:53:39.220000",
          "content": "<p>Does this hold even with torch.compile?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3239725,
              "author_name": "Bilzard",
              "author_url": "",
              "post_date": "2025-07-03T04:26:19.580000",
              "content": "<blockquote>\n  <p>Does this hold even with torch.compile?</p>\n</blockquote>\n<p>With compile, <strong>roll is the fastest</strong>.<br>\nInterestingly, FFT implementation gets slower with torch.compile.<br>\nlessen: automatic optimization is often better than manual optimization… 😂</p>\n<pre><code>    model.forward_single_step = torch.(\n        model.forward_single_step, fullgraph=, dynamic=, mode=\n    )\n</code></pre>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Fdf33cc9e90b5ae6d0b9f5002a8b031da%2FScreenshot%202025-07-03%20at%2013.22.37.png?generation=1751516626625096&amp;alt=media\" alt=\"\"></p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3239753,
              "author_name": "c-number",
              "author_url": "",
              "post_date": "2025-07-03T05:16:10.853000",
              "content": "<p>Thank you, seems reasonable</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3237775,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-07-01T09:50:32.567000",
      "content": "",
      "votes": -1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3237767": "I tried much effort to make forward model as efficient as possible.\nHowever, it seemed I cannot drastic computation improvement comparing to O(n_t * n_z * nx).\nSo let's share the useful tips to make forward model efficient.\n\nFor example, I used these tips:\n\n- convert rolling to conv5x5\n- convert conv 5x5 to: FFT->element-wise multiplication->inverse FFT\n\n**implementation detail**:\n\nThis approach uses the fact that **convolution with circular padding is equivalent to item-wise multiplication in the frequency domain**.\nTheoretically,\n\n$$\n\\text{FFT}(\\text{Conv}(w(x, y), f(x, y))) = \\text{FFT}(w(x, y)) * \\text{FFT}(f(x, y))\n$$\n\n\n```python\n    def __init__(self, ...):\n            ....\n            # build full padded kernel\n            kernel5 = self.build_kernel()\n            full = torch.zeros(nzbc, nxbc, dtype=dtype)\n            full[:5, :5] = kernel5\n            full = torch.roll(full, shifts=(-2, -2), dims=(0, 1))\n            K_freq = torch.fft.rfft2(full)\n\n    def build_kernel(self, c2=4 / 3, c3=-1 / 12):\n        return torch.tensor(\n            [\n                [0, 0, c3, 0, 0],\n                [0, 0, c2, 0, 0],\n                [c3, c2, 0, c2, c3],\n                [0, 0, c2, 0, 0],\n                [0, 0, c3, 0, 0],\n            ],\n            dtype=self.dtype,\n        )\n\n    def fft_laplacian(self, p1: torch.Tensor) -> torch.Tensor:\n        B, S, _, _ = p1.shape\n        # flatten channels for fft\n        p_flat = rearrange(p1, \"b s nzbc nxbc -> (b s) 1 nzbc nxbc\")\n        P_freq = torch.fft.rfft2(p_flat)\n        L_freq = P_freq * self.K_freq\n        lap = torch.fft.irfft2(L_freq, s=(self.nzbc, self.nxbc))\n        return rearrange(lap, \"(b s) 1 nzbc nxbc -> b s nzbc nxbc\", b=B, s=S)\n```\n**Note:**\n\n- Though FFT equivalent is 30-50% faster than conv5x5 implementation, it slightly increased error:\n    - e.g. MAE=3.56e-06 -> 2.50e-05 for FlatVel_A, 3.45e-06 -> 1.07e-04 for FlatVel_B etc.\n\n**The performance**:\n\n- 6-7 hours to generate full seismic data of OpenWFI (470K)\n    - **0.046~0.053 sec/sample** (1x RTX4090)\n\n**What I didn't tried**:\n\n- write custom cuda kernel",
    "3239808": "@cnumber @junkoda \n\n**BONUS:**\n\nWell, considering the efficiency of `torch.compile` is mainly come from fusing multiple graph computation, why don't we fuse multiple time steps?\n\n**The result:**\n\nYes, fusing multiple time step using `torch.compile` gives us further scale up in efficiency.\nFusing 50~100 time step calculation with `torch.compile` result in **7.3 times faster** result. (**9.77 times faster** comparing to the Conv2d implementation without toch.compile.)\n\nWith this trick, calculating PINN reconstruction loss in real time becomes much realistic.\n\n(Note that `nk` means number of time step computation using `torch.compile`)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2Ff0103642564c9032046c326932e404f3%2FScreenshot%202025-07-03%20at%2016.04.17.png?generation=1751526276272992&alt=media)",
    "3237769": "Using the original code with torch.compile boosted the speed to around x10",
    "3237852": "in my experiment i am surprised that roll is faster then conv2d.\nit is a pity that the conv2d cannot be converted to 2x conv1d\n",
    "3237775": ""
  }
}