{ "id": 575097, "title": "Improved UNet pipepline with larger dataset", "url": "/competitions/waveform-inversion/discussion/575097", "author_name": "Egor Trushin", "post_date": "2025-04-25T22:01:21.509000", "votes": 32, "comment_count": 12, "views": 0, "content": "

Hello everyone,

\n

To contribute to the overall progress in the competition, I have published a notebook where UNet model is employed and a part of the full OpenFWI dataset is used for training. LB of 107.5 was achieved, check corresponding notebook. LB can be improved by increasing the number of epochs, amount of data used, etc. Especially if you have the possibility to train locally.

\n

Feel free to report bugs or possible improvements. I plan to introduce updates later.

\n

Good luck in the competition.

\n

Notebooks

\n
    \n
  1. [GWI] Improved UNet pipepline with larger dataset
    2. \n
  2. [GWI] UNet with float16 dataset
    3. \n
\n

float16 datasets

\n
    \n
  1. openfwi_float16_1
    2. \n
  2. openfwi_float16_2
    3. \n
  3. openfwi_float16_test
    4. \n
", "messages": [ { "id": 3187363, "postDate": "2025-04-25T22:01:21.510Z", "content": "

Hello everyone,

\n

To contribute to the overall progress in the competition, I have published a notebook where UNet model is employed and a part of the full OpenFWI dataset is used for training. LB of 107.5 was achieved, check corresponding notebook. LB can be improved by increasing the number of epochs, amount of data used, etc. Especially if you have the possibility to train locally.

\n

Feel free to report bugs or possible improvements. I plan to introduce updates later.

\n

Good luck in the competition.

\n

Notebooks

\n
    \n
  1. [GWI] Improved UNet pipepline with larger dataset
    2. \n
  2. [GWI] UNet with float16 dataset
    3. \n
\n

float16 datasets

\n
    \n
  1. openfwi_float16_1
    2. \n
  2. openfwi_float16_2
    3. \n
  3. openfwi_float16_test
    4. \n
", "rawMarkdown": "Hello everyone,\n\nTo contribute to the overall progress in the competition, I have published a notebook where UNet model is employed and a part of the full OpenFWI dataset is used for training. LB of 107.5 was achieved, check [corresponding notebook](https://www.kaggle.com/code/egortrushin/gwi-improved-unet-pipepline-with-larger-dataset). LB can be improved by increasing the number of epochs, amount of data used, etc. Especially if you have the possibility to train locally.\n\nFeel free to report bugs or possible improvements. I plan to introduce updates later.\n\nGood luck in the competition.\n\n\n**Notebooks**\n1. [[GWI] Improved UNet pipepline with larger dataset](https://www.kaggle.com/code/egortrushin/gwi-improved-unet-pipepline-with-larger-dataset)\n1. [[GWI] UNet with float16 dataset](https://www.kaggle.com/code/egortrushin/gwi-unet-with-float16-dataset)\n\n**float16 datasets**\n1. [openfwi_float16_1](https://www.kaggle.com/datasets/egortrushin/open-wfi-1)\n1. [openfwi_float16_2](https://www.kaggle.com/datasets/egortrushin/open-wfi-2)\n1. [openfwi_float16_test](https://www.kaggle.com/datasets/egortrushin/open-wfi-test)", "votes": 31 }, { "id": 3190306, "postDate": "2025-04-30T13:04:27.857Z", "content": "

I have shared an updated notebook that uses the dataset converted from float32 to float16. This allows us to train twice faster. Using half of the openWFI dataset, an improved LB score of 93.2 was achieved using the same model as in the previous version and in the same training time.

", "rawMarkdown": "I have shared [an updated notebook](https://www.kaggle.com/code/egortrushin/gwi-unet-with-float16-dataset) that uses the dataset converted from float32 to float16. This allows us to train twice faster. Using half of the openWFI dataset, an improved LB score of 93.2 was achieved using the same model as in the previous version and in the same training time.", "votes": 7, "replies": [ { "id": 3190460, "postDate": "2025-04-30T17:29:02.147Z", "content": "

Would you say that the most viable way to reduce MAE is by utilizing as much training data as possible? I've been looking at the public notebooks, and that looks like the main strength of some of them.

", "rawMarkdown": "Would you say that the most viable way to reduce MAE is by utilizing as much training data as possible? I've been looking at the public notebooks, and that looks like the main strength of some of them.", "votes": 2, "replies": [ { "id": 3190494, "postDate": "2025-04-30T18:20:43.130Z", "content": "

It looks like one need to use as much data as possible to reduce MAE, see the discussion here: Scaling up results- Kaggle data vs full data. Of course, other things like NN architecture, hyperparameters, etc. also play a role.

", "rawMarkdown": "It looks like one need to use as much data as possible to reduce MAE, see the discussion here: [Scaling up results- Kaggle data vs full data](https://www.kaggle.com/competitions/waveform-inversion/discussion/574495). Of course, other things like NN architecture, hyperparameters, etc. also play a role.", "votes": 1 }, { "id": 3190582, "postDate": "2025-04-30T21:42:51.787Z", "content": "

Thank you. That makes sense. You mention, \"โ€ฆthe possibility to train locally.\" Would you be willing to elaborate on this? Could you clarify if people are using programs for 5+ hours continuously on their machines? Thank you so much.

", "rawMarkdown": "Thank you. That makes sense. You mention, \"...the possibility to train locally.\" Would you be willing to elaborate on this? Could you clarify if people are using programs for 5+ hours continuously on their machines? Thank you so much.", "votes": 2 }, { "id": 3191447, "postDate": "2025-05-01T18:27:20.770Z", "content": "

Yes. Some of the competitors are definitely training models outside of Kaggle, on local PCs or some web services, running 5+ hours of training if necessary.

", "rawMarkdown": "Yes. Some of the competitors are definitely training models outside of Kaggle, on local PCs or some web services, running 5+ hours of training if necessary." }, { "id": 3191537, "postDate": "2025-05-01T20:20:40.927Z", "content": "
\n

running 5+ hours of training if necessary.

\n
\n

More ๐Ÿ˜‰

", "rawMarkdown": ">running 5+ hours of training if necessary. \n\nMore ๐Ÿ˜‰", "votes": 2 }, { "id": 3191722, "postDate": "2025-05-02T04:43:10.943Z", "content": "

May I ask about your workflow? Do you have a second PC specifically for this?

", "rawMarkdown": "May I ask about your workflow? Do you have a second PC specifically for this? ", "votes": 1 }, { "id": 3193011, "postDate": "2025-05-03T17:20:02.077Z", "content": "

May I ask for how many epochs your trained for the 35 LB score? Will likely be able to scale to sub 50 using 200 epochs, but anything sub 40 seems out of reach for now.

", "rawMarkdown": "May I ask for how many epochs your trained for the 35 LB score? Will likely be able to scale to sub 50 using 200 epochs, but anything sub 40 seems out of reach for now.", "votes": 1 }, { "id": 3193014, "postDate": "2025-05-03T17:23:00.817Z", "content": "

Yes, I suspect that anyone seriously training on a 300k+ sample dataset has access to dedicated compute resources. In my case, I have access to academic clusters.

", "rawMarkdown": "Yes, I suspect that anyone seriously training on a 300k+ sample dataset has access to dedicated compute resources. In my case, I have access to academic clusters.", "votes": 2 }, { "id": 3200527, "postDate": "2025-05-12T17:55:14.347Z", "content": "

Yes I guess! A full data run for 10 epochs gave much better results then half data 20 epochs.

", "rawMarkdown": "Yes I guess! A full data run for 10 epochs gave much better results then half data 20 epochs." } ] } ] }, { "id": 3191469, "postDate": "2025-05-01T18:53:59.710Z", "content": "

Thanks for sharing this notebook @egortrushin ! It has been a huge inspiration and very good idea to convert to float16

", "rawMarkdown": "Thanks for sharing this notebook @egortrushin ! It has been a huge inspiration and very good idea to convert to float16", "votes": 2 }, { "id": 3192472, "postDate": "2025-05-02T22:07:33.767Z", "content": "

This is indeed hepful. Thank you very much.

", "rawMarkdown": "This is indeed hepful. Thank you very much.", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 3190306, "author_name": "Egor Trushin", "author_url": "", "post_date": "2025-04-30T13:04:27.857000", "content": "

I have shared an updated notebook that uses the dataset converted from float32 to float16. This allows us to train twice faster. Using half of the openWFI dataset, an improved LB score of 93.2 was achieved using the same model as in the previous version and in the same training time.

", "votes": 7, "replies": [ { "id": 3190460, "author_name": "Prahas Duggireddy", "author_url": "", "post_date": "2025-04-30T17:29:02.147000", "content": "

Would you say that the most viable way to reduce MAE is by utilizing as much training data as possible? I've been looking at the public notebooks, and that looks like the main strength of some of them.

", "votes": 2, "replies": [ { "id": 3190494, "author_name": "Egor Trushin", "author_url": "", "post_date": "2025-04-30T18:20:43.130000", "content": "

It looks like one need to use as much data as possible to reduce MAE, see the discussion here: Scaling up results- Kaggle data vs full data. Of course, other things like NN architecture, hyperparameters, etc. also play a role.

", "votes": 1, "replies": [] }, { "id": 3190582, "author_name": "Prahas Duggireddy", "author_url": "", "post_date": "2025-04-30T21:42:51.787000", "content": "

Thank you. That makes sense. You mention, \"โ€ฆthe possibility to train locally.\" Would you be willing to elaborate on this? Could you clarify if people are using programs for 5+ hours continuously on their machines? Thank you so much.

", "votes": 2, "replies": [] }, { "id": 3191447, "author_name": "Egor Trushin", "author_url": "", "post_date": "2025-05-01T18:27:20.770000", "content": "

Yes. Some of the competitors are definitely training models outside of Kaggle, on local PCs or some web services, running 5+ hours of training if necessary.

", "votes": 0, "replies": [] }, { "id": 3191537, "author_name": "greySnow", "author_url": "", "post_date": "2025-05-01T20:20:40.927000", "content": "
\n

running 5+ hours of training if necessary.

\n
\n

More ๐Ÿ˜‰

", "votes": 2, "replies": [] }, { "id": 3191722, "author_name": "Prahas Duggireddy", "author_url": "", "post_date": "2025-05-02T04:43:10.943000", "content": "

May I ask about your workflow? Do you have a second PC specifically for this?

", "votes": 1, "replies": [] }, { "id": 3193011, "author_name": "Sebastian Hoffmann", "author_url": "", "post_date": "2025-05-03T17:20:02.077000", "content": "

May I ask for how many epochs your trained for the 35 LB score? Will likely be able to scale to sub 50 using 200 epochs, but anything sub 40 seems out of reach for now.

", "votes": 1, "replies": [] }, { "id": 3193014, "author_name": "Sebastian Hoffmann", "author_url": "", "post_date": "2025-05-03T17:23:00.817000", "content": "

Yes, I suspect that anyone seriously training on a 300k+ sample dataset has access to dedicated compute resources. In my case, I have access to academic clusters.

", "votes": 2, "replies": [] }, { "id": 3200527, "author_name": "Green Kedia", "author_url": "", "post_date": "2025-05-12T17:55:14.347000", "content": "

Yes I guess! A full data run for 10 epochs gave much better results then half data 20 epochs.

", "votes": 0, "replies": [] } ] } ] }, { "id": 3191469, "author_name": "Octavi Grau", "author_url": "", "post_date": "2025-05-01T18:53:59.710000", "content": "

Thanks for sharing this notebook @egortrushin ! It has been a huge inspiration and very good idea to convert to float16

", "votes": 2, "replies": [] }, { "id": 3192472, "author_name": "", "author_url": "", "post_date": "2025-05-02T22:07:33.767000", "content": "

This is indeed hepful. Thank you very much.

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "3187363": "Hello everyone,\n\nTo contribute to the overall progress in the competition, I have published a notebook where UNet model is employed and a part of the full OpenFWI dataset is used for training. LB of 107.5 was achieved, check [corresponding notebook](https://www.kaggle.com/code/egortrushin/gwi-improved-unet-pipepline-with-larger-dataset). LB can be improved by increasing the number of epochs, amount of data used, etc. Especially if you have the possibility to train locally.\n\nFeel free to report bugs or possible improvements. I plan to introduce updates later.\n\nGood luck in the competition.\n\n\n**Notebooks**\n1. [[GWI] Improved UNet pipepline with larger dataset](https://www.kaggle.com/code/egortrushin/gwi-improved-unet-pipepline-with-larger-dataset)\n1. [[GWI] UNet with float16 dataset](https://www.kaggle.com/code/egortrushin/gwi-unet-with-float16-dataset)\n\n**float16 datasets**\n1. [openfwi_float16_1](https://www.kaggle.com/datasets/egortrushin/open-wfi-1)\n1. [openfwi_float16_2](https://www.kaggle.com/datasets/egortrushin/open-wfi-2)\n1. [openfwi_float16_test](https://www.kaggle.com/datasets/egortrushin/open-wfi-test)", "3190306": "I have shared [an updated notebook](https://www.kaggle.com/code/egortrushin/gwi-unet-with-float16-dataset) that uses the dataset converted from float32 to float16. This allows us to train twice faster. Using half of the openWFI dataset, an improved LB score of 93.2 was achieved using the same model as in the previous version and in the same training time.", "3191469": "Thanks for sharing this notebook @egortrushin ! It has been a huge inspiration and very good idea to convert to float16", "3192472": "This is indeed hepful. Thank you very much." } }