\n", "messages": [ { "id": "3224121", "postDate": "06/14/2025 12:30:07", "content": "Tried different losses / models but not able to break the Bartley's strong base LB. Improved speed of training with 'training speed discussion post'
\n
\n Target normalize / target scale are learning rate too slow then without target normalization.
\n", "rawMarkdown": "> Tried different losses / models but not able to break the Bartley's strong base LB. Improved speed of training with 'training speed discussion post'\n> Target normalize / target scale are learning rate too slow then without target normalization.", "votes": null }, { "id": "3224192", "postDate": "06/14/2025 14:23:45", "content": "Tried different losses / models but not able to break the Bartley's strong base LB. Improved speed of training with 'training speed discussion post'
\n
\n Target normalize / target scale are learning rate too slow then without target normalization.
if using target normalization:
\n sklearn.preprocessing StandardScaler\ny_scaler = StandardScaler()\ny_train_scaled = y_scaler.fit_transform(y_train.reshape(-, )).ravel()\nmake sure to:
\n1e-3 or 5e-4 if youre using MSE loss).y_train = np.log1p(y_train)\nIs the 28.8 LB score the best we can do by predicting a float16 velocity map? Could someone confirm that the they are able to get a better score predicting in float16? I think I am wasting my time trying to fine-tune a model that predicts float16 velocity maps, and that I am not able to get better scores just because the precision is not enough. I highly doubt this is the problem, but would be nice to hear other experiences.
", "rawMarkdown": "Is the 28.8 LB score the best we can do by predicting a float16 velocity map? Could someone confirm that the they are able to get a better score predicting in float16? I think I am wasting my time trying to fine-tune a model that predicts float16 velocity maps, and that I am not able to get better scores just because the precision is not enough. I highly doubt this is the problem, but would be nice to hear other experiences.", "votes": null }, { "id": "3224770", "postDate": "06/15/2025 13:07:35", "content": "Even though the CV score of my improved model improved by 3 on float16, I ran into a huge problem where its predictions were all integers! I'm not sure if it's better to use float32
", "rawMarkdown": "Even though the CV score of my improved model improved by 3 on float16, I ran into a huge problem where its predictions were all integers! I'm not sure if it's better to use float32", "votes": null }, { "id": "3224898", "postDate": "06/15/2025 16:11:51", "content": "May I ask a question? Were your capabilities enhanced through architectural modifications to a pre-trained model, or primarily through fine-tuning? Thank you kindly.ðŸ˜
", "rawMarkdown": "May I ask a question? Were your capabilities enhanced through architectural modifications to a pre-trained model, or primarily through fine-tuning? Thank you kindly.ðŸ˜", "votes": null }, { "id": "3224930", "postDate": "06/15/2025 17:11:41", "content": "My feeling is that one should use network prediction as initial solution then refine using physics differential equation. That should solve the integer value problem
", "rawMarkdown": "My feeling is that one should use network prediction as initial solution then refine using physics differential equation. That should solve the integer value problem", "votes": null }, { "id": "3224931", "postDate": "06/15/2025 17:12:53", "content": "Think of how to use the test data in training! masked auto encoders … using pyshics for self/unsupervised, refining ….
", "rawMarkdown": "Think of how to use the test data in training! masked auto encoders ... using pyshics for self/unsupervised, refining ....", "votes": null }, { "id": "3225083", "postDate": "06/16/2025 00:59:15", "content": "note that this problem is not the same as regular regression problem. because you have the wave forward modeling eqn to check the correctness of your current prediction. so the final solution could be additive or sequential, i.e. you need not retrain and retrain to get best solution.
\na winning solution could be:
\nThank you very much for your suggestion. I think it seems like a lot of work for me. I have only seen in machine learning before using the output of a model as input to another model, which has increased my knowledge.
\nI would like to ask, if err1=error (y1, y) and y1 and err1 are directly used as inputs for the new model, do you think this is different from the principle of using x1 and err0 as new inputs，and do you think this two-stage model improvement is certain?
You don't what is the true target y of test. So y cannot compute target error at test. You can only compute input reconstruction error.
", "rawMarkdown": "You don't what is the true target y of test. So y cannot compute target error at test. You can only compute input reconstruction error.", "votes": null }, { "id": "3225253", "postDate": "06/16/2025 07:04:35", "content": "Thank you for your suggestion.😘
", "rawMarkdown": "Thank you for your suggestion.😘", "votes": null }, { "id": "3225318", "postDate": "06/16/2025 08:59:18", "content": "the trick to learn iterative refinition is to have lots of data. if each \"new model input = (x,err0)\" is trained with new batch of data, results will be very good.
\nactuall you just need one
\n <-- train on (,err_t)\nand then use it iteratively.
\nyou can think of this is as data-driven version of the update step whent you solve differentiable equation iteratively and mathemtically
", "rawMarkdown": "the trick to learn iterative refinition is to have lots of data. if each \"new model input = (x,err0)\" is trained with new batch of data, results will be very good.\n\nactuall you just need one \n```\nmodel_(t+1) <-- train on (x,err_t)\n```\n\nand then use it iteratively.\n\nyou can think of this is as data-driven version of the update step whent you solve differentiable equation iteratively and mathemtically", "votes": null } ], "comments": [ { "id": 3224192, "author_name": "dakshbhatnagar08", "author_url": "", "post_date": "06/14/2025 14:23:45", "content": "if using target normalization:
\n sklearn.preprocessing StandardScaler\ny_scaler = StandardScaler()\ny_train_scaled = y_scaler.fit_transform(y_train.reshape(-, )).ravel()\nmake sure to:
\n1e-3 or 5e-4 if youre using MSE loss).y_train = np.log1p(y_train)\nIs the 28.8 LB score the best we can do by predicting a float16 velocity map? Could someone confirm that the they are able to get a better score predicting in float16? I think I am wasting my time trying to fine-tune a model that predicts float16 velocity maps, and that I am not able to get better scores just because the precision is not enough. I highly doubt this is the problem, but would be nice to hear other experiences.
", "votes": null, "replies": [ { "id": 3224770, "author_name": "oliver34", "author_url": "", "post_date": "06/15/2025 13:07:35", "content": "Even though the CV score of my improved model improved by 3 on float16, I ran into a huge problem where its predictions were all integers! I'm not sure if it's better to use float32
", "votes": null, "replies": [ { "id": 3224898, "author_name": "z0x0zn", "author_url": "", "post_date": "06/15/2025 16:11:51", "content": "May I ask a question? Were your capabilities enhanced through architectural modifications to a pre-trained model, or primarily through fine-tuning? Thank you kindly.ðŸ˜
", "votes": null, "replies": [] }, { "id": 3224930, "author_name": "hengck23", "author_url": "", "post_date": "06/15/2025 17:11:41", "content": "My feeling is that one should use network prediction as initial solution then refine using physics differential equation. That should solve the integer value problem
", "votes": null, "replies": [ { "id": 3225083, "author_name": "hengck23", "author_url": "", "post_date": "06/16/2025 00:59:15", "content": "note that this problem is not the same as regular regression problem. because you have the wave forward modeling eqn to check the correctness of your current prediction. so the final solution could be additive or sequential, i.e. you need not retrain and retrain to get best solution.
\na winning solution could be:
\nThank you very much for your suggestion. I think it seems like a lot of work for me. I have only seen in machine learning before using the output of a model as input to another model, which has increased my knowledge.
\nI would like to ask, if err1=error (y1, y) and y1 and err1 are directly used as inputs for the new model, do you think this is different from the principle of using x1 and err0 as new inputs，and do you think this two-stage model improvement is certain?
You don't what is the true target y of test. So y cannot compute target error at test. You can only compute input reconstruction error.
", "votes": null, "replies": [] } ] }, { "id": 3225253, "author_name": "z0x0zn", "author_url": "", "post_date": "06/16/2025 07:04:35", "content": "Thank you for your suggestion.😘
", "votes": null, "replies": [ { "id": 3225318, "author_name": "hengck23", "author_url": "", "post_date": "06/16/2025 08:59:18", "content": "the trick to learn iterative refinition is to have lots of data. if each \"new model input = (x,err0)\" is trained with new batch of data, results will be very good.
\nactuall you just need one
\n <-- train on (,err_t)\nand then use it iteratively.
\nyou can think of this is as data-driven version of the update step whent you solve differentiable equation iteratively and mathemtically
", "votes": null, "replies": [] } ] } ] } ] } ] } ] }, { "id": 3224931, "author_name": "hengck23", "author_url": "", "post_date": "06/15/2025 17:12:53", "content": "Think of how to use the test data in training! masked auto encoders … using pyshics for self/unsupervised, refining ….
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "3224121": "> Tried different losses / models but not able to break the Bartley's strong base LB. Improved speed of training with 'training speed discussion post'\n> Target normalize / target scale are learning rate too slow then without target normalization.", "3224192": "if using target normalization:\n\n```py\nfrom sklearn.preprocessing import StandardScaler\ny_scaler = StandardScaler()\ny_train_scaled = y_scaler.fit_transform(y_train.reshape(-1, 1)).ravel()\n```\n\nmake sure to:\n\n- use inverse_transform after prediction.\n- adjust learning rate: target scaling often needs lower LR (like `1e-3 or 5e-4` if youre using MSE loss).\n- log-transform targets if the distribution is skewed. you can try:\n\n```py\ny_train = np.log1p(y_train)\n```", "3224314": "Is the 28.8 LB score the best we can do by predicting a float16 velocity map? Could someone confirm that the they are able to get a better score predicting in float16? I think I am wasting my time trying to fine-tune a model that predicts float16 velocity maps, and that I am not able to get better scores just because the precision is not enough. I highly doubt this is the problem, but would be nice to hear other experiences.", "3224770": "Even though the CV score of my improved model improved by 3 on float16, I ran into a huge problem where its predictions were all integers! I'm not sure if it's better to use float32", "3224898": "May I ask a question? Were your capabilities enhanced through architectural modifications to a pre-trained model, or primarily through fine-tuning? Thank you kindly.ðŸ˜", "3224930": "My feeling is that one should use network prediction as initial solution then refine using physics differential equation. That should solve the integer value problem", "3224931": "Think of how to use the test data in training! masked auto encoders ... using pyshics for self/unsupervised, refining ....", "3225083": "note that this problem is not the same as regular regression problem. because you have the wave forward modeling eqn to check the correctness of your current prediction. so the final solution could be additive or sequential, i.e. you need not retrain and retrain to get best solution.\n\na winning solution could be:\n\n1. we have current model m\n2. make prediction on test y0 = m0(x)\n3. measure fitness of y0, e.g. x0=wave(y0), err0 = error(x0,x)\n4. build a new model to improve: new model input = (x,err0)\n5. so next time, y1 = m1(x,err0)\n6. repeat", "3225128": "Thank you very much for your suggestion. I think it seems like a lot of work for me. I have only seen in machine learning before using the output of a model as input to another model, which has increased my knowledge. \nI would like to ask, if err1=error (y1, y) and y1 and err1 are directly used as inputs for the new model, do you think this is different from the principle of using x1 and err0 as new inputs，and do you think this two-stage model improvement is certain?", "3225133": "You don't what is the true target y of test. So y cannot compute target error at test. You can only compute input reconstruction error.", "3225253": "Thank you for your suggestion.😘", "3225318": "the trick to learn iterative refinition is to have lots of data. if each \"new model input = (x,err0)\" is trained with new batch of data, results will be very good.\n\nactuall you just need one \n```\nmodel_(t+1) <-- train on (x,err_t)\n```\n\nand then use it iteratively.\n\nyou can think of this is as data-driven version of the update step whent you solve differentiable equation iteratively and mathemtically" }, "source": "meta" }