Hi, I am currently trying to understand how the Kagglers at the top lb are able to load their model weights into their inference notebooks without publishing them, as I want to use my own. I assume that they are not training their models in their submissions, meaning their weights are external.
\nThis discussion (https://www.kaggle.com/c/connectx/discussion/126678) explains a way to hard code the weights in. I tried this method but my model is large and I'm not sure that I can just copy paste the encoded weights in.
\nI was also thinking that they might just submit a notebook with a copy pasted csv file for the public validation data set, and then at the last minute publish their weights so it can run on the private data set. However, this method seems like it could be unreliable, as the weights might take some time to upload.
\nThank you for your help!
", "messages": [ { "id": "1395324", "postDate": "07/21/2021 06:21:05", "content": "Hi, I am currently trying to understand how the Kagglers at the top lb are able to load their model weights into their inference notebooks without publishing them, as I want to use my own. I assume that they are not training their models in their submissions, meaning their weights are external.
\nThis discussion (https://www.kaggle.com/c/connectx/discussion/126678) explains a way to hard code the weights in. I tried this method but my model is large and I'm not sure that I can just copy paste the encoded weights in.
\nI was also thinking that they might just submit a notebook with a copy pasted csv file for the public validation data set, and then at the last minute publish their weights so it can run on the private data set. However, this method seems like it could be unreliable, as the weights might take some time to upload.
\nThank you for your help!
", "rawMarkdown": "Hi, I am currently trying to understand how the Kagglers at the top lb are able to load their model weights into their inference notebooks without publishing them, as I want to use my own. I assume that they are not training their models in their submissions, meaning their weights are external. \n\nThis discussion (https://www.kaggle.com/c/connectx/discussion/126678) explains a way to hard code the weights in. I tried this method but my model is large and I'm not sure that I can just copy paste the encoded weights in.\n\nI was also thinking that they might just submit a notebook with a copy pasted csv file for the public validation data set, and then at the last minute publish their weights so it can run on the private data set. However, this method seems like it could be unreliable, as the weights might take some time to upload.\n\nThank you for your help!", "votes": null }, { "id": "1395771", "postDate": "07/21/2021 14:22:20", "content": "You can load your model weights as a dataset and include them in your notebook.
", "rawMarkdown": "You can load your model weights as a dataset and include them in your notebook.", "votes": null }, { "id": "1396073", "postDate": "07/21/2021 18:25:45", "content": "Yes, I understand that you can load your model weights from a dataset. However, as I have interpreted it, this dataset must be public. If you look at the profiles of Kagglers at the top lb, they have not posted their weights, yet have high scores. My question is how do they do this.
", "rawMarkdown": "Yes, I understand that you can load your model weights from a dataset. However, as I have interpreted it, this dataset must be public. If you look at the profiles of Kagglers at the top lb, they have not posted their weights, yet have high scores. My question is how do they do this.", "votes": null }, { "id": "1396605", "postDate": "07/22/2021 09:10:07", "content": "The dataset doesn't need to be public only external data meaning images need to be public
", "rawMarkdown": "The dataset doesn't need to be public only external data meaning images need to be public", "votes": null } ], "comments": [ { "id": 1395771, "author_name": "davidbroberts", "author_url": "", "post_date": "07/21/2021 14:22:20", "content": "You can load your model weights as a dataset and include them in your notebook.
", "votes": null, "replies": [ { "id": 1396073, "author_name": "soondubu", "author_url": "", "post_date": "07/21/2021 18:25:45", "content": "Yes, I understand that you can load your model weights from a dataset. However, as I have interpreted it, this dataset must be public. If you look at the profiles of Kagglers at the top lb, they have not posted their weights, yet have high scores. My question is how do they do this.
", "votes": null, "replies": [] }, { "id": 1396605, "author_name": "simon111", "author_url": "", "post_date": "07/22/2021 09:10:07", "content": "The dataset doesn't need to be public only external data meaning images need to be public
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1395324": "Hi, I am currently trying to understand how the Kagglers at the top lb are able to load their model weights into their inference notebooks without publishing them, as I want to use my own. I assume that they are not training their models in their submissions, meaning their weights are external. \n\nThis discussion (https://www.kaggle.com/c/connectx/discussion/126678) explains a way to hard code the weights in. I tried this method but my model is large and I'm not sure that I can just copy paste the encoded weights in.\n\nI was also thinking that they might just submit a notebook with a copy pasted csv file for the public validation data set, and then at the last minute publish their weights so it can run on the private data set. However, this method seems like it could be unreliable, as the weights might take some time to upload.\n\nThank you for your help!", "1395771": "You can load your model weights as a dataset and include them in your notebook.", "1396073": "Yes, I understand that you can load your model weights from a dataset. However, as I have interpreted it, this dataset must be public. If you look at the profiles of Kagglers at the top lb, they have not posted their weights, yet have high scores. My question is how do they do this.", "1396605": "The dataset doesn't need to be public only external data meaning images need to be public" }, "source": "meta" }