I have tried with some public notebooks to generate the same results but failed. Though the output of those notebooks yields the same LB score, If I try to commit the notebook then it can't produce the same results. Moreover, it produces very poor results. Does an else facing this problem??\n* Melanoma EfficientNetB6 inference[LB:.89 -> LB:.84]\n* Melanoma TPU EfficientNet B5_dense_head[LB:91 -> LB:.85]
I have tried with some public notebooks to generate the same results but failed. Though the output of those notebooks yields the same LB score, If I try to commit the notebook then it can't produce the same results. Moreover, it produces very poor results. Does an else facing this problem??\n* Melanoma EfficientNetB6 inference[LB:.89 -> LB:.84]\n* Melanoma TPU EfficientNet B5_dense_head[LB:91 -> LB:.85]
Random seeds are not fixed.
", "rawMarkdown": "Random seeds are not fixed.", "votes": null }, { "id": "883664", "postDate": "06/12/2020 20:14:15", "content": "I have tried for quite some time with different notebooks and they are showing deviating and very poor result..
", "rawMarkdown": "I have tried for quite some time with different notebooks and they are showing deviating and very poor result..", "votes": null }, { "id": "883690", "postDate": "06/12/2020 20:37:03", "content": "I think that it may be because LB score is calculated with approx. 30% of the data.
", "rawMarkdown": "I think that it may be because LB score is calculated with approx. 30% of the data.", "votes": null }, { "id": "883693", "postDate": "06/12/2020 20:44:19", "content": "I think 30% data is fixed so score should be same for all submission
", "rawMarkdown": "I think 30% data is fixed so score should be same for all submission", "votes": null }, { "id": "883782", "postDate": "06/12/2020 23:12:30", "content": "Hi, indeed results obtained with multi-core TPUs aren't deterministic (see discussion here). This is why it is important to save models and results each time you run a multi-core TPU kernel.
\n\nYou've just not been lucky enough if you had worse results for both kernels, or on the contrary, the authors of the kernels you're sharing have been lucky enough to have good results for the versions they shared.
", "rawMarkdown": "Hi, indeed results obtained with multi-core TPUs aren't deterministic (see discussion [here](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/140706)). This is why it is important to save models and results each time you run a multi-core TPU kernel.\n\nYou've just not been lucky enough if you had worse results for both kernels, or on the contrary, the authors of the kernels you're sharing have been lucky enough to have good results for the versions they shared.", "votes": null } ], "comments": [ { "id": 883616, "author_name": "stanislavblinov", "author_url": "", "post_date": "06/12/2020 19:20:20", "content": "Random seeds are not fixed.
", "votes": null, "replies": [ { "id": 883664, "author_name": "awsaf49", "author_url": "", "post_date": "06/12/2020 20:14:15", "content": "I have tried for quite some time with different notebooks and they are showing deviating and very poor result..
", "votes": null, "replies": [] } ] }, { "id": 883690, "author_name": "janmalin", "author_url": "", "post_date": "06/12/2020 20:37:03", "content": "I think that it may be because LB score is calculated with approx. 30% of the data.
", "votes": null, "replies": [ { "id": 883693, "author_name": "awsaf49", "author_url": "", "post_date": "06/12/2020 20:44:19", "content": "I think 30% data is fixed so score should be same for all submission
", "votes": null, "replies": [] } ] }, { "id": 883782, "author_name": "mika30", "author_url": "", "post_date": "06/12/2020 23:12:30", "content": "Hi, indeed results obtained with multi-core TPUs aren't deterministic (see discussion here). This is why it is important to save models and results each time you run a multi-core TPU kernel.
\n\nYou've just not been lucky enough if you had worse results for both kernels, or on the contrary, the authors of the kernels you're sharing have been lucky enough to have good results for the versions they shared.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "883512": "I have tried with some public notebooks to generate the same results but failed. Though the output of those notebooks yields the same LB score, If I try to commit the notebook then it can't produce the same results. Moreover, it produces very poor results. Does an else facing this problem??\n* [Melanoma EfficientNetB6 inference](https://www.kaggle.com/soham1024/melanoma-efficientnetb6-inference)`[LB:.89 -> LB:.84]`\n* [Melanoma TPU EfficientNet B5_dense_head](https://www.kaggle.com/ajaykumar7778/melanoma-tpu-efficientnet-b5-dense-head)`[LB:91 -> LB:.85]`", "883616": "Random seeds are not fixed.", "883664": "I have tried for quite some time with different notebooks and they are showing deviating and very poor result..", "883690": "I think that it may be because LB score is calculated with approx. 30% of the data.", "883693": "I think 30% data is fixed so score should be same for all submission", "883782": "Hi, indeed results obtained with multi-core TPUs aren't deterministic (see discussion [here](https://www.kaggle.com/c/jigsaw-multilingual-toxic-comment-classification/discussion/140706)). This is why it is important to save models and results each time you run a multi-core TPU kernel.\n\nYou've just not been lucky enough if you had worse results for both kernels, or on the contrary, the authors of the kernels you're sharing have been lucky enough to have good results for the versions they shared." }, "source": "meta" }