Hi, this is my first Kaggle competition and I'm having troubles saving files and recovering them later in order to save time.
\nIn this specific case, I do some pre processing on the images and then I want to save the new images on the kaggle/working directory. After doing it with the proper Python command ( ax.imwrite() ), I commit the kernel with \"Save and run all\" and set the advanced options to \"always save output\". But when I log out from Kaggle and then log in again… It all has disappeared and I have to pre-process everything again, with the code taking close to 1h to run and even more to commit!
It's impossible to work under these conditions… Any advice on how to solve this issue?
", "messages": [ { "id": "1768949", "postDate": "04/26/2022 19:07:28", "content": "Hi, this is my first Kaggle competition and I'm having troubles saving files and recovering them later in order to save time.
\nIn this specific case, I do some pre processing on the images and then I want to save the new images on the kaggle/working directory. After doing it with the proper Python command ( ax.imwrite() ), I commit the kernel with \"Save and run all\" and set the advanced options to \"always save output\". But when I log out from Kaggle and then log in again… It all has disappeared and I have to pre-process everything again, with the code taking close to 1h to run and even more to commit!
It's impossible to work under these conditions… Any advice on how to solve this issue?
", "rawMarkdown": "Hi, this is my first Kaggle competition and I'm having troubles saving files and recovering them later in order to save time.\nIn this specific case, I do some pre processing on the images and then I want to save the new images on the kaggle/working directory. After doing it with the proper Python command ( ax.imwrite() ), I commit the kernel with \"Save and run all\" and set the advanced options to \"always save output\". But when I log out from Kaggle and then log in again... It all has disappeared and I have to pre-process everything again, with the code taking close to 1h to run and even more to commit!\n\nIt's impossible to work under these conditions... Any advice on how to solve this issue?", "votes": null }, { "id": "1768961", "postDate": "04/26/2022 19:49:18", "content": "@nevrocerasani When you do \"Save & Run All\" it re-runs your notebook top to bottom in a background copy and creates an immutable version of the notebook that ran, and the files that were written to /kaggle/working.
\nIf you want to recover those files, you can add your saved version notebook output as a dataset with \"Add Data\" in the editor. Then the files will be read-only added as a dataset in /kaggle/input/$notebook_name/.
\nI am currently working on a feature to automatically save/restore files in /kaggle/working in the editor to make use cases like yours easier.
", "rawMarkdown": "nevrocerasani When you do \"Save & Run All\" it re-runs your notebook top to bottom in a background copy and creates an immutable version of the notebook that ran, and the files that were written to /kaggle/working.\n\nIf you want to recover those files, you can add your saved version notebook output as a dataset with \"Add Data\" in the editor. Then the files will be read-only added as a dataset in /kaggle/input/$notebook_name/.\n\nI am currently working on a feature to automatically save/restore files in /kaggle/working in the editor to make use cases like yours easier.", "votes": null }, { "id": "1768965", "postDate": "04/26/2022 19:56:46", "content": "@herbison Thank you for your clear and prompt response!
", "rawMarkdown": "herbison Thank you for your clear and prompt response!", "votes": null } ], "comments": [ { "id": 1768961, "author_name": "herbison", "author_url": "", "post_date": "04/26/2022 19:49:18", "content": "@nevrocerasani When you do \"Save & Run All\" it re-runs your notebook top to bottom in a background copy and creates an immutable version of the notebook that ran, and the files that were written to /kaggle/working.
\nIf you want to recover those files, you can add your saved version notebook output as a dataset with \"Add Data\" in the editor. Then the files will be read-only added as a dataset in /kaggle/input/$notebook_name/.
\nI am currently working on a feature to automatically save/restore files in /kaggle/working in the editor to make use cases like yours easier.
", "votes": null, "replies": [ { "id": 1768965, "author_name": "nevrocerasani", "author_url": "", "post_date": "04/26/2022 19:56:46", "content": "@herbison Thank you for your clear and prompt response!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1768949": "Hi, this is my first Kaggle competition and I'm having troubles saving files and recovering them later in order to save time.\nIn this specific case, I do some pre processing on the images and then I want to save the new images on the kaggle/working directory. After doing it with the proper Python command ( ax.imwrite() ), I commit the kernel with \"Save and run all\" and set the advanced options to \"always save output\". But when I log out from Kaggle and then log in again... It all has disappeared and I have to pre-process everything again, with the code taking close to 1h to run and even more to commit!\n\nIt's impossible to work under these conditions... Any advice on how to solve this issue?", "1768961": "nevrocerasani When you do \"Save & Run All\" it re-runs your notebook top to bottom in a background copy and creates an immutable version of the notebook that ran, and the files that were written to /kaggle/working.\n\nIf you want to recover those files, you can add your saved version notebook output as a dataset with \"Add Data\" in the editor. Then the files will be read-only added as a dataset in /kaggle/input/$notebook_name/.\n\nI am currently working on a feature to automatically save/restore files in /kaggle/working in the editor to make use cases like yours easier.", "1768965": "herbison Thank you for your clear and prompt response!" }, "source": "meta" }