This is my first Code Competition, and I just recently joined, so I am acclimating myself to the online notebook environment and how it differs from the setup I use on my own machines. I am testing an R script that is supposed to eventually produce a submission file. When I upload the script and press the \"Run All\" button, I can tell from the Console messages that my script is running, but it seems to go on \"forever\" without producing further output. This is probably due to a bug in my code, which I can certainly investigate, but the weird thing is that the \"Cancel run\" button appears active and clickable but doesn't seem to do anything when I press it. I have to press \"Stop session\" to get it to stop. Meanwhile, \"View Session Metrics\" shows \"CPU 0.00%\", suggesting that whatever state my run is in it's not hung on the CPU.
\nAny ideas on what is going on with the \"Cancel run\" button?
\nThanks.
", "messages": [ { "id": "995614", "postDate": "09/02/2020 16:00:51", "content": "This is my first Code Competition, and I just recently joined, so I am acclimating myself to the online notebook environment and how it differs from the setup I use on my own machines. I am testing an R script that is supposed to eventually produce a submission file. When I upload the script and press the \"Run All\" button, I can tell from the Console messages that my script is running, but it seems to go on \"forever\" without producing further output. This is probably due to a bug in my code, which I can certainly investigate, but the weird thing is that the \"Cancel run\" button appears active and clickable but doesn't seem to do anything when I press it. I have to press \"Stop session\" to get it to stop. Meanwhile, \"View Session Metrics\" shows \"CPU 0.00%\", suggesting that whatever state my run is in it's not hung on the CPU.
\nAny ideas on what is going on with the \"Cancel run\" button?
\nThanks.
", "rawMarkdown": "This is my first Code Competition, and I just recently joined, so I am acclimating myself to the online notebook environment and how it differs from the setup I use on my own machines. I am testing an R script that is supposed to eventually produce a submission file. When I upload the script and press the \"Run All\" button, I can tell from the Console messages that my script is running, but it seems to go on \"forever\" without producing further output. This is probably due to a bug in my code, which I can certainly investigate, but the weird thing is that the \"Cancel run\" button appears active and clickable but doesn't seem to do anything when I press it. I have to press \"Stop session\" to get it to stop. Meanwhile, \"View Session Metrics\" shows \"CPU 0.00%\", suggesting that whatever state my run is in it's not hung on the CPU.\n\nAny ideas on what is going on with the \"Cancel run\" button?\n\nThanks.", "votes": null }, { "id": "995679", "postDate": "09/02/2020 17:05:34", "content": "@dslate This has happened to me a few times when using Jupyter as the notebook environment. From what i've noticed is that this happens when a cell block is executing memory heavy operation and runs into an error. I'm not sure if it's a Kaggle bug or simply because the entire notebook freezes due to the heavy task.
\nThe only way i found to fix the non-responsive run/cancel button was to refresh the page without restarting the notebook session although the output of already run cells disappear.
", "rawMarkdown": "dslate This has happened to me a few times when using Jupyter as the notebook environment. From what i've noticed is that this happens when a cell block is executing memory heavy operation and runs into an error. I'm not sure if it's a Kaggle bug or simply because the entire notebook freezes due to the heavy task.\n\nThe only way i found to fix the non-responsive run/cancel button was to refresh the page without restarting the notebook session although the output of already run cells disappear.", "votes": null }, { "id": "995776", "postDate": "09/02/2020 19:00:06", "content": "Thanks Yovin for your response. In my case when my run hangs and the \"Cancel run\" button becomes inoperative my RAM use shows only 1.2GB out of Max 16GB, so it's not memory use that is causing the problem.
", "rawMarkdown": "Thanks Yovin for your response. In my case when my run hangs and the \"Cancel run\" button becomes inoperative my RAM use shows only 1.2GB out of Max 16GB, so it's not memory use that is causing the problem.", "votes": null }, { "id": "1005842", "postDate": "09/10/2020 19:59:47", "content": "Hi @dslate - sorry you're having trouble. I'm not able to reproduce this issue. Do you see any errors in the developer console? If you would be willing to share the notebook with me, I could take a look.
", "rawMarkdown": "Hi @dslate - sorry you're having trouble. I'm not able to reproduce this issue. Do you see any errors in the developer console? If you would be willing to share the notebook with me, I could take a look.", "votes": null }, { "id": "1005882", "postDate": "09/10/2020 20:49:25", "content": "Thanks Jim Plotts for your reply. Turns out I had some bugs in my code, and after I fixed them the problem went away and my runs completed successfully. My speculative hypothesis is that the bugs manifested themselves in subprocesses that I had spun off via mclapply, possibly resulting in \"zombie\" processes that the parent was waiting around forever for.
", "rawMarkdown": "Thanks Jim Plotts for your reply. Turns out I had some bugs in my code, and after I fixed them the problem went away and my runs completed successfully. My speculative hypothesis is that the bugs manifested themselves in subprocesses that I had spun off via mclapply, possibly resulting in \"zombie\" processes that the parent was waiting around forever for.", "votes": null }, { "id": "1006662", "postDate": "09/11/2020 13:18:04", "content": "Ah, glad you got it worked out. Thanks for the update!
", "rawMarkdown": "Ah, glad you got it worked out. Thanks for the update!", "votes": null }, { "id": "1008396", "postDate": "09/13/2020 05:00:58", "content": "Hi, @jplotts, here is another update regarding this issue:
\nI've had some more R script runs hang idle indefinitely with the \"Cancel run\" button ineffective. These again appear to be due to subprocesses crashing, this time because all calls to the function fslr::fslentropy are failing. The same calls succeed on my own machines, but of course my environment is different from that of the online R notebooks. In particular, although R package fslr is installed, as well as its R package dependencies, it looks like the required fsl system support is lacking. But that's okay, I can proceed without the fslr functions. However, the \"Cancel run\" button seems not to work even with runs that are still actively using the CPU, so I wonder whether it ever works at all. Perhaps that is something you could look into.
\nAlso, one thing that made it harder for me to debug what my hung runs were doing is that apparently it is not possible to enter R commands in the Console while a script is running, even if that script has stopped doing anything useful. Perhaps that is intentional, although I'm not sure why.
", "rawMarkdown": "Hi, @jplotts, here is another update regarding this issue:\n\nI've had some more R script runs hang idle indefinitely with the \"Cancel run\" button ineffective. These again appear to be due to subprocesses crashing, this time because all calls to the function fslr::fslentropy are failing. The same calls succeed on my own machines, but of course my environment is different from that of the online R notebooks. In particular, although R package fslr is installed, as well as its R package dependencies, it looks like the required fsl system support is lacking. But that's okay, I can proceed without the fslr functions. However, the \"Cancel run\" button seems not to work even with runs that are still actively using the CPU, so I wonder whether it ever works at all. Perhaps that is something you could look into.\n\nAlso, one thing that made it harder for me to debug what my hung runs were doing is that apparently it is not possible to enter R commands in the Console while a script is running, even if that script has stopped doing anything useful. Perhaps that is intentional, although I'm not sure why.", "votes": null }, { "id": "1009369", "postDate": "09/13/2020 23:27:05", "content": "One more point about this issue: on my own machines (where I run Ubuntu Linux), when all the child processes launched by R function mclapply from my program's parent process terminate, the parent continues execution, detects the failures, and terminates as well. But in your online notebook environment it seems like mclapply never returns, and the parent process continues idly waiting for the child processes to finish. If this is true, it's kind of weird.
", "rawMarkdown": "One more point about this issue: on my own machines (where I run Ubuntu Linux), when all the child processes launched by R function mclapply from my program's parent process terminate, the parent continues execution, detects the failures, and terminates as well. But in your online notebook environment it seems like mclapply never returns, and the parent process continues idly waiting for the child processes to finish. If this is true, it's kind of weird.", "votes": null } ], "comments": [ { "id": 995679, "author_name": "yovinyahathugoda", "author_url": "", "post_date": "09/02/2020 17:05:34", "content": "@dslate This has happened to me a few times when using Jupyter as the notebook environment. From what i've noticed is that this happens when a cell block is executing memory heavy operation and runs into an error. I'm not sure if it's a Kaggle bug or simply because the entire notebook freezes due to the heavy task.
\nThe only way i found to fix the non-responsive run/cancel button was to refresh the page without restarting the notebook session although the output of already run cells disappear.
", "votes": null, "replies": [ { "id": 995776, "author_name": "dslate", "author_url": "", "post_date": "09/02/2020 19:00:06", "content": "Thanks Yovin for your response. In my case when my run hangs and the \"Cancel run\" button becomes inoperative my RAM use shows only 1.2GB out of Max 16GB, so it's not memory use that is causing the problem.
", "votes": null, "replies": [] } ] }, { "id": 1005842, "author_name": "jplotts", "author_url": "", "post_date": "09/10/2020 19:59:47", "content": "Hi @dslate - sorry you're having trouble. I'm not able to reproduce this issue. Do you see any errors in the developer console? If you would be willing to share the notebook with me, I could take a look.
", "votes": null, "replies": [ { "id": 1005882, "author_name": "dslate", "author_url": "", "post_date": "09/10/2020 20:49:25", "content": "Thanks Jim Plotts for your reply. Turns out I had some bugs in my code, and after I fixed them the problem went away and my runs completed successfully. My speculative hypothesis is that the bugs manifested themselves in subprocesses that I had spun off via mclapply, possibly resulting in \"zombie\" processes that the parent was waiting around forever for.
", "votes": null, "replies": [] }, { "id": 1006662, "author_name": "jplotts", "author_url": "", "post_date": "09/11/2020 13:18:04", "content": "Ah, glad you got it worked out. Thanks for the update!
", "votes": null, "replies": [] }, { "id": 1008396, "author_name": "dslate", "author_url": "", "post_date": "09/13/2020 05:00:58", "content": "Hi, @jplotts, here is another update regarding this issue:
\nI've had some more R script runs hang idle indefinitely with the \"Cancel run\" button ineffective. These again appear to be due to subprocesses crashing, this time because all calls to the function fslr::fslentropy are failing. The same calls succeed on my own machines, but of course my environment is different from that of the online R notebooks. In particular, although R package fslr is installed, as well as its R package dependencies, it looks like the required fsl system support is lacking. But that's okay, I can proceed without the fslr functions. However, the \"Cancel run\" button seems not to work even with runs that are still actively using the CPU, so I wonder whether it ever works at all. Perhaps that is something you could look into.
\nAlso, one thing that made it harder for me to debug what my hung runs were doing is that apparently it is not possible to enter R commands in the Console while a script is running, even if that script has stopped doing anything useful. Perhaps that is intentional, although I'm not sure why.
", "votes": null, "replies": [] }, { "id": 1009369, "author_name": "dslate", "author_url": "", "post_date": "09/13/2020 23:27:05", "content": "One more point about this issue: on my own machines (where I run Ubuntu Linux), when all the child processes launched by R function mclapply from my program's parent process terminate, the parent continues execution, detects the failures, and terminates as well. But in your online notebook environment it seems like mclapply never returns, and the parent process continues idly waiting for the child processes to finish. If this is true, it's kind of weird.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "995614": "This is my first Code Competition, and I just recently joined, so I am acclimating myself to the online notebook environment and how it differs from the setup I use on my own machines. I am testing an R script that is supposed to eventually produce a submission file. When I upload the script and press the \"Run All\" button, I can tell from the Console messages that my script is running, but it seems to go on \"forever\" without producing further output. This is probably due to a bug in my code, which I can certainly investigate, but the weird thing is that the \"Cancel run\" button appears active and clickable but doesn't seem to do anything when I press it. I have to press \"Stop session\" to get it to stop. Meanwhile, \"View Session Metrics\" shows \"CPU 0.00%\", suggesting that whatever state my run is in it's not hung on the CPU.\n\nAny ideas on what is going on with the \"Cancel run\" button?\n\nThanks.", "995679": "dslate This has happened to me a few times when using Jupyter as the notebook environment. From what i've noticed is that this happens when a cell block is executing memory heavy operation and runs into an error. I'm not sure if it's a Kaggle bug or simply because the entire notebook freezes due to the heavy task.\n\nThe only way i found to fix the non-responsive run/cancel button was to refresh the page without restarting the notebook session although the output of already run cells disappear.", "995776": "Thanks Yovin for your response. In my case when my run hangs and the \"Cancel run\" button becomes inoperative my RAM use shows only 1.2GB out of Max 16GB, so it's not memory use that is causing the problem.", "1005842": "Hi @dslate - sorry you're having trouble. I'm not able to reproduce this issue. Do you see any errors in the developer console? If you would be willing to share the notebook with me, I could take a look.", "1005882": "Thanks Jim Plotts for your reply. Turns out I had some bugs in my code, and after I fixed them the problem went away and my runs completed successfully. My speculative hypothesis is that the bugs manifested themselves in subprocesses that I had spun off via mclapply, possibly resulting in \"zombie\" processes that the parent was waiting around forever for.", "1006662": "Ah, glad you got it worked out. Thanks for the update!", "1008396": "Hi, @jplotts, here is another update regarding this issue:\n\nI've had some more R script runs hang idle indefinitely with the \"Cancel run\" button ineffective. These again appear to be due to subprocesses crashing, this time because all calls to the function fslr::fslentropy are failing. The same calls succeed on my own machines, but of course my environment is different from that of the online R notebooks. In particular, although R package fslr is installed, as well as its R package dependencies, it looks like the required fsl system support is lacking. But that's okay, I can proceed without the fslr functions. However, the \"Cancel run\" button seems not to work even with runs that are still actively using the CPU, so I wonder whether it ever works at all. Perhaps that is something you could look into.\n\nAlso, one thing that made it harder for me to debug what my hung runs were doing is that apparently it is not possible to enter R commands in the Console while a script is running, even if that script has stopped doing anything useful. Perhaps that is intentional, although I'm not sure why.", "1009369": "One more point about this issue: on my own machines (where I run Ubuntu Linux), when all the child processes launched by R function mclapply from my program's parent process terminate, the parent continues execution, detects the failures, and terminates as well. But in your online notebook environment it seems like mclapply never returns, and the parent process continues idly waiting for the child processes to finish. If this is true, it's kind of weird." }, "source": "meta" }