Hi there - not sure if this is a dumb question or already answered but - are we allowed to process data locally and upload into the Kaggle notebook to visualize and summarize for the notebook submission portion? The tracking dataset is rather large and not sure how long it'll take to re-do the same data processing in the notebook.
", "messages": [ { "id": "701939", "postDate": "12/24/2019 04:39:21", "content": "Hi there - not sure if this is a dumb question or already answered but - are we allowed to process data locally and upload into the Kaggle notebook to visualize and summarize for the notebook submission portion? The tracking dataset is rather large and not sure how long it'll take to re-do the same data processing in the notebook.
", "rawMarkdown": "Hi there - not sure if this is a dumb question or already answered but - are we allowed to process data locally and upload into the Kaggle notebook to visualize and summarize for the notebook submission portion? The tracking dataset is rather large and not sure how long it'll take to re-do the same data processing in the notebook.", "votes": null }, { "id": "701998", "postDate": "12/24/2019 06:31:51", "content": "I have similar issues: I simply run out of memory in Kaggle environment.
", "rawMarkdown": "I have similar issues: I simply run out of memory in Kaggle environment.", "votes": null }, { "id": "702282", "postDate": "12/24/2019 13:50:08", "content": "My understanding is that is not allowed. See https://www.kaggle.com/c/nfl-playing-surface-analytics/discussion/120488
", "rawMarkdown": "My understanding is that is not allowed. See https://www.kaggle.com/c/nfl-playing-surface-analytics/discussion/120488", "votes": null }, { "id": "702569", "postDate": "12/24/2019 21:08:18", "content": "Thank you! Appreciate the help.
", "rawMarkdown": "Thank you! Appreciate the help.", "votes": null }, { "id": "702718", "postDate": "12/25/2019 04:05:34", "content": "You can try chaining notebooks. For example, process the tracks file in a notebook and save the output as a parquet or feather file. Then pull that file into another notebook as a dataset and do more processing. Repeat as necessary. You can also use libraries like Dask although there's a learning curve.
", "rawMarkdown": "You can try chaining notebooks. For example, process the tracks file in a notebook and save the output as a parquet or feather file. Then pull that file into another notebook as a dataset and do more processing. Repeat as necessary. You can also use libraries like Dask although there's a learning curve.", "votes": null }, { "id": "703959", "postDate": "12/26/2019 21:40:33", "content": "Thanks John!
", "rawMarkdown": "Thanks John!", "votes": null }, { "id": "703960", "postDate": "12/26/2019 21:41:11", "content": "Yeah we're running into a lot of issues with memory allocation because of how we're processing the tracking data
", "rawMarkdown": "Yeah we're running into a lot of issues with memory allocation because of how we're processing the tracking data", "votes": null } ], "comments": [ { "id": 701998, "author_name": "aleksandradeis", "author_url": "", "post_date": "12/24/2019 06:31:51", "content": "I have similar issues: I simply run out of memory in Kaggle environment.
", "votes": null, "replies": [ { "id": 703960, "author_name": "cathyha", "author_url": "", "post_date": "12/26/2019 21:41:11", "content": "Yeah we're running into a lot of issues with memory allocation because of how we're processing the tracking data
", "votes": null, "replies": [] } ] }, { "id": 702282, "author_name": "ericfreeman", "author_url": "", "post_date": "12/24/2019 13:50:08", "content": "My understanding is that is not allowed. See https://www.kaggle.com/c/nfl-playing-surface-analytics/discussion/120488
", "votes": null, "replies": [ { "id": 702569, "author_name": "cathyha", "author_url": "", "post_date": "12/24/2019 21:08:18", "content": "Thank you! Appreciate the help.
", "votes": null, "replies": [] } ] }, { "id": 702718, "author_name": "jpmiller", "author_url": "", "post_date": "12/25/2019 04:05:34", "content": "You can try chaining notebooks. For example, process the tracks file in a notebook and save the output as a parquet or feather file. Then pull that file into another notebook as a dataset and do more processing. Repeat as necessary. You can also use libraries like Dask although there's a learning curve.
", "votes": null, "replies": [ { "id": 703959, "author_name": "cathyha", "author_url": "", "post_date": "12/26/2019 21:40:33", "content": "Thanks John!
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "701939": "Hi there - not sure if this is a dumb question or already answered but - are we allowed to process data locally and upload into the Kaggle notebook to visualize and summarize for the notebook submission portion? The tracking dataset is rather large and not sure how long it'll take to re-do the same data processing in the notebook.", "701998": "I have similar issues: I simply run out of memory in Kaggle environment.", "702282": "My understanding is that is not allowed. See https://www.kaggle.com/c/nfl-playing-surface-analytics/discussion/120488", "702569": "Thank you! Appreciate the help.", "702718": "You can try chaining notebooks. For example, process the tracks file in a notebook and save the output as a parquet or feather file. Then pull that file into another notebook as a dataset and do more processing. Repeat as necessary. You can also use libraries like Dask although there's a learning curve.", "703959": "Thanks John!", "703960": "Yeah we're running into a lot of issues with memory allocation because of how we're processing the tracking data" }, "source": "meta" }