This is my first Code Only Competition. In the code requirements page, the following item is mentioned \"No custom packages enabled in kernels\". And I just checked, standard kernel does not have huggingface-transformers installed (which, by the way, is a great NLP library). So does that mean I cannot use it for this competition? Or can I just copy the source code into the kernel (it would be quite a clustter though). And what is the point of having this constraint in the first place (I can understand about run-time constraint) Thanks and happy Kaggling!
", "messages": [ { "id": "661983", "postDate": "10/31/2019 00:14:33", "content": "This is my first Code Only Competition. In the code requirements page, the following item is mentioned \"No custom packages enabled in kernels\". And I just checked, standard kernel does not have huggingface-transformers installed (which, by the way, is a great NLP library). So does that mean I cannot use it for this competition? Or can I just copy the source code into the kernel (it would be quite a clustter though). And what is the point of having this constraint in the first place (I can understand about run-time constraint) Thanks and happy Kaggling!
", "rawMarkdown": "This is my first Code Only Competition. In the code requirements page, the following item is mentioned \"No custom packages enabled in kernels\". And I just checked, standard kernel does not have huggingface-transformers installed (which, by the way, is a great NLP library). So does that mean I cannot use it for this competition? Or can I just copy the source code into the kernel (it would be quite a clustter though). And what is the point of having this constraint in the first place (I can understand about run-time constraint) Thanks and happy Kaggling!", "votes": null }, { "id": "662040", "postDate": "10/31/2019 02:52:06", "content": "In a code competition your submission must come from a Kaggle notebook, and it’s true this competition prohibits custom packages. But you are allowed to use external data, input notebooks as a data source, or utility scripts. This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts. You can take a look at the starter notebook that Phil shared, which shows how this was done using a basic Bert implementation.
", "rawMarkdown": "In a code competition your submission must come from a Kaggle notebook, and it’s true this competition prohibits custom packages. But you are allowed to use external data, input notebooks as a data source, or utility scripts. This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts. You can take a look at the starter notebook that Phil shared, which shows how this was done using a basic Bert implementation.", "votes": null }, { "id": "662342", "postDate": "10/31/2019 12:59:05", "content": "Got it. Thank you
", "rawMarkdown": "Got it. Thank you", "votes": null }, { "id": "666579", "postDate": "11/06/2019 08:47:22", "content": "> This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts.
\n\nThis is all good and well and I think it is a good step in making competitions fair and code reproducible. Though what about the dependencies of these custom libraries? Not being able to use pip is just really inconvenient for developing ML models, since ML is based upon open source libraries that in turn depend on other open source libraries and so on : )\nA detailed tutorial would be great on how to package all these dependencies and upload it to your kaggle notebook so you can use it (and imports inside other libraries do not break as well).
", "rawMarkdown": "> This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts.\n\nThis is all good and well and I think it is a good step in making competitions fair and code reproducible. Though what about the dependencies of these custom libraries? Not being able to use pip is just really inconvenient for developing ML models, since ML is based upon open source libraries that in turn depend on other open source libraries and so on : )\nA detailed tutorial would be great on how to package all these dependencies and upload it to your kaggle notebook so you can use it (and imports inside other libraries do not break as well).", "votes": null } ], "comments": [ { "id": 662040, "author_name": "juliaelliott", "author_url": "", "post_date": "10/31/2019 02:52:06", "content": "In a code competition your submission must come from a Kaggle notebook, and it’s true this competition prohibits custom packages. But you are allowed to use external data, input notebooks as a data source, or utility scripts. This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts. You can take a look at the starter notebook that Phil shared, which shows how this was done using a basic Bert implementation.
", "votes": null, "replies": [ { "id": 662342, "author_name": "zhenlanwang", "author_url": "", "post_date": "10/31/2019 12:59:05", "content": "Got it. Thank you
", "votes": null, "replies": [] }, { "id": 666579, "author_name": "timoeller", "author_url": "", "post_date": "11/06/2019 08:47:22", "content": "> This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts.
\n\nThis is all good and well and I think it is a good step in making competitions fair and code reproducible. Though what about the dependencies of these custom libraries? Not being able to use pip is just really inconvenient for developing ML models, since ML is based upon open source libraries that in turn depend on other open source libraries and so on : )\nA detailed tutorial would be great on how to package all these dependencies and upload it to your kaggle notebook so you can use it (and imports inside other libraries do not break as well).
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "661983": "This is my first Code Only Competition. In the code requirements page, the following item is mentioned \"No custom packages enabled in kernels\". And I just checked, standard kernel does not have huggingface-transformers installed (which, by the way, is a great NLP library). So does that mean I cannot use it for this competition? Or can I just copy the source code into the kernel (it would be quite a clustter though). And what is the point of having this constraint in the first place (I can understand about run-time constraint) Thanks and happy Kaggling!", "662040": "In a code competition your submission must come from a Kaggle notebook, and it’s true this competition prohibits custom packages. But you are allowed to use external data, input notebooks as a data source, or utility scripts. This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts. You can take a look at the starter notebook that Phil shared, which shows how this was done using a basic Bert implementation.", "662342": "Got it. Thank you", "666579": "> This means you can bring in huggingface with those methods, including training locally and then importing the model into your Kaggle notebook as an external data source, or adding any pretrained models or libraries as their own utility scripts.\n\nThis is all good and well and I think it is a good step in making competitions fair and code reproducible. Though what about the dependencies of these custom libraries? Not being able to use pip is just really inconvenient for developing ML models, since ML is based upon open source libraries that in turn depend on other open source libraries and so on : )\nA detailed tutorial would be great on how to package all these dependencies and upload it to your kaggle notebook so you can use it (and imports inside other libraries do not break as well)." }, "source": "meta" }