{ "id": 177414, "title": "External Data Disclosure Thread", "url": "/competitions/lyft-motion-prediction-autonomous-vehicles/discussion/177414", "author_name": "Vladimir Iglovikov", "post_date": "2020-08-25T20:22:44.371000", "votes": 35, "comment_count": 54, "views": 0, "content": "

External data is allowed in this competition.

\n

External data includes:

\n\n

To use external data in your solution you need:

\n
    \n
  1. Verify that the license is consistent with rules.
    2. \n
  2. Verify that this source of the data was not discussed before.
    3. \n
  3. Write a post in this thread with a short description (2-3 sentences) and link to the external data.
    4. \n
  4. One request per post. A set of links will not work.
    5. \n
  5. Post without description will not work.
    6. \n
\n

If you did all these steps, it does not mean that you can use the requested data or pre-trained models.

\n

Hosts need to review it and explicitly confirm that this is allowed in this challenge.

\n\n

Reviewing the request can take time.

\n\n

To start:

\n\n

Are allowed.

\n

Feel free to share more in this thread and do not forget about description.

", "messages": [ { "id": 985560, "postDate": "2020-08-25T20:22:44.370Z", "content": "

External data is allowed in this competition.

\n

External data includes:

\n\n

To use external data in your solution you need:

\n
    \n
  1. Verify that the license is consistent with rules.
    2. \n
  2. Verify that this source of the data was not discussed before.
    3. \n
  3. Write a post in this thread with a short description (2-3 sentences) and link to the external data.
    4. \n
  4. One request per post. A set of links will not work.
    5. \n
  5. Post without description will not work.
    6. \n
\n

If you did all these steps, it does not mean that you can use the requested data or pre-trained models.

\n

Hosts need to review it and explicitly confirm that this is allowed in this challenge.

\n\n

Reviewing the request can take time.

\n\n

To start:

\n\n

Are allowed.

\n

Feel free to share more in this thread and do not forget about description.

", "rawMarkdown": "External data is allowed in this competition.\n\nExternal data includes:\n* Datasets.\n* Pre-trained models.\n\nTo use external data in your solution you need:\n\n1. Verify that the license is consistent with [rules](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/rules).\n2. Verify that this source of the data was not discussed before.\n3. Write a post in this thread with a short description (2-3 sentences) and link to the external data.\n4. One request per post. A [set of links](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121203#762841) will not work.\n5. Post without description will not work.\n\nIf you did all these steps, it **does not** mean that you can use the requested data or pre-trained models.\n\nHosts need to review it and explicitly confirm that this is allowed in this challenge.\n\n* If the host says: “Yes, it is allowed” => it is allowed\n* If the host says: “No, it is not allowed” => it is **not** allowed\n* If the host does not say anything => it is **not** allowed\n\nReviewing the request can take time. \n* Do not expect the answer instantly.\n* List your requests for the external data as early as possible.\n\n\nTo start:\n* Pre-trained models from [torchvision](https://pytorch.org/docs/stable/torchvision/models.html).\n* Pre-trained models from the [timm library](https://github.com/rwightman/pytorch-image-models) by @rwightman.\n\nAre allowed.\n\nFeel free to share more in this thread and do not forget about description.", "votes": 34 }, { "id": 1046052, "postDate": "2020-10-11T10:06:27.660Z", "content": "

There is already a message about ResNeSt, but answer is not clear for me. Can we use https://github.com/zhanghang1989/ResNeSt pretrained models? It comes with Apache-2.0 license.

", "rawMarkdown": "There is already a message about ResNeSt, but answer is not clear for me. Can we use https://github.com/zhanghang1989/ResNeSt pretrained models? It comes with Apache-2.0 license.", "votes": 1, "replies": [ { "id": 1049674, "postDate": "2020-10-14T16:42:12.130Z", "content": "

Pre-trained models from https://github.com/zhanghang1989/ResNeSt allowed.

", "rawMarkdown": "Pre-trained models from https://github.com/zhanghang1989/ResNeSt **allowed**." } ] }, { "id": 1020626, "postDate": "2020-09-21T10:04:11.563Z", "content": "

models with pre-trained weights in tf.keras.applications
\nhttps://www.tensorflow.org/api_docs/python/tf/keras/applications

", "rawMarkdown": "models with pre-trained weights in tf.keras.applications\nhttps://www.tensorflow.org/api_docs/python/tf/keras/applications", "votes": 1, "replies": [ { "id": 1021163, "postDate": "2020-09-21T17:09:53.010Z", "content": "

Pre-trained models from tf.keras.applications are allowed.

", "rawMarkdown": "Pre-trained models from `tf.keras.applications` are **allowed**.", "votes": 1 }, { "id": 1039906, "postDate": "2020-10-06T21:38:32.770Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 986295, "postDate": "2020-08-26T10:57:52.277Z", "content": "

Custom fork of l5kit as a data source?
\nI'd like to add some additional functionality to the l5kit to experiment with different data manipulations.

", "rawMarkdown": "Custom fork of `l5kit` as a data source?\nI'd like to add some additional functionality to the l5kit to experiment with different data manipulations.", "votes": 1, "replies": [ { "id": 986607, "postDate": "2020-08-26T16:20:59.507Z", "content": "
    \n
  1. Feel free to build on top of l5kit.
    2. \n
  2. You may consider creating a pull request and merging your improvements to the l5kit repository.
    3. \n
\n

You cannot take pieces of the code from some other repo that has GPL or other restrictive licenses and copy and paste them into your fork of l5kit.

", "rawMarkdown": "1. Feel free to build on top of `l5kit`.\n2. You may consider creating a pull request and merging your improvements to the `l5kit` repository. \n\nYou cannot take pieces of the code from some other repo that has GPL or other restrictive licenses and copy and paste them into your fork of l5kit.", "votes": 2 } ] }, { "id": 985650, "postDate": "2020-08-25T22:49:58.447Z", "content": "

Can we pre-train on Argoverse? https://www.argoverse.org/

", "rawMarkdown": "Can we pre-train on Argoverse? https://www.argoverse.org/ ", "votes": 1, "replies": [ { "id": 985671, "postDate": "2020-08-25T23:29:30.997Z", "content": "
\n

Verify that the license is consistent with rules.

\n
\n

Did you check?

", "rawMarkdown": "> Verify that the license is consistent with rules.\n\nDid you check?\n\n" }, { "id": 985691, "postDate": "2020-08-26T00:07:40.740Z", "content": "

Well, I'm confused to be honest, so that's why I asked. Argoverse data and Lyft data are both CC-BY-NC-SA-4.0 (see Argoverse https://www.argoverse.org/about.html#terms-of-use ). The competition rules only really talk about code licenses; in particular, the winners must grant a license for \"source code used to generate the Submission,\" but (maybe I missed it?) the rules don't talk about data or trained models. For the Lyft Detection challenge, Argoverse was deemed OK though.

", "rawMarkdown": "Well, I'm confused to be honest, so that's why I asked. Argoverse data and Lyft data are both CC-BY-NC-SA-4.0 (see Argoverse https://www.argoverse.org/about.html#terms-of-use ). The competition rules only really talk about code licenses; in particular, the winners must grant a license for \"source code used to generate the Submission,\" but (maybe I missed it?) the rules don't talk about data or trained models. For the Lyft Detection challenge, Argoverse was deemed OK though. " }, { "id": 985723, "postDate": "2020-08-26T01:14:33.897Z", "content": "

For sure, the legal language is confusing. DIfferent restrictions are scattered across the rules.

\n

To answer your question:

\n", "rawMarkdown": "For sure, the legal language is confusing. DIfferent restrictions are scattered across the rules.\n\nTo answer your question:\n\n* Argoverse Dataset is **not allowed**.\n* Models pre-trained on the Argoverse Dataset are **not allowed**.", "votes": 4 } ] }, { "id": 985595, "postDate": "2020-08-25T21:02:58.900Z", "content": "

I plan to experiment efficientNet implementation here: https://github.com/lukemelas/EfficientNet-PyTorch
\nIt has efficientNet implementation code and pretrained weights, its license is Apache 2.0. Is this allowed?

", "rawMarkdown": "I plan to experiment efficientNet implementation here: https://github.com/lukemelas/EfficientNet-PyTorch\nIt has efficientNet implementation code and pretrained weights, its license is Apache 2.0. Is this allowed?", "votes": 1, "replies": [ { "id": 985675, "postDate": "2020-08-25T23:31:27.923Z", "content": "

Is there a strong preference to use https://github.com/lukemelas/EfficientNet-PyTorch

\n

vs https://github.com/rwightman/pytorch-image-models?

\n

Second has a larger set of pre-trained models, including EffiicientNet: https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv

", "rawMarkdown": "Is there a strong preference to use https://github.com/lukemelas/EfficientNet-PyTorch\n\nvs https://github.com/rwightman/pytorch-image-models?\n\nSecond has a larger set of pre-trained models, including EffiicientNet: https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv", "votes": 2 }, { "id": 985679, "postDate": "2020-08-25T23:44:21.217Z", "content": "

I see, I did not know about this library before this post. I guess the main preference is that one (lukemelas/Eff..) I have used before, though I agree that the one you mentioned has better flexibility. I will try to start migrating to pytorch-image-models, though could you please confirm that the link I provided is still acceptable just in case?

", "rawMarkdown": "I see, I did not know about this library before this post. I guess the main preference is that one (lukemelas/Eff..) I have used before, though I agree that the one you mentioned has better flexibility. I will try to start migrating to `pytorch-image-models`, though could you please confirm that the link I provided is still acceptable just in case?", "votes": 2 }, { "id": 985704, "postDate": "2020-08-26T00:34:39.967Z", "content": "

Pre-trained models from https://github.com/rwightman/pytorch-image-models are allowed.

", "rawMarkdown": "Pre-trained models from https://github.com/rwightman/pytorch-image-models are **allowed**.", "votes": 5 }, { "id": 989314, "postDate": "2020-08-28T17:52:35.643Z", "content": "

Pre-trained models from https://github.com/lukemelas/EfficientNet-PyTorch are allowed.

", "rawMarkdown": "Pre-trained models from https://github.com/lukemelas/EfficientNet-PyTorch are **allowed**.", "votes": 2 } ] }, { "id": 1033831, "postDate": "2020-10-01T09:50:08.983Z", "content": "

I have some questions regarding the rules:

\n
    \n

  1. Notebooks with internet activated are accepted for submissions, but do we need to turn off internet for the final submission notebook and copy the necessary libraries as a dataset? There are notebooks just uploading a CSV, is this against the rules for the final submission notebook? Or is it just important to give the whole end2end pipeline in the case of getting \"into the money\"?

    2. \n

  2. The rasterization is the bottle neck, is it ok to generate the images for the test set and upload them as a dataset or do they have to be generated during runtime for the final notebook? Is the test data for the private LB the same test.zarr?

    3. \n
\n

Thank you!

", "rawMarkdown": "I have some questions regarding the rules:\n\n1. Notebooks with internet activated are accepted for submissions, but do we need to turn off internet for the final submission notebook and copy the necessary libraries as a dataset? There are notebooks just uploading a CSV, is this against the rules for the final submission notebook? Or is it just important to give the whole end2end pipeline in the case of getting \"into the money\"?\n\n2. The rasterization is the bottle neck, is it ok to generate the images for the test set and upload them as a dataset or do they have to be generated during runtime for the final notebook? Is the test data for the private LB the same test.zarr? \n\nThank you!", "votes": 2, "replies": [ { "id": 1034502, "postDate": "2020-10-01T20:21:04.547Z", "content": "", "rawMarkdown": "- You can do everything offline.\n- You can upload the csv to the inference kernel and that is it.\n- If you get into \"money\" we will need code that reproduces your submission from the data. But even in this case, until the end of the competition, you can submit csv's.\n- All submissions that are made now, count both for Public and for Private test sets, no additional work is required.\n\n", "votes": 4 }, { "id": 1034504, "postDate": "2020-10-01T20:26:16.577Z", "content": "

Great! Thank you very much.

", "rawMarkdown": "Great! Thank you very much.", "votes": 1 }, { "id": 1042148, "postDate": "2020-10-08T05:07:56.080Z", "content": "

What do @iglovikov means of :

\n
\n

If you get into \"money\" we will need code that reproduces your submission from the data. But even in this case, until the end of the competition, you can submit CSV's'

\n
\n

I don't get you as much as everyone. I just need a clarity

\n\n
Please Clarify me? @iglovikov
", "rawMarkdown": "What do @iglovikov means of :\n> If you get into \"money\" we will need code that reproduces your submission from the data. But even in this case, until the end of the competition, you can submit CSV's'\n\nI don't get you as much as everyone. I just need a clarity\n\n- we could take submission file from the public kernel for inference also our own private notebook submission files\n- At the anyone is happened to win in this competition only by this blending.\n- if that so we need to give the code of whole documents of code of public and private\n\n###### Please Clarify me? @iglovikov" }, { "id": 1043246, "postDate": "2020-10-08T19:51:00.603Z", "content": "

If you use the submission from a public kernel in your \"blender\" and get in \"money\" you will also need to provide the code that produced that submission.

\n

=> If you are aiming for the top, using public kernels that have csv but do not have a training pipeline is not recommended.

", "rawMarkdown": "If you use the submission from a public kernel in your \"blender\" and get in \"money\" you will also need to provide the code that produced that submission.\n\n=> If you are aiming for the top, using public kernels that have csv but do not have a training pipeline is not recommended.", "votes": 1 }, { "id": 1043278, "postDate": "2020-10-08T20:37:32.173Z", "content": "

public leaderboard might be saturated with just blending the output csv files , i dont think thats a good practise for a competetion , if we want the state art of the art result at the end of this competetion !

", "rawMarkdown": "public leaderboard might be saturated with just blending the output csv files , i dont think thats a good practise for a competetion , if we want the state art of the art result at the end of this competetion !", "votes": -2 } ] }, { "id": 1086089, "postDate": "2020-11-21T11:09:46.437Z", "content": "

Can we use this competition test dataset for unsupervised / semisupervised training and domain adaptation?

", "rawMarkdown": "Can we use this competition test dataset for unsupervised / semisupervised training and domain adaptation?", "replies": [ { "id": 1088879, "postDate": "2020-11-24T02:46:53.283Z", "content": "

As I understand, you are asking about pseudo labeling.

\n

Sure. It is allowed.

", "rawMarkdown": "As I understand, you are asking about pseudo labeling.\n\nSure. It is **allowed**." } ] }, { "id": 1081806, "postDate": "2020-11-17T09:58:59.763Z", "content": "

Pretrained models from https://github.com/qubvel/segmentation_models.pytorch

", "rawMarkdown": "Pretrained models from https://github.com/qubvel/segmentation_models.pytorch", "replies": [ { "id": 1082158, "postDate": "2020-11-17T16:41:44.343Z", "content": "

Pre-trained models from https://github.com/qubvel/segmentation_models.pytorch are allowed

", "rawMarkdown": "Pre-trained models from https://github.com/qubvel/segmentation_models.pytorch are **allowed**", "votes": 1 } ] }, { "id": 1072789, "postDate": "2020-11-08T17:26:38.040Z", "content": "

Can I use optimizers code from https://github.com/mgrankin/over9000? It has implemented many different optimizers and has Apache-2.0 License

", "rawMarkdown": "Can I use optimizers code from https://github.com/mgrankin/over9000? It has implemented many different optimizers and has Apache-2.0 License", "replies": [ { "id": 1073602, "postDate": "2020-11-09T17:36:42.927Z", "content": "

Optimizers from https://github.com/mgrankin/over9000 are allowed.

", "rawMarkdown": "Optimizers from https://github.com/mgrankin/over9000 are **allowed**." } ] }, { "id": 1049697, "postDate": "2020-10-14T17:10:31.397Z", "content": "

Is it allowed to use fastai models from https://github.com/fastai/fastai? It has Apache-2.0

", "rawMarkdown": "Is it allowed to use fastai models from https://github.com/fastai/fastai? It has Apache-2.0", "replies": [ { "id": 1049752, "postDate": "2020-10-14T18:49:33.427Z", "content": "

Could you please share a link to the models? I do not see them in the readme.

", "rawMarkdown": "Could you please share a link to the models? I do not see them in the readme. " }, { "id": 1049765, "postDate": "2020-10-14T18:58:50.527Z", "content": "

Sure, these ones https://github.com/fastai/fastai/tree/master/fastai/vision/models

", "rawMarkdown": "Sure, these ones https://github.com/fastai/fastai/tree/master/fastai/vision/models" }, { "id": 1049928, "postDate": "2020-10-14T22:34:07.073Z", "content": "

Models from https://github.com/fastai/fastai/blob/master/fastai/vision/models/xresnet.py are allowed.

", "rawMarkdown": "Models from https://github.com/fastai/fastai/blob/master/fastai/vision/models/xresnet.py are **allowed**." } ] }, { "id": 1043421, "postDate": "2020-10-09T01:12:21.237Z", "content": "

is the data in the full training set is the same format as in the competition dataset @iglovikov

", "rawMarkdown": "is the data in the full training set is the same format as in the competition dataset @iglovikov " }, { "id": 1043414, "postDate": "2020-10-09T01:00:24.723Z", "content": "

Models from pretrainedmodels or torch.hub is allowded

\n

https://github.com/Cadene/pretrained-models.pytorch (pretrained models)
\nhttps://pytorch.org/hub/ (torch.hub.load)

", "rawMarkdown": "Models from pretrainedmodels or torch.hub is allowded\n\nhttps://github.com/Cadene/pretrained-models.pytorch (pretrained models)\nhttps://pytorch.org/hub/ (torch.hub.load)", "replies": [ { "id": 1044490, "postDate": "2020-10-09T20:12:25.660Z", "content": "

Repo https://github.com/Cadene/pretrained-models.pytorch is not actively supported.

\n

I would prefer it if you use models from https://github.com/rwightman/pytorch-image-models.

\n

The library has a better API and a larger set of pre-trained models. https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv

", "rawMarkdown": "Repo [https://github.com/Cadene/pretrained-models.pytorch](https://github.com/Cadene/pretrained-models.pytorch) is not actively supported.\n\nI would prefer it if you use models from [https://github.com/rwightman/pytorch-image-models](https://github.com/rwightman/pytorch-image-models). \n\nThe library has a better API and a larger set of pre-trained models. [https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv](https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv)" }, { "id": 1044491, "postDate": "2020-10-09T20:14:37.803Z", "content": "

Feel free to create one post per model (link and description) that you want from pytorch.org/hub and I will take a look.

", "rawMarkdown": "Feel free to create one post per model (link and description) that you want from pytorch.org/hub and I will take a look." }, { "id": 1044673, "postDate": "2020-10-10T03:09:23.250Z", "content": "

torch.hub.load('torch/vision:v0.6.0', resnext10132x8d')

", "rawMarkdown": "torch.hub.load('torch/vision:v0.6.0', resnext10132x8d')" }, { "id": 1049680, "postDate": "2020-10-14T16:43:43.623Z", "content": "

Does this work?

\n
import timm\n\nmodel = timm.create_model(\"swsl_resnext101_32x8d\", pretrained=True)\n
", "rawMarkdown": "Does this work?\n\n```\nimport timm\n\nmodel = timm.create_model(\"swsl_resnext101_32x8d\", pretrained=True)\n```" }, { "id": 1081805, "postDate": "2020-11-17T09:58:14.863Z", "content": "

@iglovikov

\n

We'd like to confirm https://github.com/Cadene/pretrained-models.pytorch can be used.
\nEven though there is more actively supported/recommended library now, this is participants choice what model/pretrained weight to use.

", "rawMarkdown": "@iglovikov \n\nWe'd like to confirm https://github.com/Cadene/pretrained-models.pytorch can be used.\nEven though there is more actively supported/recommended library now, this is participants choice what model/pretrained weight to use." }, { "id": 1082458, "postDate": "2020-11-17T23:04:04.783Z", "content": "

@iglovikov ping, could you confirm?

", "rawMarkdown": "@iglovikov ping, could you confirm?", "votes": 1 }, { "id": 1082494, "postDate": "2020-11-18T00:36:34.113Z", "content": "

Pre-trained models from https://github.com/Cadene/pretrained-models.pytorch are allowed

", "rawMarkdown": "Pre-trained models from https://github.com/Cadene/pretrained-models.pytorch are **allowed**", "votes": 1 } ] }, { "id": 1026669, "postDate": "2020-09-25T13:33:09.317Z", "content": "

Can we use the EfficientNet implementation for Keras? Link here. For this you also need to install kerasapplications, link here. I don't know if I have to create a separate post to install kerasapplications so please let me know.

", "rawMarkdown": "Can we use the EfficientNet implementation for Keras? Link [here](https://www.kaggle.com/vgarshin/efficientnet). For this you also need to install kerasapplications, link [here](ttps://www.kaggle.com/vgarshin/kerasapplications). I don't know if I have to create a separate post to install kerasapplications so please let me know.", "replies": [ { "id": 1026970, "postDate": "2020-09-25T17:44:58.700Z", "content": "

Overall I prefer two separate posts for two links, but let's go with two links here.

\n

Could you please provide links to the GitHub for these packages?

", "rawMarkdown": "Overall I prefer two separate posts for two links, but let's go with two links here.\n\nCould you please provide links to the GitHub for these packages?\n", "votes": 1 }, { "id": 1028059, "postDate": "2020-09-26T15:12:10.423Z", "content": "

Sure, EfficientNet implementation, and kerasapplications.

", "rawMarkdown": "Sure, [EfficientNet implementation](https://github.com/qubvel/efficientnet), and [kerasapplications](https://github.com/Solankimitra/kerasapplication/tree/master/keras-applications-master). " }, { "id": 1028440, "postDate": "2020-09-26T21:09:29.420Z", "content": "

EfficientNet implementation is allowed.
\nkerasapplications is allowed.

", "rawMarkdown": "[EfficientNet implementation](https://github.com/qubvel/efficientnet) is **allowed**.\n[kerasapplications](https://github.com/Solankimitra/kerasapplication/tree/master/keras-applications-master) is **allowed**.\n", "votes": 1 } ] }, { "id": 1025557, "postDate": "2020-09-24T16:32:05.560Z", "content": "

can I train model locally, then use this weights

", "rawMarkdown": "can I train model locally, then use this weights ", "replies": [ { "id": 1025929, "postDate": "2020-09-24T22:09:05.553Z", "content": "

Yes, you can.

\n

But if you get into \"money\" you will need to provide the end2end pipeline that reproduces train and inference pipeline.

", "rawMarkdown": "Yes, you can. \n\nBut if you get into \"money\" you will need to provide the end2end pipeline that reproduces train and inference pipeline.", "votes": 1 } ] }, { "id": 997537, "postDate": "2020-09-04T05:27:09.427Z", "content": "

interesting

", "rawMarkdown": "interesting\n" }, { "id": 987648, "postDate": "2020-08-27T12:21:41.030Z", "content": "

Is it possible to use this? https://github.com/zhanghang1989/ResNeSt

", "rawMarkdown": "Is it possible to use this? https://github.com/zhanghang1989/ResNeSt", "replies": [ { "id": 987889, "postDate": "2020-08-27T15:41:27.397Z", "content": "
\n

Write a post in this thread with a short description (2-3 sentences) and link to the external data.

\n
", "rawMarkdown": "> Write a post in this thread with a short description (2-3 sentences) and link to the external data.", "votes": 1 }, { "id": 1043416, "postDate": "2020-10-09T01:01:35.070Z", "content": "

Can we make a resnest private dataset and use of it

", "rawMarkdown": "Can we make a resnest private dataset and use of it" } ] }, { "id": 1025646, "postDate": "2020-09-24T17:28:07.090Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 1025927, "postDate": "2020-09-24T22:07:56.957Z", "content": "
Write a post in this thread with a short description (2-3 sentences) and link to the external data.\n
", "rawMarkdown": "```\nWrite a post in this thread with a short description (2-3 sentences) and link to the external data.\n```" } ] }, { "id": 1028802, "postDate": "2020-09-27T08:09:00.080Z", "content": "

seems very interesting thanks!

", "rawMarkdown": "seems very interesting thanks!", "votes": 1 } ], "comments": [ { "id": 1046052, "author_name": "Vladislav Ostankovich", "author_url": "", "post_date": "2020-10-11T10:06:27.660000", "content": "

There is already a message about ResNeSt, but answer is not clear for me. Can we use https://github.com/zhanghang1989/ResNeSt pretrained models? It comes with Apache-2.0 license.

", "votes": 1, "replies": [ { "id": 1049674, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-14T16:42:12.130000", "content": "

Pre-trained models from https://github.com/zhanghang1989/ResNeSt allowed.

", "votes": 0, "replies": [] } ] }, { "id": 1020626, "author_name": "outrunner", "author_url": "", "post_date": "2020-09-21T10:04:11.563000", "content": "

models with pre-trained weights in tf.keras.applications
\nhttps://www.tensorflow.org/api_docs/python/tf/keras/applications

", "votes": 1, "replies": [ { "id": 1021163, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-09-21T17:09:53.010000", "content": "

Pre-trained models from tf.keras.applications are allowed.

", "votes": 1, "replies": [] }, { "id": 1039906, "author_name": "", "author_url": "", "post_date": "2020-10-06T21:38:32.770000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 986295, "author_name": "Khushal B", "author_url": "", "post_date": "2020-08-26T10:57:52.277000", "content": "

Custom fork of l5kit as a data source?
\nI'd like to add some additional functionality to the l5kit to experiment with different data manipulations.

", "votes": 1, "replies": [ { "id": 986607, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-26T16:20:59.507000", "content": "
    \n
  1. Feel free to build on top of l5kit.
    2. \n
  2. You may consider creating a pull request and merging your improvements to the l5kit repository.
    3. \n
\n

You cannot take pieces of the code from some other repo that has GPL or other restrictive licenses and copy and paste them into your fork of l5kit.

", "votes": 2, "replies": [] } ] }, { "id": 985650, "author_name": "oarph", "author_url": "", "post_date": "2020-08-25T22:49:58.447000", "content": "

Can we pre-train on Argoverse? https://www.argoverse.org/

", "votes": 1, "replies": [ { "id": 985671, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-25T23:29:30.997000", "content": "
\n

Verify that the license is consistent with rules.

\n
\n

Did you check?

", "votes": 0, "replies": [] }, { "id": 985691, "author_name": "oarph", "author_url": "", "post_date": "2020-08-26T00:07:40.740000", "content": "

Well, I'm confused to be honest, so that's why I asked. Argoverse data and Lyft data are both CC-BY-NC-SA-4.0 (see Argoverse https://www.argoverse.org/about.html#terms-of-use ). The competition rules only really talk about code licenses; in particular, the winners must grant a license for \"source code used to generate the Submission,\" but (maybe I missed it?) the rules don't talk about data or trained models. For the Lyft Detection challenge, Argoverse was deemed OK though.

", "votes": 0, "replies": [] }, { "id": 985723, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-26T01:14:33.897000", "content": "

For sure, the legal language is confusing. DIfferent restrictions are scattered across the rules.

\n

To answer your question:

\n", "votes": 4, "replies": [] } ] }, { "id": 985595, "author_name": "Brian Lee", "author_url": "", "post_date": "2020-08-25T21:02:58.900000", "content": "

I plan to experiment efficientNet implementation here: https://github.com/lukemelas/EfficientNet-PyTorch
\nIt has efficientNet implementation code and pretrained weights, its license is Apache 2.0. Is this allowed?

", "votes": 1, "replies": [ { "id": 985675, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-25T23:31:27.923000", "content": "

Is there a strong preference to use https://github.com/lukemelas/EfficientNet-PyTorch

\n

vs https://github.com/rwightman/pytorch-image-models?

\n

Second has a larger set of pre-trained models, including EffiicientNet: https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv

", "votes": 2, "replies": [] }, { "id": 985679, "author_name": "Brian Lee", "author_url": "", "post_date": "2020-08-25T23:44:21.217000", "content": "

I see, I did not know about this library before this post. I guess the main preference is that one (lukemelas/Eff..) I have used before, though I agree that the one you mentioned has better flexibility. I will try to start migrating to pytorch-image-models, though could you please confirm that the link I provided is still acceptable just in case?

", "votes": 2, "replies": [] }, { "id": 985704, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-26T00:34:39.967000", "content": "

Pre-trained models from https://github.com/rwightman/pytorch-image-models are allowed.

", "votes": 5, "replies": [] }, { "id": 989314, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-28T17:52:35.643000", "content": "

Pre-trained models from https://github.com/lukemelas/EfficientNet-PyTorch are allowed.

", "votes": 2, "replies": [] } ] }, { "id": 1033831, "author_name": "Marek Wyborski", "author_url": "", "post_date": "2020-10-01T09:50:08.983000", "content": "

I have some questions regarding the rules:

\n
    \n

  1. Notebooks with internet activated are accepted for submissions, but do we need to turn off internet for the final submission notebook and copy the necessary libraries as a dataset? There are notebooks just uploading a CSV, is this against the rules for the final submission notebook? Or is it just important to give the whole end2end pipeline in the case of getting \"into the money\"?

    2. \n

  2. The rasterization is the bottle neck, is it ok to generate the images for the test set and upload them as a dataset or do they have to be generated during runtime for the final notebook? Is the test data for the private LB the same test.zarr?

    3. \n
\n

Thank you!

", "votes": 2, "replies": [ { "id": 1034502, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-01T20:21:04.547000", "content": "", "votes": 4, "replies": [] }, { "id": 1034504, "author_name": "Marek Wyborski", "author_url": "", "post_date": "2020-10-01T20:26:16.577000", "content": "

Great! Thank you very much.

", "votes": 1, "replies": [] }, { "id": 1042148, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2020-10-08T05:07:56.080000", "content": "

What do @iglovikov means of :

\n
\n

If you get into \"money\" we will need code that reproduces your submission from the data. But even in this case, until the end of the competition, you can submit CSV's'

\n
\n

I don't get you as much as everyone. I just need a clarity

\n\n
Please Clarify me? @iglovikov
", "votes": 0, "replies": [] }, { "id": 1043246, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-08T19:51:00.603000", "content": "

If you use the submission from a public kernel in your \"blender\" and get in \"money\" you will also need to provide the code that produced that submission.

\n

=> If you are aiming for the top, using public kernels that have csv but do not have a training pipeline is not recommended.

", "votes": 1, "replies": [] }, { "id": 1043278, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2020-10-08T20:37:32.173000", "content": "

public leaderboard might be saturated with just blending the output csv files , i dont think thats a good practise for a competetion , if we want the state art of the art result at the end of this competetion !

", "votes": -2, "replies": [] } ] }, { "id": 1086089, "author_name": "Josef Slavicek", "author_url": "", "post_date": "2020-11-21T11:09:46.437000", "content": "

Can we use this competition test dataset for unsupervised / semisupervised training and domain adaptation?

", "votes": 0, "replies": [ { "id": 1088879, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-11-24T02:46:53.283000", "content": "

As I understand, you are asking about pseudo labeling.

\n

Sure. It is allowed.

", "votes": 0, "replies": [] } ] }, { "id": 1081806, "author_name": "corochann", "author_url": "", "post_date": "2020-11-17T09:58:59.763000", "content": "

Pretrained models from https://github.com/qubvel/segmentation_models.pytorch

", "votes": 0, "replies": [ { "id": 1082158, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-11-17T16:41:44.343000", "content": "

Pre-trained models from https://github.com/qubvel/segmentation_models.pytorch are allowed

", "votes": 1, "replies": [] } ] }, { "id": 1072789, "author_name": "Random_prediction", "author_url": "", "post_date": "2020-11-08T17:26:38.040000", "content": "

Can I use optimizers code from https://github.com/mgrankin/over9000? It has implemented many different optimizers and has Apache-2.0 License

", "votes": 0, "replies": [ { "id": 1073602, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-11-09T17:36:42.927000", "content": "

Optimizers from https://github.com/mgrankin/over9000 are allowed.

", "votes": 0, "replies": [] } ] }, { "id": 1049697, "author_name": "Rauf Yagfarov", "author_url": "", "post_date": "2020-10-14T17:10:31.397000", "content": "

Is it allowed to use fastai models from https://github.com/fastai/fastai? It has Apache-2.0

", "votes": 0, "replies": [ { "id": 1049752, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-14T18:49:33.427000", "content": "

Could you please share a link to the models? I do not see them in the readme.

", "votes": 0, "replies": [] }, { "id": 1049765, "author_name": "Rauf Yagfarov", "author_url": "", "post_date": "2020-10-14T18:58:50.527000", "content": "

Sure, these ones https://github.com/fastai/fastai/tree/master/fastai/vision/models

", "votes": 0, "replies": [] }, { "id": 1049928, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-14T22:34:07.073000", "content": "

Models from https://github.com/fastai/fastai/blob/master/fastai/vision/models/xresnet.py are allowed.

", "votes": 0, "replies": [] } ] }, { "id": 1043421, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2020-10-09T01:12:21.237000", "content": "

is the data in the full training set is the same format as in the competition dataset @iglovikov

", "votes": 0, "replies": [] }, { "id": 1043414, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2020-10-09T01:00:24.723000", "content": "

Models from pretrainedmodels or torch.hub is allowded

\n

https://github.com/Cadene/pretrained-models.pytorch (pretrained models)
\nhttps://pytorch.org/hub/ (torch.hub.load)

", "votes": 0, "replies": [ { "id": 1044490, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-09T20:12:25.660000", "content": "

Repo https://github.com/Cadene/pretrained-models.pytorch is not actively supported.

\n

I would prefer it if you use models from https://github.com/rwightman/pytorch-image-models.

\n

The library has a better API and a larger set of pre-trained models. https://github.com/rwightman/pytorch-image-models/blob/master/results/results-imagenet.csv

", "votes": 0, "replies": [] }, { "id": 1044491, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-09T20:14:37.803000", "content": "

Feel free to create one post per model (link and description) that you want from pytorch.org/hub and I will take a look.

", "votes": 0, "replies": [] }, { "id": 1044673, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2020-10-10T03:09:23.250000", "content": "

torch.hub.load('torch/vision:v0.6.0', resnext10132x8d')

", "votes": 0, "replies": [] }, { "id": 1049680, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-10-14T16:43:43.623000", "content": "

Does this work?

\n
import timm\n\nmodel = timm.create_model(\"swsl_resnext101_32x8d\", pretrained=True)\n
", "votes": 0, "replies": [] }, { "id": 1081805, "author_name": "corochann", "author_url": "", "post_date": "2020-11-17T09:58:14.863000", "content": "

@iglovikov

\n

We'd like to confirm https://github.com/Cadene/pretrained-models.pytorch can be used.
\nEven though there is more actively supported/recommended library now, this is participants choice what model/pretrained weight to use.

", "votes": 0, "replies": [] }, { "id": 1082458, "author_name": "corochann", "author_url": "", "post_date": "2020-11-17T23:04:04.783000", "content": "

@iglovikov ping, could you confirm?

", "votes": 1, "replies": [] }, { "id": 1082494, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-11-18T00:36:34.113000", "content": "

Pre-trained models from https://github.com/Cadene/pretrained-models.pytorch are allowed

", "votes": 1, "replies": [] } ] }, { "id": 1026669, "author_name": "Fateh Aliyev", "author_url": "", "post_date": "2020-09-25T13:33:09.317000", "content": "

Can we use the EfficientNet implementation for Keras? Link here. For this you also need to install kerasapplications, link here. I don't know if I have to create a separate post to install kerasapplications so please let me know.

", "votes": 0, "replies": [ { "id": 1026970, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-09-25T17:44:58.700000", "content": "

Overall I prefer two separate posts for two links, but let's go with two links here.

\n

Could you please provide links to the GitHub for these packages?

", "votes": 1, "replies": [] }, { "id": 1028059, "author_name": "Fateh Aliyev", "author_url": "", "post_date": "2020-09-26T15:12:10.423000", "content": "

Sure, EfficientNet implementation, and kerasapplications.

", "votes": 0, "replies": [] }, { "id": 1028440, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-09-26T21:09:29.420000", "content": "

EfficientNet implementation is allowed.
\nkerasapplications is allowed.

", "votes": 1, "replies": [] } ] }, { "id": 1025557, "author_name": "Manh Lab", "author_url": "", "post_date": "2020-09-24T16:32:05.560000", "content": "

can I train model locally, then use this weights

", "votes": 0, "replies": [ { "id": 1025929, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-09-24T22:09:05.553000", "content": "

Yes, you can.

\n

But if you get into \"money\" you will need to provide the end2end pipeline that reproduces train and inference pipeline.

", "votes": 1, "replies": [] } ] }, { "id": 997537, "author_name": "Pavan HEMMEGE", "author_url": "", "post_date": "2020-09-04T05:27:09.427000", "content": "

interesting

", "votes": 0, "replies": [] }, { "id": 987648, "author_name": "Trigram", "author_url": "", "post_date": "2020-08-27T12:21:41.030000", "content": "

Is it possible to use this? https://github.com/zhanghang1989/ResNeSt

", "votes": 0, "replies": [ { "id": 987889, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-08-27T15:41:27.397000", "content": "
\n

Write a post in this thread with a short description (2-3 sentences) and link to the external data.

\n
", "votes": 1, "replies": [] }, { "id": 1043416, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2020-10-09T01:01:35.070000", "content": "

Can we make a resnest private dataset and use of it

", "votes": 0, "replies": [] } ] }, { "id": 1025646, "author_name": "", "author_url": "", "post_date": "2020-09-24T17:28:07.090000", "content": "", "votes": 0, "replies": [ { "id": 1025927, "author_name": "Vladimir Iglovikov", "author_url": "", "post_date": "2020-09-24T22:07:56.957000", "content": "
Write a post in this thread with a short description (2-3 sentences) and link to the external data.\n
", "votes": 0, "replies": [] } ] }, { "id": 1028802, "author_name": "Archit Singh", "author_url": "", "post_date": "2020-09-27T08:09:00.080000", "content": "

seems very interesting thanks!

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "985560": "External data is allowed in this competition.\n\nExternal data includes:\n* Datasets.\n* Pre-trained models.\n\nTo use external data in your solution you need:\n\n1. Verify that the license is consistent with [rules](https://www.kaggle.com/c/lyft-motion-prediction-autonomous-vehicles/rules).\n2. Verify that this source of the data was not discussed before.\n3. Write a post in this thread with a short description (2-3 sentences) and link to the external data.\n4. One request per post. A [set of links](https://www.kaggle.com/c/deepfake-detection-challenge/discussion/121203#762841) will not work.\n5. Post without description will not work.\n\nIf you did all these steps, it **does not** mean that you can use the requested data or pre-trained models.\n\nHosts need to review it and explicitly confirm that this is allowed in this challenge.\n\n* If the host says: “Yes, it is allowed” => it is allowed\n* If the host says: “No, it is not allowed” => it is **not** allowed\n* If the host does not say anything => it is **not** allowed\n\nReviewing the request can take time. \n* Do not expect the answer instantly.\n* List your requests for the external data as early as possible.\n\n\nTo start:\n* Pre-trained models from [torchvision](https://pytorch.org/docs/stable/torchvision/models.html).\n* Pre-trained models from the [timm library](https://github.com/rwightman/pytorch-image-models) by @rwightman.\n\nAre allowed.\n\nFeel free to share more in this thread and do not forget about description.", "1046052": "There is already a message about ResNeSt, but answer is not clear for me. Can we use https://github.com/zhanghang1989/ResNeSt pretrained models? It comes with Apache-2.0 license.", "1020626": "models with pre-trained weights in tf.keras.applications\nhttps://www.tensorflow.org/api_docs/python/tf/keras/applications", "986295": "Custom fork of `l5kit` as a data source?\nI'd like to add some additional functionality to the l5kit to experiment with different data manipulations.", "985650": "Can we pre-train on Argoverse? https://www.argoverse.org/ ", "985595": "I plan to experiment efficientNet implementation here: https://github.com/lukemelas/EfficientNet-PyTorch\nIt has efficientNet implementation code and pretrained weights, its license is Apache 2.0. Is this allowed?", "1033831": "I have some questions regarding the rules:\n\n1. Notebooks with internet activated are accepted for submissions, but do we need to turn off internet for the final submission notebook and copy the necessary libraries as a dataset? There are notebooks just uploading a CSV, is this against the rules for the final submission notebook? Or is it just important to give the whole end2end pipeline in the case of getting \"into the money\"?\n\n2. The rasterization is the bottle neck, is it ok to generate the images for the test set and upload them as a dataset or do they have to be generated during runtime for the final notebook? Is the test data for the private LB the same test.zarr? \n\nThank you!", "1086089": "Can we use this competition test dataset for unsupervised / semisupervised training and domain adaptation?", "1081806": "Pretrained models from https://github.com/qubvel/segmentation_models.pytorch", "1072789": "Can I use optimizers code from https://github.com/mgrankin/over9000? It has implemented many different optimizers and has Apache-2.0 License", "1049697": "Is it allowed to use fastai models from https://github.com/fastai/fastai? It has Apache-2.0", "1043421": "is the data in the full training set is the same format as in the competition dataset @iglovikov ", "1043414": "Models from pretrainedmodels or torch.hub is allowded\n\nhttps://github.com/Cadene/pretrained-models.pytorch (pretrained models)\nhttps://pytorch.org/hub/ (torch.hub.load)", "1026669": "Can we use the EfficientNet implementation for Keras? Link [here](https://www.kaggle.com/vgarshin/efficientnet). For this you also need to install kerasapplications, link [here](ttps://www.kaggle.com/vgarshin/kerasapplications). I don't know if I have to create a separate post to install kerasapplications so please let me know.", "1025557": "can I train model locally, then use this weights ", "997537": "interesting\n", "987648": "Is it possible to use this? https://github.com/zhanghang1989/ResNeSt", "1025646": "", "1028802": "seems very interesting thanks!" } }