Hi Kagglers, I am a bit frustrated right now, as I somehow at a dead end at the moment.\nMy highest LB at the moment is 7.66, by having enhanced the Keras Baseline Kernel (https://www.kaggle.com/ratan123/aptos-2019-keras-baseline) to my needs.
\n\nCurrently, the things that I am using are:\n- Normal Classification (no modal regression, no regression)\n- Custom Kappa loss\n- 2 epochs warm up, with frozen layers and 30 epochs with unforzen layers\n- DenseNet121\n- Ben's & Croppings Preprocessing of Images (https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping)\n- No TTA,\n- No Old Competition data.
\n\nI wanted to ask you guys, what were some implementations, that boosted your LB score? I don't necesseraly want you to tell me your secret weapon, but I would be pretty glad about getting a Kappa of 0.8 :)
\n\n(Furthermore, how long do your kernels usually take to train. As the generator is customized, such that the pictures are preprocessed when read in, it takes aaaaaaages..~8 hours)
", "messages": [ { "id": "589779", "postDate": "08/01/2019 11:15:27", "content": "Hi Kagglers, I am a bit frustrated right now, as I somehow at a dead end at the moment.\nMy highest LB at the moment is 7.66, by having enhanced the Keras Baseline Kernel (https://www.kaggle.com/ratan123/aptos-2019-keras-baseline) to my needs.
\n\nCurrently, the things that I am using are:\n- Normal Classification (no modal regression, no regression)\n- Custom Kappa loss\n- 2 epochs warm up, with frozen layers and 30 epochs with unforzen layers\n- DenseNet121\n- Ben's & Croppings Preprocessing of Images (https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping)\n- No TTA,\n- No Old Competition data.
\n\nI wanted to ask you guys, what were some implementations, that boosted your LB score? I don't necesseraly want you to tell me your secret weapon, but I would be pretty glad about getting a Kappa of 0.8 :)
\n\n(Furthermore, how long do your kernels usually take to train. As the generator is customized, such that the pictures are preprocessed when read in, it takes aaaaaaages..~8 hours)
", "rawMarkdown": "Hi Kagglers, I am a bit frustrated right now, as I somehow at a dead end at the moment.\nMy highest LB at the moment is 7.66, by having enhanced the Keras Baseline Kernel (https://www.kaggle.com/ratan123/aptos-2019-keras-baseline) to my needs. \n\nCurrently, the things that I am using are:\n- Normal Classification (no modal regression, no regression)\n- Custom Kappa loss\n- 2 epochs warm up, with frozen layers and 30 epochs with unforzen layers\n- DenseNet121\n- Ben's & Croppings Preprocessing of Images (https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping)\n- No TTA,\n- No Old Competition data.\n\nI wanted to ask you guys, what were some implementations, that boosted your LB score? I don't necesseraly want you to tell me your secret weapon, but I would be pretty glad about getting a Kappa of 0.8 :)\n\n(Furthermore, how long do your kernels usually take to train. As the generator is customized, such that the pictures are preprocessed when read in, it takes aaaaaaages..~8 hours)", "votes": null }, { "id": "589794", "postDate": "08/01/2019 11:37:41", "content": "You can try Efficientnets and pre-training on old competition data, go through the previous discussion threads they are more than enough to get you above 0.8
", "rawMarkdown": "You can try Efficientnets and pre-training on old competition data, go through the previous discussion threads they are more than enough to get you above 0.8", "votes": null }, { "id": "589850", "postDate": "08/01/2019 13:26:42", "content": "May be you can preprocess(and/or augment) images offline, upload it as dataset and then train to reduce train time and utilize GPU efficiently.
", "rawMarkdown": "May be you can preprocess(and/or augment) images offline, upload it as dataset and then train to reduce train time and utilize GPU efficiently.", "votes": null }, { "id": "589854", "postDate": "08/01/2019 13:31:57", "content": "I don't know how I couldn't think about this myself... Thanks man!
", "rawMarkdown": "I don't know how I couldn't think about this myself... Thanks man!", "votes": null }, { "id": "590026", "postDate": "08/01/2019 17:52:02", "content": "no problem, it happens with me too 😊
", "rawMarkdown": "no problem, it happens with me too 😊", "votes": null }, { "id": "590570", "postDate": "08/02/2019 10:54:19", "content": "I wanted to do that too but never even tried because of this comment on this kernel:\n.png?generation=1564743142056741&alt=media)
I am also suffering from eternal training times so If you somehow succeed please let me know how! :)
", "rawMarkdown": "I wanted to do that too but never even tried because of this comment on [this](https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping/comments) kernel:\n.png?generation=1564743142056741&alt=media)\n\nI am also suffering from eternal training times so If you somehow succeed please let me know how! :)", "votes": null }, { "id": "590574", "postDate": "08/02/2019 10:59:00", "content": "You can try making it a multi-label problem where every label includes all the previous labels, e.g. [0,0,1,0,0] ---> [1,1,1,0,0], [0,1,0,0,0] ---> [1,1,0,0,0].\nThis approach improved my LB score a bit. The intuition behind it is that there is a hierarchical structure in the classes in which each class possess features of the previous classes, this is usually the case when dealing with the levels severity of a disease.
", "rawMarkdown": "You can try making it a multi-label problem where every label includes all the previous labels, e.g. [0,0,1,0,0] ---> [1,1,1,0,0], [0,1,0,0,0] ---> [1,1,0,0,0].\nThis approach improved my LB score a bit. The intuition behind it is that there is a hierarchical structure in the classes in which each class possess features of the previous classes, this is usually the case when dealing with the levels severity of a disease.", "votes": null }, { "id": "590670", "postDate": "08/02/2019 13:15:19", "content": "I am only using single channel of image for training purposes, this speeds up training a lot and allows for smaller models (in terms of parameters). My current score is with model having ~1.1 million params and many tricks are still there in my sleeve 🙂
\n\nYou can read about superconvergence to get some idea about speeding up your training process
", "rawMarkdown": "1. You can definitely preprocess your train data an upload for kernel training, and then create a separate kernel with processing and submission code.\n\nI am only using single channel of image for training purposes, this speeds up training a lot and allows for smaller models (in terms of parameters). My current score is with model having ~1.1 million params and many tricks are still there in my sleeve 🙂\n\nYou can read about **superconvergence** to get some idea about speeding up your training process", "votes": null } ], "comments": [ { "id": 589794, "author_name": "rishabhiitbhu", "author_url": "", "post_date": "08/01/2019 11:37:41", "content": "You can try Efficientnets and pre-training on old competition data, go through the previous discussion threads they are more than enough to get you above 0.8
", "votes": null, "replies": [] }, { "id": 589850, "author_name": "ashwan1", "author_url": "", "post_date": "08/01/2019 13:26:42", "content": "May be you can preprocess(and/or augment) images offline, upload it as dataset and then train to reduce train time and utilize GPU efficiently.
", "votes": null, "replies": [ { "id": 589854, "author_name": "fanconic", "author_url": "", "post_date": "08/01/2019 13:31:57", "content": "I don't know how I couldn't think about this myself... Thanks man!
", "votes": null, "replies": [] }, { "id": 590026, "author_name": "ashwan1", "author_url": "", "post_date": "08/01/2019 17:52:02", "content": "no problem, it happens with me too 😊
", "votes": null, "replies": [] }, { "id": 590570, "author_name": "sam1320", "author_url": "", "post_date": "08/02/2019 10:54:19", "content": "I wanted to do that too but never even tried because of this comment on this kernel:\n.png?generation=1564743142056741&alt=media)
I am also suffering from eternal training times so If you somehow succeed please let me know how! :)
", "votes": null, "replies": [] }, { "id": 590670, "author_name": "ashwan1", "author_url": "", "post_date": "08/02/2019 13:15:19", "content": "I am only using single channel of image for training purposes, this speeds up training a lot and allows for smaller models (in terms of parameters). My current score is with model having ~1.1 million params and many tricks are still there in my sleeve 🙂
\n\nYou can read about superconvergence to get some idea about speeding up your training process
", "votes": null, "replies": [] } ] }, { "id": 590574, "author_name": "sam1320", "author_url": "", "post_date": "08/02/2019 10:59:00", "content": "You can try making it a multi-label problem where every label includes all the previous labels, e.g. [0,0,1,0,0] ---> [1,1,1,0,0], [0,1,0,0,0] ---> [1,1,0,0,0].\nThis approach improved my LB score a bit. The intuition behind it is that there is a hierarchical structure in the classes in which each class possess features of the previous classes, this is usually the case when dealing with the levels severity of a disease.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "589779": "Hi Kagglers, I am a bit frustrated right now, as I somehow at a dead end at the moment.\nMy highest LB at the moment is 7.66, by having enhanced the Keras Baseline Kernel (https://www.kaggle.com/ratan123/aptos-2019-keras-baseline) to my needs. \n\nCurrently, the things that I am using are:\n- Normal Classification (no modal regression, no regression)\n- Custom Kappa loss\n- 2 epochs warm up, with frozen layers and 30 epochs with unforzen layers\n- DenseNet121\n- Ben's & Croppings Preprocessing of Images (https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping)\n- No TTA,\n- No Old Competition data.\n\nI wanted to ask you guys, what were some implementations, that boosted your LB score? I don't necesseraly want you to tell me your secret weapon, but I would be pretty glad about getting a Kappa of 0.8 :)\n\n(Furthermore, how long do your kernels usually take to train. As the generator is customized, such that the pictures are preprocessed when read in, it takes aaaaaaages..~8 hours)", "589794": "You can try Efficientnets and pre-training on old competition data, go through the previous discussion threads they are more than enough to get you above 0.8", "589850": "May be you can preprocess(and/or augment) images offline, upload it as dataset and then train to reduce train time and utilize GPU efficiently.", "589854": "I don't know how I couldn't think about this myself... Thanks man!", "590026": "no problem, it happens with me too 😊", "590570": "I wanted to do that too but never even tried because of this comment on [this](https://www.kaggle.com/ratthachat/aptos-updatedv14-preprocessing-ben-s-cropping/comments) kernel:\n.png?generation=1564743142056741&alt=media)\n\nI am also suffering from eternal training times so If you somehow succeed please let me know how! :)", "590574": "You can try making it a multi-label problem where every label includes all the previous labels, e.g. [0,0,1,0,0] ---> [1,1,1,0,0], [0,1,0,0,0] ---> [1,1,0,0,0].\nThis approach improved my LB score a bit. The intuition behind it is that there is a hierarchical structure in the classes in which each class possess features of the previous classes, this is usually the case when dealing with the levels severity of a disease.", "590670": "1. You can definitely preprocess your train data an upload for kernel training, and then create a separate kernel with processing and submission code.\n\nI am only using single channel of image for training purposes, this speeds up training a lot and allows for smaller models (in terms of parameters). My current score is with model having ~1.1 million params and many tricks are still there in my sleeve 🙂\n\nYou can read about **superconvergence** to get some idea about speeding up your training process" }, "source": "meta" }