This competition provides us very large dataset(training: 2.42 million, test: 1.61 million). It takes so long long time to train models by all the training data.
\nShould we use all data at this time？ I don't think so because we still have more than two months until the competition deadline.
\nIt may be better to do trial and error with some of data, and train models by all the data sometimes.
\n
\nFor your reference, here are the results of training chemical formula prediction in my local environment.
| % of data | \ntraining time (1 fold, 20 epoch) | \nOOF Levenshtein Distance (5 Fold) | \n
|---|---|---|
| 10% | \n1.2 hours | \n0.499 | \n
| 20% | \n2.4 hours | \n0.343 | \n
| 50% | \n6 hours | \n0.210 | \n
| 100% | \n12 hours | \n0.154 | \n
I think at most 50% of data is sufficient for trial and error.
", "messages": [ { "id": "1252210", "postDate": "03/25/2021 13:57:35", "content": "This competition provides us very large dataset(training: 2.42 million, test: 1.61 million). It takes so long long time to train models by all the training data.
\nShould we use all data at this time？ I don't think so because we still have more than two months until the competition deadline.
\nIt may be better to do trial and error with some of data, and train models by all the data sometimes.
\n
\nFor your reference, here are the results of training chemical formula prediction in my local environment.
| % of data | \ntraining time (1 fold, 20 epoch) | \nOOF Levenshtein Distance (5 Fold) | \n
|---|---|---|
| 10% | \n1.2 hours | \n0.499 | \n
| 20% | \n2.4 hours | \n0.343 | \n
| 50% | \n6 hours | \n0.210 | \n
| 100% | \n12 hours | \n0.154 | \n
I think at most 50% of data is sufficient for trial and error.
", "rawMarkdown": "This competition provides us very large dataset(training: 2.42 million, test: 1.61 million). It takes so long long time to train models by all the training data.\n\nShould we use all data at this time？ I don't think so because we still have more than two months until the competition deadline.\nIt may be better to do trial and error with some of data, and train models by all the data sometimes.\nCan confirm.. I run experiments with 1 million images … it usually translate well when upscaling =)
", "rawMarkdown": "Can confirm.. I run experiments with 1 million images ... it usually translate well when upscaling =)", "votes": null }, { "id": "1252697", "postDate": "03/25/2021 23:52:47", "content": "Thank you for sharing.
\nDo you choose 1 million data in a specfic manner? I choose data by random sampling now but think this is not a good way.
", "rawMarkdown": "Thank you for sharing.\n\nDo you choose 1 million data in a specfic manner? I choose data by random sampling now but think this is not a good way.", "votes": null }, { "id": "1252714", "postDate": "03/26/2021 00:41:05", "content": "Thanks for sharing, I'm curious, what hardware do you use? TPUs?
", "rawMarkdown": "Thanks for sharing, I'm curious, what hardware do you use? TPUs?", "votes": null }, { "id": "1252721", "postDate": "03/26/2021 01:00:22", "content": "It was random… But I agree its not a optimal …
", "rawMarkdown": "It was random... But I agree its not a optimal ...", "votes": null }, { "id": "1252740", "postDate": "03/26/2021 01:45:24", "content": "OK, thanks.
\nI think considering number of atoms or InChI length may be an easy and good way.
", "rawMarkdown": "OK, thanks.\n\nI think considering number of atoms or InChI length may be an easy and good way.", "votes": null }, { "id": "1252746", "postDate": "03/26/2021 01:50:17", "content": "I use a single TitanRTX(24GB). But only 3~4GB was used for the above experiments because image size and batch size are not large.
", "rawMarkdown": "I use a single TitanRTX(24GB). But only 3~4GB was used for the above experiments because image size and batch size are not large.", "votes": null }, { "id": "1252967", "postDate": "03/26/2021 08:33:34", "content": "I use all the train data.(2.42 million images val 1fold) So it takes about 2~5 hours to train 1epoch.
\nAlso, I have experienced poor accuracy when training with batch size larger than 256 in my case.
\nI feel that training with a large batch size is difficult.
Thank you @ttahara for the insight
\nI believe we can take a percentage of the sample (30, 40, or 50%) in random using numpy random at this stage. This get us close to 80% to 90& of our target goal from where we can fine tune for greater performance.
Thanks for sharing.
\nI train models with batch size of 64.
", "rawMarkdown": "Thanks for sharing.\n\nI train models with batch size of 64.", "votes": null }, { "id": "1254102", "postDate": "03/27/2021 10:55:18", "content": "How are you spitting the data for train and validation? randomly ?
", "rawMarkdown": "How are you spitting the data for train and validation? randomly ?", "votes": null }, { "id": "1254265", "postDate": "03/27/2021 13:33:36", "content": "I split data into train and valid in Multi-Label Stratified KFold(K=5) manner, using whether or not each atom is included in molecular as multi-label.
\nI'm sharing splitting process in my notebook.
", "rawMarkdown": "I split data into train and valid in Multi-Label Stratified KFold(K=5) manner, using whether or not each atom is included in molecular as multi-label.\n\nI'm sharing splitting process in [my notebook](https://www.kaggle.com/ttahara/bms-mt-chemical-formula-regression-training).", "votes": null }, { "id": "1296173", "postDate": "05/07/2021 03:43:55", "content": "I have done some experiments with different Encoders (CNN models) and different percentage of the training data and I have concluded that experimenting with 10% training samples is ok and reliable when scaling. Maybe less than 10% is ok as well.
\nBut for the final score and submission it's crucial to train with all data available, it is not worthy to reduce the training set.
", "rawMarkdown": "I have done some experiments with different Encoders (CNN models) and different percentage of the training data and I have concluded that experimenting with 10% training samples is ok and reliable when scaling. Maybe less than 10% is ok as well.\n\nBut for the final score and submission it's crucial to train with all data available, it is not worthy to reduce the training set.", "votes": null } ], "comments": [ { "id": 1252635, "author_name": "drhabib", "author_url": "", "post_date": "03/25/2021 21:32:42", "content": "Can confirm.. I run experiments with 1 million images … it usually translate well when upscaling =)
", "votes": null, "replies": [ { "id": 1252697, "author_name": "ttahara", "author_url": "", "post_date": "03/25/2021 23:52:47", "content": "Thank you for sharing.
\nDo you choose 1 million data in a specfic manner? I choose data by random sampling now but think this is not a good way.
", "votes": null, "replies": [] }, { "id": 1252721, "author_name": "drhabib", "author_url": "", "post_date": "03/26/2021 01:00:22", "content": "It was random… But I agree its not a optimal …
", "votes": null, "replies": [] }, { "id": 1252740, "author_name": "ttahara", "author_url": "", "post_date": "03/26/2021 01:45:24", "content": "OK, thanks.
\nI think considering number of atoms or InChI length may be an easy and good way.
", "votes": null, "replies": [] } ] }, { "id": 1252714, "author_name": "michaelwolff", "author_url": "", "post_date": "03/26/2021 00:41:05", "content": "Thanks for sharing, I'm curious, what hardware do you use? TPUs?
", "votes": null, "replies": [ { "id": 1252746, "author_name": "ttahara", "author_url": "", "post_date": "03/26/2021 01:50:17", "content": "I use a single TitanRTX(24GB). But only 3~4GB was used for the above experiments because image size and batch size are not large.
", "votes": null, "replies": [] } ] }, { "id": 1252967, "author_name": "kanbehmw", "author_url": "", "post_date": "03/26/2021 08:33:34", "content": "I use all the train data.(2.42 million images val 1fold) So it takes about 2~5 hours to train 1epoch.
\nAlso, I have experienced poor accuracy when training with batch size larger than 256 in my case.
\nI feel that training with a large batch size is difficult.
Thanks for sharing.
\nI train models with batch size of 64.
", "votes": null, "replies": [] } ] }, { "id": 1252998, "author_name": "olusesiadebisi", "author_url": "", "post_date": "03/26/2021 09:25:56", "content": "Thank you @ttahara for the insight
\nI believe we can take a percentage of the sample (30, 40, or 50%) in random using numpy random at this stage. This get us close to 80% to 90& of our target goal from where we can fine tune for greater performance.
How are you spitting the data for train and validation? randomly ?
", "votes": null, "replies": [ { "id": 1254265, "author_name": "ttahara", "author_url": "", "post_date": "03/27/2021 13:33:36", "content": "I split data into train and valid in Multi-Label Stratified KFold(K=5) manner, using whether or not each atom is included in molecular as multi-label.
\nI'm sharing splitting process in my notebook.
", "votes": null, "replies": [] } ] }, { "id": 1296173, "author_name": "hinepo", "author_url": "", "post_date": "05/07/2021 03:43:55", "content": "I have done some experiments with different Encoders (CNN models) and different percentage of the training data and I have concluded that experimenting with 10% training samples is ok and reliable when scaling. Maybe less than 10% is ok as well.
\nBut for the final score and submission it's crucial to train with all data available, it is not worthy to reduce the training set.
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "1252210": "This competition provides us very large dataset(training: 2.42 million, test: 1.61 million). It takes so long long time to train models by all the training data.\n\nShould we use all data at this time？ I don't think so because we still have more than two months until the competition deadline.\nIt may be better to do trial and error with some of data, and train models by all the data sometimes.\n