{ "id": 78698, "title": "Is it just me or this competition is really looooooong?", "url": "/competitions/quora-insincere-questions-classification/discussion/78698", "author_name": "Khoi Nguyen", "post_date": "2019-01-27T03:57:34.782000", "votes": 8, "comment_count": 12, "views": 0, "content": "

I thought it will be over in 10 days but apparently we'll have to wait for another 6 weeks for the results to finalize. I think this should be a 2-month competitions with 4-6 weeks finalization to make it 3 month like most of the others. Personally I got really unmotivated at crunching out 0.00x halfway through, but this should be my problem as I can see a lot of people making huge improvements in the last few days (team merging also helped maybe?)

", "messages": [ { "id": 461816, "postDate": "2019-01-27T03:57:34.783Z", "content": "

I thought it will be over in 10 days but apparently we'll have to wait for another 6 weeks for the results to finalize. I think this should be a 2-month competitions with 4-6 weeks finalization to make it 3 month like most of the others. Personally I got really unmotivated at crunching out 0.00x halfway through, but this should be my problem as I can see a lot of people making huge improvements in the last few days (team merging also helped maybe?)

", "rawMarkdown": "I thought it will be over in 10 days but apparently we'll have to wait for another 6 weeks for the results to finalize. I think this should be a 2-month competitions with 4-6 weeks finalization to make it 3 month like most of the others. Personally I got really unmotivated at crunching out 0.00x halfway through, but this should be my problem as I can see a lot of people making huge improvements in the last few days (team merging also helped maybe?)", "votes": 8 }, { "id": 461874, "postDate": "2019-01-27T08:28:09.857Z", "content": "

I remembered Mercari competition took less than a day to finalize, but we were only allowed to use CPU kernels. Such long finalizing time of Quora competition could be caused by limited GPU resource, I guess?\nThis competition became motivating for me after adjusting the local CV. My LB score easily jumped from 0.700 to 0.704 in merely two days with a relatively more reliable CV score.\nCan't wait to get another medal and rank up to Expert. I'm super excited now :-)!

", "rawMarkdown": "I remembered Mercari competition took less than a day to finalize, but we were only allowed to use CPU kernels. Such long finalizing time of Quora competition could be caused by limited GPU resource, I guess?\nThis competition became motivating for me after adjusting the local CV. My LB score easily jumped from 0.700 to 0.704 in merely two days with a relatively more reliable CV score.\nCan't wait to get another medal and rank up to Expert. I'm super excited now :-)!", "votes": 1, "replies": [ { "id": 461887, "postDate": "2019-01-27T09:09:14.213Z", "content": "

I'm surprised by your \"easily jumped from 0.700 to 0.704\". Thanks for your motivation. But I'm curious about how you adjusted your local CV. Did you change the seed for splitting or change the model(s)?

", "rawMarkdown": "I'm surprised by your \"easily jumped from 0.700 to 0.704\". Thanks for your motivation. But I'm curious about how you adjusted your local CV. Did you change the seed for splitting or change the model(s)?", "votes": 1 }, { "id": 461953, "postDate": "2019-01-27T10:21:05.047Z", "content": "

No fancy tricks. It's more about basic concepts of ML. In order to polish my knowledge, I restudy the part about validation from the ML MOOC I took before . Here's some advice:

\n\n
    \n
  1. Don't judge your model with F1 only. Try to observe the patterns of loss, precision, and recall. (Draw curves and calculate standard deviations for easier observation)
    2. \n
  2. Don't compare two models with average loss and CV metrics only. Compare through every epochs of every folds. You don't want to miss any details there!
    3. \n
  3. If you want to lower the folds and epochs for faster tuning. Try using the best seed in your model as a baseline, and randomize the seed while tuning it.
    4. \n
", "rawMarkdown": "No fancy tricks. It's more about basic concepts of ML. In order to polish my knowledge, I restudy the part about validation from the ML MOOC I took before . Here's some advice:\n\n1. Don't judge your model with F1 only. Try to observe the patterns of loss, precision, and recall. (Draw curves and calculate standard deviations for easier observation)\n2. Don't compare two models with average loss and CV metrics only. Compare through every epochs of every folds. You don't want to miss any details there!\n3. If you want to lower the folds and epochs for faster tuning. Try using the best seed in your model as a baseline, and randomize the seed while tuning it.", "votes": 11 }, { "id": 461985, "postDate": "2019-01-27T11:25:03.447Z", "content": "

Your advice are really helpful and I have a lot of things to try now. Thanks again. But I'm wondering, from 0.700 to 0.704, whether you also modified the preprocessing part or you only tuned the model training part? I would be more surprised if just the model tuning alone can make such a big difference.

", "rawMarkdown": "Your advice are really helpful and I have a lot of things to try now. Thanks again. But I'm wondering, from 0.700 to 0.704, whether you also modified the preprocessing part or you only tuned the model training part? I would be more surprised if just the model tuning alone can make such a big difference." }, { "id": 462014, "postDate": "2019-01-27T13:50:25.387Z", "content": "

I didn't make any change to the preprocessing. The fun fact was, I initially made 6 features that seem to perform well on my old CV, but I eventually removed 4 of them according to my new CV approach. On top of that, I fine-tuned the model, the optimizer, and some hyperparams like batch size. A reliable CV really helped me get the most out of my features and model.

", "rawMarkdown": "I didn't make any change to the preprocessing. The fun fact was, I initially made 6 features that seem to perform well on my old CV, but I eventually removed 4 of them according to my new CV approach. On top of that, I fine-tuned the model, the optimizer, and some hyperparams like batch size. A reliable CV really helped me get the most out of my features and model.", "votes": 1 }, { "id": 462145, "postDate": "2019-01-27T19:10:25.283Z", "content": "

can you tell me your local cv?

", "rawMarkdown": "can you tell me your local cv? " }, { "id": 462162, "postDate": "2019-01-27T19:51:50.883Z", "content": "

CV --> LB
\n0.680 --> 0.700
\n0.681 --> 0.701
\n0.682 --> 0.702
\n0.683 --> 0.704

", "rawMarkdown": "CV --> LB \n0.680 --> 0.700 \n0.681 --> 0.701 \n0.682 --> 0.702 \n0.683 --> 0.704" }, { "id": 462237, "postDate": "2019-01-27T23:45:06.457Z", "content": "

Thanks for the response. my cv is 0.691, but lb scores are between 0.689 and 0.694. So confusing!

", "rawMarkdown": "Thanks for the response. my cv is 0.691, but lb scores are between 0.689 and 0.694. So confusing! " }, { "id": 462253, "postDate": "2019-01-28T01:05:09.223Z", "content": "

I am also currently at CV 0.691~0.692, best LB is around 0.695, a lot of fluctuations in LB score.

", "rawMarkdown": "I am also currently at CV 0.691~0.692, best LB is around 0.695, a lot of fluctuations in LB score." }, { "id": 462563, "postDate": "2019-01-28T14:02:35.493Z", "content": "

Interesting Jerry @jerrykuo7727 . Would you mind sharing your insight on parameter tuning after the end of competition ?

", "rawMarkdown": "Interesting Jerry @jerrykuo7727 . Would you mind sharing your insight on parameter tuning after the end of competition ? ", "votes": 1 }, { "id": 464865, "postDate": "2019-02-01T17:17:37.120Z", "content": "

Hi @jerrykuo7727 , could you give me some advice on what kind of hyper parameters I should fine tune? My current LB score is around 0.7 and I have no idea on how to further improve my model.

", "rawMarkdown": "Hi @jerrykuo7727 , could you give me some advice on what kind of hyper parameters I should fine tune? My current LB score is around 0.7 and I have no idea on how to further improve my model. " } ] }, { "id": 461834, "postDate": "2019-01-27T05:11:59.740Z", "rawMarkdown": "", "votes": -3, "isDeleted": true } ], "comments": [ { "id": 461874, "author_name": "Seal", "author_url": "", "post_date": "2019-01-27T08:28:09.857000", "content": "

I remembered Mercari competition took less than a day to finalize, but we were only allowed to use CPU kernels. Such long finalizing time of Quora competition could be caused by limited GPU resource, I guess?\nThis competition became motivating for me after adjusting the local CV. My LB score easily jumped from 0.700 to 0.704 in merely two days with a relatively more reliable CV score.\nCan't wait to get another medal and rank up to Expert. I'm super excited now :-)!

", "votes": 1, "replies": [ { "id": 461887, "author_name": "Weiwei Chen", "author_url": "", "post_date": "2019-01-27T09:09:14.213000", "content": "

I'm surprised by your \"easily jumped from 0.700 to 0.704\". Thanks for your motivation. But I'm curious about how you adjusted your local CV. Did you change the seed for splitting or change the model(s)?

", "votes": 1, "replies": [] }, { "id": 461953, "author_name": "Seal", "author_url": "", "post_date": "2019-01-27T10:21:05.047000", "content": "

No fancy tricks. It's more about basic concepts of ML. In order to polish my knowledge, I restudy the part about validation from the ML MOOC I took before . Here's some advice:

\n\n
    \n
  1. Don't judge your model with F1 only. Try to observe the patterns of loss, precision, and recall. (Draw curves and calculate standard deviations for easier observation)
    2. \n
  2. Don't compare two models with average loss and CV metrics only. Compare through every epochs of every folds. You don't want to miss any details there!
    3. \n
  3. If you want to lower the folds and epochs for faster tuning. Try using the best seed in your model as a baseline, and randomize the seed while tuning it.
    4. \n
", "votes": 11, "replies": [] }, { "id": 461985, "author_name": "Weiwei Chen", "author_url": "", "post_date": "2019-01-27T11:25:03.447000", "content": "

Your advice are really helpful and I have a lot of things to try now. Thanks again. But I'm wondering, from 0.700 to 0.704, whether you also modified the preprocessing part or you only tuned the model training part? I would be more surprised if just the model tuning alone can make such a big difference.

", "votes": 0, "replies": [] }, { "id": 462014, "author_name": "Seal", "author_url": "", "post_date": "2019-01-27T13:50:25.387000", "content": "

I didn't make any change to the preprocessing. The fun fact was, I initially made 6 features that seem to perform well on my old CV, but I eventually removed 4 of them according to my new CV approach. On top of that, I fine-tuned the model, the optimizer, and some hyperparams like batch size. A reliable CV really helped me get the most out of my features and model.

", "votes": 1, "replies": [] }, { "id": 462145, "author_name": "MengYe", "author_url": "", "post_date": "2019-01-27T19:10:25.283000", "content": "

can you tell me your local cv?

", "votes": 0, "replies": [] }, { "id": 462162, "author_name": "Seal", "author_url": "", "post_date": "2019-01-27T19:51:50.883000", "content": "

CV --> LB
\n0.680 --> 0.700
\n0.681 --> 0.701
\n0.682 --> 0.702
\n0.683 --> 0.704

", "votes": 0, "replies": [] }, { "id": 462237, "author_name": "MengYe", "author_url": "", "post_date": "2019-01-27T23:45:06.457000", "content": "

Thanks for the response. my cv is 0.691, but lb scores are between 0.689 and 0.694. So confusing!

", "votes": 0, "replies": [] }, { "id": 462253, "author_name": "luyaxin", "author_url": "", "post_date": "2019-01-28T01:05:09.223000", "content": "

I am also currently at CV 0.691~0.692, best LB is around 0.695, a lot of fluctuations in LB score.

", "votes": 0, "replies": [] }, { "id": 462563, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-01-28T14:02:35.493000", "content": "

Interesting Jerry @jerrykuo7727 . Would you mind sharing your insight on parameter tuning after the end of competition ?

", "votes": 1, "replies": [] }, { "id": 464865, "author_name": "ironrro", "author_url": "", "post_date": "2019-02-01T17:17:37.120000", "content": "

Hi @jerrykuo7727 , could you give me some advice on what kind of hyper parameters I should fine tune? My current LB score is around 0.7 and I have no idea on how to further improve my model.

", "votes": 0, "replies": [] } ] }, { "id": 461834, "author_name": "", "author_url": "", "post_date": "2019-01-27T05:11:59.740000", "content": "", "votes": -3, "replies": [] } ], "raw_markdown_by_id": { "461816": "I thought it will be over in 10 days but apparently we'll have to wait for another 6 weeks for the results to finalize. I think this should be a 2-month competitions with 4-6 weeks finalization to make it 3 month like most of the others. Personally I got really unmotivated at crunching out 0.00x halfway through, but this should be my problem as I can see a lot of people making huge improvements in the last few days (team merging also helped maybe?)", "461874": "I remembered Mercari competition took less than a day to finalize, but we were only allowed to use CPU kernels. Such long finalizing time of Quora competition could be caused by limited GPU resource, I guess?\nThis competition became motivating for me after adjusting the local CV. My LB score easily jumped from 0.700 to 0.704 in merely two days with a relatively more reliable CV score.\nCan't wait to get another medal and rank up to Expert. I'm super excited now :-)!", "461834": "" } }