![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1025394%2F1d24af17ebcd9493b56d8d7038b4fae3%2Fflowchart.PNG?generation=1661442385065366&alt=media\")
A quick explaination of my approach is explained by the flowling flowchat:
\n
Generally what I did was mainly the feature engineering, stacking and created 110k+ pseudo-labels for semi-supervised learning at 2nd Level stacking. Model optimizations I havn't performed at all, only used the default.
\nI had however great fun. Gonna take a rest for a couple of days, see you guys soon in the next.
", "messages": [ { "id": "1913954", "postDate": "08/25/2022 15:47:49", "content": "A quick explaination of my approach is explained by the flowling flowchat:
\n
Generally what I did was mainly the feature engineering, stacking and created 110k+ pseudo-labels for semi-supervised learning at 2nd Level stacking. Model optimizations I havn't performed at all, only used the default.
\nI had however great fun. Gonna take a rest for a couple of days, see you guys soon in the next.
", "rawMarkdown": "A quick explaination of my approach is explained by the flowling flowchat:\n\n\n\nGenerally what I did was mainly the feature engineering, stacking and created 110k+ pseudo-labels for semi-supervised learning at 2nd Level stacking. Model optimizations I havn't performed at all, only used the default.\n\n~~As the 1st place in the silver zone, was sure a pity. ~~ I had however great fun. Gonna take a rest for a couple of days, see you guys soon in the next.", "votes": null }, { "id": "1913969", "postDate": "08/25/2022 15:58:10", "content": "Bad luck this time Mengfei. What do you mean by pseudo-label?
", "rawMarkdown": "Bad luck this time Mengfei. What do you mean by pseudo-label?", "votes": null }, { "id": "1913976", "postDate": "08/25/2022 16:01:46", "content": "I used rank-averaged prediction, select >= 0.95 and <=0.05 part, set the target to be 1 and 0 in order to genearte extra data and add them to the trainning dataset of each fold.
", "rawMarkdown": "I used rank-averaged prediction, select >= 0.95 and <=0.05 part, set the target to be 1 and 0 in order to genearte extra data and add them to the trainning dataset of each fold.", "votes": null }, { "id": "1913981", "postDate": "08/25/2022 16:07:11", "content": "understood, interesting thank you!
", "rawMarkdown": "understood, interesting thank you!", "votes": null }, { "id": "1913982", "postDate": "08/25/2022 16:07:24", "content": "Congratulations @meli19!
\nJust want to know, in terms of the CV scores, do they represent the average of the 5 folds or just the highest score in one of them?
thanks, hi, average.
", "rawMarkdown": "thanks, hi, average.", "votes": null }, { "id": "1913996", "postDate": "08/25/2022 16:19:49", "content": "Thank you for sharing your method in this competition and I might have known your general ideas. By the way, how long do you take time for the above? If I make the same, I may need to focus on one or two competitions. Anyway you would be a grand master soon.
", "rawMarkdown": "Thank you for sharing your method in this competition and I might have known your general ideas. By the way, how long do you take time for the above? If I make the same, I may need to focus on one or two competitions. Anyway you would be a grand master soon.", "votes": null }, { "id": "1913999", "postDate": "08/25/2022 16:23:15", "content": "Thank you Daisy. This competetion needs heavy computation resources. I have to work in parallel and take care of my small child. I don't think I can manage more than one competetion at one time. Maybe later when I got more time and better skills, so that I can be like some of the best players managing several projects in parallel. I will try.
", "rawMarkdown": "Thank you Daisy. This competetion needs heavy computation resources. I have to work in parallel and take care of my small child. I don't think I can manage more than one competetion at one time. Maybe later when I got more time and better skills, so that I can be like some of the best players managing several projects in parallel. I will try.", "votes": null }, { "id": "1914006", "postDate": "08/25/2022 16:36:22", "content": "Mengfei, sorry that you missed gold but still it was a strong solo finish, so I want to say congrats!
\nI was wondering if you could give a bit more detail on your NNs. What's your NN structure and what's the secret sauce for achieving such a high CV? My attempts at MLP achieved only CV ~793 and I think it was due to improper representation for missing values. Very curious about how you made it work so well.
Hi, Tonghui, thanks. I don't think I have a special or secrete structures for it. mostly it is I think because of FE and I have used np.log1p() function to standarize the data. rectified Adam instead of Adam as the optimizer. No magic. For stacking, it brings almost no benifit compared to my 0.791 Version, due to lack of the feature diversity.
", "rawMarkdown": "Hi, Tonghui, thanks. I don't think I have a special or secrete structures for it. mostly it is I think because of FE and I have used np.log1p() function to standarize the data. rectified Adam instead of Adam as the optimizer. No magic. For stacking, it brings almost no benifit compared to my 0.791 Version, due to lack of the feature diversity.", "votes": null }, { "id": "1914021", "postDate": "08/25/2022 16:49:32", "content": "Thank you for the reply! How did you deal with missing values?
", "rawMarkdown": "Thank you for the reply! How did you deal with missing values?", "votes": null }, { "id": "1914023", "postDate": "08/25/2022 16:51:05", "content": "I just filled them all with -100 :). np.log1p() * np.sign(df).
", "rawMarkdown": "I just filled them all with -100 :). np.log1p() * np.sign(df).", "votes": null }, { "id": "1914061", "postDate": "08/25/2022 17:32:10", "content": "Hi, @meli19 congrats on your achievement Doing so much single-handedly is really an inspiration for me, can you please tell the reason behind choosing np.log1p() * np.sign(df) as standardization also what scheduler did you use with rectified adam?
", "rawMarkdown": "Hi, @meli19 congrats on your achievement Doing so much single-handedly is really an inspiration for me, can you please tell the reason behind choosing **np.log1p() * np.sign(df)** as standardization also what scheduler did you use with rectified adam?", "votes": null }, { "id": "1914069", "postDate": "08/25/2022 17:42:04", "content": "@chaudharypriyanshu, I cannot explain to be honest, it was an experiment and performed much better with my features compared to max_min or standard sclaer;
\ntalking about rectified adam, I didn't adjust too much. opt = tfa.optimizers.RectifiedAdam(\n lr=1e-5,\n total_steps=10000,\n warmup_proportion=0.1,\n min_lr=1e-8,\n) similar to that. I have to start my workingstation to find it. maybe tomorrow I can tell you, today I just relax with my laptop.
Hearty congratulations for the approach and the result it gave you! Your efforts are commendable!
", "rawMarkdown": "Hearty congratulations for the approach and the result it gave you! Your efforts are commendable!", "votes": null }, { "id": "1914138", "postDate": "08/25/2022 19:02:17", "content": "Thank you !
", "rawMarkdown": "Thank you !", "votes": null }, { "id": "1915294", "postDate": "08/26/2022 20:41:38", "content": "Congratulations @meli19 winning solo gold!
", "rawMarkdown": "Congratulations @meli19 winning solo gold!", "votes": null }, { "id": "1915297", "postDate": "08/26/2022 20:48:53", "content": "Hi, Chris, thank you. I also noticed that :) I am feeling great now!
", "rawMarkdown": "Hi, Chris, thank you. I also noticed that :) I am feeling great now!", "votes": null }, { "id": "1915298", "postDate": "08/26/2022 20:49:01", "content": "@meli19 Mengfei, sorry that you missed gold I am so happy that I can take this back!! Congrats on the solo gold!!!
Thanks Tonghui !
", "rawMarkdown": "Thanks Tonghui !", "votes": null }, { "id": "1915310", "postDate": "08/26/2022 21:18:23", "content": "congrats @meli19 happy u won gold 🙌
", "rawMarkdown": "congrats @meli19 happy u won gold 🙌", "votes": null }, { "id": "1915314", "postDate": "08/26/2022 21:27:06", "content": ":) :) ty ty
", "rawMarkdown": ":) :) ty ty", "votes": null }, { "id": "1915319", "postDate": "08/26/2022 21:29:56", "content": "Correction… good luck this time congrats!
", "rawMarkdown": "Correction… good luck this time congrats!", "votes": null }, { "id": "1915423", "postDate": "08/27/2022 00:55:24", "content": "Hey, Congratulations🎉!! I was sad for you to be honest you deserved to be in the gold range, but now it came true. Cheers!!
", "rawMarkdown": "Hey, Congratulations🎉!! I was sad for you to be honest you deserved to be in the gold range, but now it came true. Cheers!!", "votes": null }, { "id": "1915429", "postDate": "08/27/2022 01:06:08", "content": "Congrats on the edit :)
", "rawMarkdown": "Congrats on the edit :)", "votes": null }, { "id": "1932755", "postDate": "09/09/2022 22:22:52", "content": "Thanks for sharing and congratulations for finally getting a gold! What do you mean with S_2_standarization?
", "rawMarkdown": "Thanks for sharing and congratulations for finally getting a gold! What do you mean with S_2_standarization?", "votes": null }, { "id": "1933040", "postDate": "09/10/2022 08:06:51", "content": "Hi, thanks, I meant: convert S_2 to numerical values and do a / (S_2.max()-S_2.min()) for each coustomer_id.
", "rawMarkdown": "Hi, thanks, I meant: convert S_2 to numerical values and do a / (S_2.max()-S_2.min()) for each coustomer_id.", "votes": null } ], "comments": [ { "id": 1913969, "author_name": "joseantonioalatorre", "author_url": "", "post_date": "08/25/2022 15:58:10", "content": "Bad luck this time Mengfei. What do you mean by pseudo-label?
", "votes": null, "replies": [ { "id": 1913976, "author_name": "meli19", "author_url": "", "post_date": "08/25/2022 16:01:46", "content": "I used rank-averaged prediction, select >= 0.95 and <=0.05 part, set the target to be 1 and 0 in order to genearte extra data and add them to the trainning dataset of each fold.
", "votes": null, "replies": [] }, { "id": 1913981, "author_name": "joseantonioalatorre", "author_url": "", "post_date": "08/25/2022 16:07:11", "content": "understood, interesting thank you!
", "votes": null, "replies": [] }, { "id": 1915319, "author_name": "joseantonioalatorre", "author_url": "", "post_date": "08/26/2022 21:29:56", "content": "Correction… good luck this time congrats!
", "votes": null, "replies": [] } ] }, { "id": 1913982, "author_name": "mohammad2012191", "author_url": "", "post_date": "08/25/2022 16:07:24", "content": "Congratulations @meli19!
\nJust want to know, in terms of the CV scores, do they represent the average of the 5 folds or just the highest score in one of them?
thanks, hi, average.
", "votes": null, "replies": [] } ] }, { "id": 1913996, "author_name": "hdynamics", "author_url": "", "post_date": "08/25/2022 16:19:49", "content": "Thank you for sharing your method in this competition and I might have known your general ideas. By the way, how long do you take time for the above? If I make the same, I may need to focus on one or two competitions. Anyway you would be a grand master soon.
", "votes": null, "replies": [ { "id": 1913999, "author_name": "meli19", "author_url": "", "post_date": "08/25/2022 16:23:15", "content": "Thank you Daisy. This competetion needs heavy computation resources. I have to work in parallel and take care of my small child. I don't think I can manage more than one competetion at one time. Maybe later when I got more time and better skills, so that I can be like some of the best players managing several projects in parallel. I will try.
", "votes": null, "replies": [] } ] }, { "id": 1914006, "author_name": "raphael1123", "author_url": "", "post_date": "08/25/2022 16:36:22", "content": "Mengfei, sorry that you missed gold but still it was a strong solo finish, so I want to say congrats!
\nI was wondering if you could give a bit more detail on your NNs. What's your NN structure and what's the secret sauce for achieving such a high CV? My attempts at MLP achieved only CV ~793 and I think it was due to improper representation for missing values. Very curious about how you made it work so well.
Hi, Tonghui, thanks. I don't think I have a special or secrete structures for it. mostly it is I think because of FE and I have used np.log1p() function to standarize the data. rectified Adam instead of Adam as the optimizer. No magic. For stacking, it brings almost no benifit compared to my 0.791 Version, due to lack of the feature diversity.
", "votes": null, "replies": [] }, { "id": 1914021, "author_name": "raphael1123", "author_url": "", "post_date": "08/25/2022 16:49:32", "content": "Thank you for the reply! How did you deal with missing values?
", "votes": null, "replies": [] }, { "id": 1914023, "author_name": "meli19", "author_url": "", "post_date": "08/25/2022 16:51:05", "content": "I just filled them all with -100 :). np.log1p() * np.sign(df).
", "votes": null, "replies": [] }, { "id": 1914061, "author_name": "chaudharypriyanshu", "author_url": "", "post_date": "08/25/2022 17:32:10", "content": "Hi, @meli19 congrats on your achievement Doing so much single-handedly is really an inspiration for me, can you please tell the reason behind choosing np.log1p() * np.sign(df) as standardization also what scheduler did you use with rectified adam?
", "votes": null, "replies": [] }, { "id": 1914069, "author_name": "meli19", "author_url": "", "post_date": "08/25/2022 17:42:04", "content": "@chaudharypriyanshu, I cannot explain to be honest, it was an experiment and performed much better with my features compared to max_min or standard sclaer;
\ntalking about rectified adam, I didn't adjust too much. opt = tfa.optimizers.RectifiedAdam(\n lr=1e-5,\n total_steps=10000,\n warmup_proportion=0.1,\n min_lr=1e-8,\n) similar to that. I have to start my workingstation to find it. maybe tomorrow I can tell you, today I just relax with my laptop.
@meli19 Mengfei, sorry that you missed gold I am so happy that I can take this back!! Congrats on the solo gold!!!
Thanks Tonghui !
", "votes": null, "replies": [] } ] }, { "id": 1914134, "author_name": "ravi20076", "author_url": "", "post_date": "08/25/2022 18:57:37", "content": "Hearty congratulations for the approach and the result it gave you! Your efforts are commendable!
", "votes": null, "replies": [ { "id": 1914138, "author_name": "meli19", "author_url": "", "post_date": "08/25/2022 19:02:17", "content": "Thank you !
", "votes": null, "replies": [] } ] }, { "id": 1915294, "author_name": "cdeotte", "author_url": "", "post_date": "08/26/2022 20:41:38", "content": "Congratulations @meli19 winning solo gold!
", "votes": null, "replies": [ { "id": 1915297, "author_name": "meli19", "author_url": "", "post_date": "08/26/2022 20:48:53", "content": "Hi, Chris, thank you. I also noticed that :) I am feeling great now!
", "votes": null, "replies": [] } ] }, { "id": 1915310, "author_name": "gauravbrills", "author_url": "", "post_date": "08/26/2022 21:18:23", "content": "congrats @meli19 happy u won gold 🙌
", "votes": null, "replies": [ { "id": 1915314, "author_name": "meli19", "author_url": "", "post_date": "08/26/2022 21:27:06", "content": ":) :) ty ty
", "votes": null, "replies": [] } ] }, { "id": 1915423, "author_name": "chaudharypriyanshu", "author_url": "", "post_date": "08/27/2022 00:55:24", "content": "Hey, Congratulations🎉!! I was sad for you to be honest you deserved to be in the gold range, but now it came true. Cheers!!
", "votes": null, "replies": [] }, { "id": 1915429, "author_name": "roberthatch", "author_url": "", "post_date": "08/27/2022 01:06:08", "content": "Congrats on the edit :)
", "votes": null, "replies": [] }, { "id": 1932755, "author_name": "delai50", "author_url": "", "post_date": "09/09/2022 22:22:52", "content": "Thanks for sharing and congratulations for finally getting a gold! What do you mean with S_2_standarization?
", "votes": null, "replies": [ { "id": 1933040, "author_name": "meli19", "author_url": "", "post_date": "09/10/2022 08:06:51", "content": "Hi, thanks, I meant: convert S_2 to numerical values and do a / (S_2.max()-S_2.min()) for each coustomer_id.
", "votes": null, "replies": [] } ] } ], "raw_markdown_by_id": { "1913954": "A quick explaination of my approach is explained by the flowling flowchat:\n\n\n\nGenerally what I did was mainly the feature engineering, stacking and created 110k+ pseudo-labels for semi-supervised learning at 2nd Level stacking. Model optimizations I havn't performed at all, only used the default.\n\n~~As the 1st place in the silver zone, was sure a pity. ~~ I had however great fun. Gonna take a rest for a couple of days, see you guys soon in the next.", "1913969": "Bad luck this time Mengfei. What do you mean by pseudo-label?", "1913976": "I used rank-averaged prediction, select >= 0.95 and <=0.05 part, set the target to be 1 and 0 in order to genearte extra data and add them to the trainning dataset of each fold.", "1913981": "understood, interesting thank you!", "1913982": "Congratulations @meli19!\nJust want to know, in terms of the CV scores, do they represent the average of the 5 folds or just the highest score in one of them?", "1913985": "thanks, hi, average.", "1913996": "Thank you for sharing your method in this competition and I might have known your general ideas. By the way, how long do you take time for the above? If I make the same, I may need to focus on one or two competitions. Anyway you would be a grand master soon.", "1913999": "Thank you Daisy. This competetion needs heavy computation resources. I have to work in parallel and take care of my small child. I don't think I can manage more than one competetion at one time. Maybe later when I got more time and better skills, so that I can be like some of the best players managing several projects in parallel. I will try.", "1914006": "Mengfei, sorry that you missed gold but still it was a strong solo finish, so I want to say congrats!\nI was wondering if you could give a bit more detail on your NNs. What's your NN structure and what's the secret sauce for achieving such a high CV? My attempts at MLP achieved only CV ~793 and I think it was due to improper representation for missing values. Very curious about how you made it work so well.", "1914011": "Hi, Tonghui, thanks. I don't think I have a special or secrete structures for it. mostly it is I think because of FE and I have used np.log1p() function to standarize the data. rectified Adam instead of Adam as the optimizer. No magic. For stacking, it brings almost no benifit compared to my 0.791 Version, due to lack of the feature diversity.", "1914021": "Thank you for the reply! How did you deal with missing values?", "1914023": "I just filled them all with -100 :). np.log1p() * np.sign(df).", "1914061": "Hi, @meli19 congrats on your achievement Doing so much single-handedly is really an inspiration for me, can you please tell the reason behind choosing **np.log1p() * np.sign(df)** as standardization also what scheduler did you use with rectified adam?", "1914069": "chaudharypriyanshu, I cannot explain to be honest, it was an experiment and performed much better with my features compared to max_min or standard sclaer;\n\ntalking about rectified adam, I didn't adjust too much. `opt = tfa.optimizers.RectifiedAdam(\n lr=1e-5,\n total_steps=10000,\n warmup_proportion=0.1,\n min_lr=1e-8,\n)` similar to that. I have to start my workingstation to find it. maybe tomorrow I can tell you, today I just relax with my laptop.", "1914134": "Hearty congratulations for the approach and the result it gave you! Your efforts are commendable!", "1914138": "Thank you !", "1915294": "Congratulations @meli19 winning solo gold!", "1915297": "Hi, Chris, thank you. I also noticed that :) I am feeling great now!", "1915298": "meli19 `Mengfei, sorry that you missed gold` I am so happy that I can take this back!! Congrats on the solo gold!!!", "1915299": "Thanks Tonghui !", "1915310": "congrats @meli19 happy u won gold 🙌", "1915314": ":) :) ty ty", "1915319": "Correction… good luck this time congrats!", "1915423": "Hey, Congratulations🎉!! I was sad for you to be honest you deserved to be in the gold range, but now it came true. Cheers!!", "1915429": "Congrats on the edit :)", "1932755": "Thanks for sharing and congratulations for finally getting a gold! What do you mean with S_2_standarization?", "1933040": "Hi, thanks, I meant: convert S_2 to numerical values and do a / (S_2.max()-S_2.min()) for each coustomer_id." }, "source": "meta" }