![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2Fae1a93d87f1bea8729689880d87334e8%2Fslide2.jpg?generation=1593041342691529&alt=media\")
I shared some tips and tricks in a live talk on YouTube Thursday Jun 25th, 10AM PDT. And Abhishek gave a live talk at 10:30AM. Both sessions were recorded and have been posted at https://youtu.be/DEuvGh4ZwaY
\n\nWe discussed how to approach Kaggle competitions. IMO, the two most important things are (1) reliable local validation scheme (2) fast experimentation. Using accelerators such as GPUs are important for fast experimentation.
\n\nExperimentation is done in five areas (1) preprocess and feature engineering (2) data augmentation and external datasets (3) model and loss (4) learning schedule and optimizer (5) post process.
\n\nRecord all your experiments and save all your models. Then use your best model to predict Kaggle's test dataset and submit. Also consider ensembling your best models. During my talk we worked together using my starter notebook posted here.
\n\nEnjoy!
\n\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1723677%2Fa5556375572e6bf196877ee7db166ac6%2Fslide3.jpg?generation=1593041352585432&alt=media\")
I shared some tips and tricks in a live talk on YouTube Thursday Jun 25th, 10AM PDT. And Abhishek gave a live talk at 10:30AM. Both sessions were recorded and have been posted at https://youtu.be/DEuvGh4ZwaY
\n\nWe discussed how to approach Kaggle competitions. IMO, the two most important things are (1) reliable local validation scheme (2) fast experimentation. Using accelerators such as GPUs are important for fast experimentation.
\n\nExperimentation is done in five areas (1) preprocess and feature engineering (2) data augmentation and external datasets (3) model and loss (4) learning schedule and optimizer (5) post process.
\n\nRecord all your experiments and save all your models. Then use your best model to predict Kaggle's test dataset and submit. Also consider ensembling your best models. During my talk we worked together using my starter notebook posted here.
\n\nEnjoy!
\n\n
Today's Accelerator Power Hour was a huge success. There were about 1000 viewers in the audience. I discussed many competition tips and tricks using TensorFlow GPU and Abhishek showed how to code with PyTorch TPU.
\n\nIf you missed the live stream, you can watch the recorded stream on YouTube here: https://youtu.be/DEuvGh4ZwaY
", "rawMarkdown": "Today's Accelerator Power Hour was a huge success. There were about 1000 viewers in the audience. I discussed many competition tips and tricks using TensorFlow GPU and Abhishek showed how to code with PyTorch TPU.\n\n If you missed the live stream, you can watch the recorded stream on YouTube here: https://youtu.be/DEuvGh4ZwaY", "votes": 5, "replies": [ { "id": 902052, "postDate": "2020-06-25T22:23:27.113Z", "content": "Yes, it was very good. Hope to see more of this in the future.
", "rawMarkdown": "Yes, it was very good. Hope to see more of this in the future.", "votes": 1 } ] }, { "id": 900754, "postDate": "2020-06-25T03:37:53.643Z", "content": "I have also received a mail from kaggle about your conversation about the above. I will definitely follow your sharing, it will definitely help me for upcoming competitions and even for this one ^ ^
", "rawMarkdown": "I have also received a mail from kaggle about your conversation about the above. I will definitely follow your sharing, it will definitely help me for upcoming competitions and even for this one ^ ^", "votes": 3 }, { "id": 906088, "postDate": "2020-06-29T04:21:23.353Z", "content": "Hi Chris. In the sharing regarding to your initial learning rate of 0, you explained that it is because pre-trained models have some intelligence and we do not want to a high initial learning rate to affect the network weights too much... but training through one epoch with LR of 0 actually does not do anything to the networks, is it a waste of resource? Should we start with very small learning rate instead?
", "rawMarkdown": "Hi Chris. In the sharing regarding to your initial learning rate of 0, you explained that it is because pre-trained models have some intelligence and we do not want to a high initial learning rate to affect the network weights too much... but training through one epoch with LR of 0 actually does not do anything to the networks, is it a waste of resource? Should we start with very small learning rate instead?", "votes": 4, "replies": [ { "id": 906648, "postDate": "2020-06-29T13:18:21.940Z", "content": "Yes, you are very observant. Training with LR = 0 is pointless. I explained it wrong in my talk. The first epoch doesn't use 0, it uses 1e-5, then second epoch is 0.0002, then third is 0.0004, then fourth is 0.0006, then fifth is 0.0008, then sixth is 0.001. Then plateau for 10 epochs. Then step decay.
\n\nThere is a variable LR_START = 1e-5, so the first LR was max(0,LR_START) and not zero.
Thanks for sharing! 👍
", "rawMarkdown": "Thanks for sharing! 👍 ", "votes": 1 } ] }, { "id": 900618, "postDate": "2020-06-25T00:12:23.547Z", "content": "Unfortunately, I'll be at work (stacking potatoes). Will this be available in some other form for later consumption?
", "rawMarkdown": "Unfortunately, I'll be at work (stacking potatoes). Will this be available in some other form for later consumption?", "votes": 4, "replies": [ { "id": 900620, "postDate": "2020-06-25T00:16:16.277Z", "content": "Yes, i believe it will stay on YouTube afterward
", "rawMarkdown": "Yes, i believe it will stay on YouTube afterward", "votes": 2 } ] }, { "id": 903506, "postDate": "2020-06-26T22:18:44.113Z", "content": "Cool! Thanks for sharing! It's very informative! 👍 👍
", "rawMarkdown": "Cool! Thanks for sharing! It's very informative! 👍 👍 ", "votes": 1 }, { "id": 902966, "postDate": "2020-06-26T13:43:49.753Z", "content": "That was an amazing presentation @cdeotte , great job! you have amazing talking skill, I hope we can see more of that in the future!
", "rawMarkdown": "That was an amazing presentation @cdeotte , great job! you have amazing talking skill, I hope we can see more of that in the future!", "votes": 1, "replies": [ { "id": 903113, "postDate": "2020-06-26T15:23:06.793Z", "content": "Thanks
", "rawMarkdown": "Thanks", "votes": 1 } ] }, { "id": 902537, "postDate": "2020-06-26T07:57:49.830Z", "content": "I enjoyed the talk! thank you Chris and Abhishek
", "rawMarkdown": "I enjoyed the talk! thank you Chris and Abhishek", "votes": 1 }, { "id": 902370, "postDate": "2020-06-26T05:27:33.177Z", "content": "Thanks great talk! Attended live at 03:00 hours from Sydney and found it totally worth it!
\n\nI don't understand the post processing bit, are there any resources that you could guide me to please? How did you end up using three different mat and using dot products and how does this maximise CV score please?
I explain the post processing in the discussion post here
", "rawMarkdown": "I explain the post processing in the discussion post [here][1]\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021", "votes": 1 }, { "id": 903465, "postDate": "2020-06-26T21:02:54.613Z", "content": "Thank you ! :)
", "rawMarkdown": "Thank you ! :)", "votes": 1 } ] }, { "id": 902299, "postDate": "2020-06-26T03:55:21.740Z", "content": "Great talk! I especially liked your idea of applying transfer learning to humans :-) ! Nice joke!
", "rawMarkdown": "Great talk! I especially liked your idea of applying transfer learning to humans :-) ! Nice joke!", "votes": 1, "replies": [ { "id": 902311, "postDate": "2020-06-26T04:18:27.710Z", "content": "😄 thanks
", "rawMarkdown": "😄 thanks", "votes": 1 } ] }, { "id": 901443, "postDate": "2020-06-25T13:41:36.097Z", "content": "subscribed... looking forward to learn from best player
", "rawMarkdown": "subscribed... looking forward to learn from best player", "votes": 1 }, { "id": 906544, "postDate": "2020-06-29T12:01:55.587Z", "content": "Thanks @cdeotte for all the knowledge you are sharing! \nI read the top solutions of the IEEE-CIS Fraud Detection competition where you applied post processing to AUC.
\nBoth competitions have something in common, we have:\n* Many observations of the same patient in this comp.\n* Many transactions of the same customer in the IEEE-CIS Fraud Detection comp.
Do you think the idea of aggregating the predictions of the same patient as you did in the other comp. (the malignant ones) can somehow boost the score? I tried but it drastically lowered the score to 0.81 AUC :)
", "rawMarkdown": "Thanks @cdeotte for all the knowledge you are sharing! \nI read the top solutions of the IEEE-CIS Fraud Detection competition where you applied post processing to AUC. \nBoth competitions have something in common, we have:\n* Many observations of the same patient in this comp.\n* Many transactions of the same customer in the IEEE-CIS Fraud Detection comp. \n\nDo you think the idea of aggregating the predictions of the same patient as you did in the other comp. (the malignant ones) can somehow boost the score? I tried but it drastically lowered the score to 0.81 AUC :)", "votes": 2, "replies": [ { "id": 906656, "postDate": "2020-06-29T13:21:29.157Z", "content": "Great idea Amin. I also believe there is a way to use patient id for post process. We cannot average all the same patient because the same patient has both benign and malignant. But if we can figure out which images of patient are benign and which are malignant. We can post process average those two groups separately.
\n\nI have not tried this yet, but this is one type of post process i plan to experiment with.
", "rawMarkdown": "Great idea Amin. I also believe there is a way to use patient id for post process. We cannot average all the same patient because the same patient has both benign and malignant. But if we can figure out which images of patient are benign and which are malignant. We can post process average those two groups separately.\n\nI have not tried this yet, but this is one type of post process i plan to experiment with.", "votes": 1 } ] }, { "id": 905537, "postDate": "2020-06-28T16:14:19.380Z", "content": "very very good talk Chris, very funny, deep and entertaining.\nyou showed a learning rate scheduler to tame a pre-trained model ... does the same apply when you need to resume the training of a model that due to hardware limitations takes a long time during each training epoch?
", "rawMarkdown": "very very good talk Chris, very funny, deep and entertaining.\nyou showed a learning rate scheduler to tame a pre-trained model ... does the same apply when you need to resume the training of a model that due to hardware limitations takes a long time during each training epoch?", "votes": 2, "replies": [ { "id": 905546, "postDate": "2020-06-28T16:22:16.040Z", "content": "When i resume training, i just start the learning rate from where it left off and continue the schedule from there. (This works well if you want to train models for more than 9 hours in Kaggle notebooks). But perhaps using a few epochs to warm up to that left off rate would work well too.
", "rawMarkdown": "When i resume training, i just start the learning rate from where it left off and continue the schedule from there. (This works well if you want to train models for more than 9 hours in Kaggle notebooks). But perhaps using a few epochs to warm up to that left off rate would work well too.", "votes": 3 }, { "id": 905637, "postDate": "2020-06-28T17:31:23.923Z", "content": "👍
", "rawMarkdown": "👍 " } ] }, { "id": 1379116, "postDate": "2021-07-07T05:43:59.697Z", "content": "For those guys who found Kaggle's GPU is not sufficient, I recommend an MLOps tool called AIbro. The company is giving a lot of free credit to use cloud GPU. You can use it to train AI models on any instance from AWS in one line. https://colab.research.google.com/drive/19sXZ4kbic681zqEsrl_CZfB5cegUwuIB#scrollTo=CSgozfXs572M
", "rawMarkdown": "For those guys who found Kaggle's GPU is not sufficient, I recommend an MLOps tool called AIbro. The company is giving a lot of free credit to use cloud GPU. You can use it to train AI models on any instance from AWS in one line. https://colab.research.google.com/drive/19sXZ4kbic681zqEsrl_CZfB5cegUwuIB#scrollTo=CSgozfXs572M" }, { "id": 967792, "postDate": "2020-08-12T14:08:16.363Z", "content": "Hi! I am new here. I have some python computations, and these take 1 Hour by CPU. Can you help me how to use my GPU for this? (these python computations are not ML model, just computing in big data)
", "rawMarkdown": "Hi! I am new here. I have some python computations, and these take 1 Hour by CPU. Can you help me how to use my GPU for this? (these python computations are not ML model, just computing in big data) " }, { "id": 905372, "postDate": "2020-06-28T13:55:29.130Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 905397, "postDate": "2020-06-28T14:19:35.340Z", "content": "Yes, the trick specifically works on macro recall. I explain more detail here
", "rawMarkdown": "Yes, the trick specifically works on macro recall. I explain more detail [here][1]\n\n[1]: https://www.kaggle.com/c/bengaliai-cv19/discussion/136021" }, { "id": 905412, "postDate": "2020-06-28T14:32:19.003Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 903827, "postDate": "2020-06-27T06:16:34.523Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 904399, "postDate": "2020-06-27T15:37:02.113Z", "rawMarkdown": "", "isDeleted": true }, { "id": 904770, "postDate": "2020-06-27T22:44:56.083Z", "content": "Hi Ayaan. My teammate choices depend on the comp and my goals at the time. If i want to win, then i'll choose teammates who are currently doing well and indicate knowing something important. If i want to have fun and just learn stuff then I'll choose teammates who demonstrate they are both knowledgeable and fun to work with.
", "rawMarkdown": "Hi Ayaan. My teammate choices depend on the comp and my goals at the time. If i want to win, then i'll choose teammates who are currently doing well and indicate knowing something important. If i want to have fun and just learn stuff then I'll choose teammates who demonstrate they are both knowledgeable and fun to work with." }, { "id": 904846, "postDate": "2020-06-28T02:17:21.843Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] }, { "id": 902209, "postDate": "2020-06-26T01:56:39.693Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 902235, "postDate": "2020-06-26T02:43:19.293Z", "content": "Thanks Synked
", "rawMarkdown": "Thanks Synked" } ] }, { "id": 900610, "postDate": "2020-06-24T23:53:02.570Z", "content": "thanks for reminding
", "rawMarkdown": "thanks for reminding", "votes": 1 } ], "comments": [ { "id": 902042, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-25T22:07:00.763000", "content": "Today's Accelerator Power Hour was a huge success. There were about 1000 viewers in the audience. I discussed many competition tips and tricks using TensorFlow GPU and Abhishek showed how to code with PyTorch TPU.
\n\nIf you missed the live stream, you can watch the recorded stream on YouTube here: https://youtu.be/DEuvGh4ZwaY
", "votes": 5, "replies": [ { "id": 902052, "author_name": "Anon", "author_url": "", "post_date": "2020-06-25T22:23:27.113000", "content": "Yes, it was very good. Hope to see more of this in the future.
", "votes": 1, "replies": [] } ] }, { "id": 900754, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-06-25T03:37:53.643000", "content": "I have also received a mail from kaggle about your conversation about the above. I will definitely follow your sharing, it will definitely help me for upcoming competitions and even for this one ^ ^
", "votes": 3, "replies": [] }, { "id": 906088, "author_name": "ZHU CHAO", "author_url": "", "post_date": "2020-06-29T04:21:23.353000", "content": "Hi Chris. In the sharing regarding to your initial learning rate of 0, you explained that it is because pre-trained models have some intelligence and we do not want to a high initial learning rate to affect the network weights too much... but training through one epoch with LR of 0 actually does not do anything to the networks, is it a waste of resource? Should we start with very small learning rate instead?
", "votes": 4, "replies": [ { "id": 906648, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-29T13:18:21.940000", "content": "Yes, you are very observant. Training with LR = 0 is pointless. I explained it wrong in my talk. The first epoch doesn't use 0, it uses 1e-5, then second epoch is 0.0002, then third is 0.0004, then fourth is 0.0006, then fifth is 0.0008, then sixth is 0.001. Then plateau for 10 epochs. Then step decay.
\n\nThere is a variable LR_START = 1e-5, so the first LR was max(0,LR_START) and not zero.
Thanks for sharing! 👍
", "votes": 1, "replies": [] } ] }, { "id": 900618, "author_name": "Bruce Young", "author_url": "", "post_date": "2020-06-25T00:12:23.547000", "content": "Unfortunately, I'll be at work (stacking potatoes). Will this be available in some other form for later consumption?
", "votes": 4, "replies": [ { "id": 900620, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-25T00:16:16.277000", "content": "Yes, i believe it will stay on YouTube afterward
", "votes": 2, "replies": [] } ] }, { "id": 903506, "author_name": "Helen", "author_url": "", "post_date": "2020-06-26T22:18:44.113000", "content": "Cool! Thanks for sharing! It's very informative! 👍 👍
", "votes": 1, "replies": [] }, { "id": 902966, "author_name": "DimitreOliveira", "author_url": "", "post_date": "2020-06-26T13:43:49.753000", "content": "That was an amazing presentation @cdeotte , great job! you have amazing talking skill, I hope we can see more of that in the future!
", "votes": 1, "replies": [ { "id": 903113, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-26T15:23:06.793000", "content": "Thanks
", "votes": 1, "replies": [] } ] }, { "id": 902537, "author_name": "Seifeddine Fezzani", "author_url": "", "post_date": "2020-06-26T07:57:49.830000", "content": "I enjoyed the talk! thank you Chris and Abhishek
", "votes": 1, "replies": [] }, { "id": 902370, "author_name": "Aman Arora", "author_url": "", "post_date": "2020-06-26T05:27:33.177000", "content": "Thanks great talk! Attended live at 03:00 hours from Sydney and found it totally worth it!
\n\nI don't understand the post processing bit, are there any resources that you could guide me to please? How did you end up using three different mat and using dot products and how does this maximise CV score please?
I explain the post processing in the discussion post here
", "votes": 1, "replies": [] }, { "id": 903465, "author_name": "Aman Arora", "author_url": "", "post_date": "2020-06-26T21:02:54.613000", "content": "Thank you ! :)
", "votes": 1, "replies": [] } ] }, { "id": 902299, "author_name": "Alexey Pronin", "author_url": "", "post_date": "2020-06-26T03:55:21.740000", "content": "Great talk! I especially liked your idea of applying transfer learning to humans :-) ! Nice joke!
", "votes": 1, "replies": [ { "id": 902311, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-26T04:18:27.710000", "content": "😄 thanks
", "votes": 1, "replies": [] } ] }, { "id": 901443, "author_name": "ZHU CHAO", "author_url": "", "post_date": "2020-06-25T13:41:36.097000", "content": "subscribed... looking forward to learn from best player
", "votes": 1, "replies": [] }, { "id": 906544, "author_name": "Amin", "author_url": "", "post_date": "2020-06-29T12:01:55.587000", "content": "Thanks @cdeotte for all the knowledge you are sharing! \nI read the top solutions of the IEEE-CIS Fraud Detection competition where you applied post processing to AUC.
\nBoth competitions have something in common, we have:\n* Many observations of the same patient in this comp.\n* Many transactions of the same customer in the IEEE-CIS Fraud Detection comp.
Do you think the idea of aggregating the predictions of the same patient as you did in the other comp. (the malignant ones) can somehow boost the score? I tried but it drastically lowered the score to 0.81 AUC :)
", "votes": 2, "replies": [ { "id": 906656, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-29T13:21:29.157000", "content": "Great idea Amin. I also believe there is a way to use patient id for post process. We cannot average all the same patient because the same patient has both benign and malignant. But if we can figure out which images of patient are benign and which are malignant. We can post process average those two groups separately.
\n\nI have not tried this yet, but this is one type of post process i plan to experiment with.
", "votes": 1, "replies": [] } ] }, { "id": 905537, "author_name": "Marcelo Kittlein", "author_url": "", "post_date": "2020-06-28T16:14:19.380000", "content": "very very good talk Chris, very funny, deep and entertaining.\nyou showed a learning rate scheduler to tame a pre-trained model ... does the same apply when you need to resume the training of a model that due to hardware limitations takes a long time during each training epoch?
", "votes": 2, "replies": [ { "id": 905546, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-28T16:22:16.040000", "content": "When i resume training, i just start the learning rate from where it left off and continue the schedule from there. (This works well if you want to train models for more than 9 hours in Kaggle notebooks). But perhaps using a few epochs to warm up to that left off rate would work well too.
", "votes": 3, "replies": [] }, { "id": 905637, "author_name": "Marcelo Kittlein", "author_url": "", "post_date": "2020-06-28T17:31:23.923000", "content": "👍
", "votes": 0, "replies": [] } ] }, { "id": 1379116, "author_name": "Yuki Lee", "author_url": "", "post_date": "2021-07-07T05:43:59.697000", "content": "For those guys who found Kaggle's GPU is not sufficient, I recommend an MLOps tool called AIbro. The company is giving a lot of free credit to use cloud GPU. You can use it to train AI models on any instance from AWS in one line. https://colab.research.google.com/drive/19sXZ4kbic681zqEsrl_CZfB5cegUwuIB#scrollTo=CSgozfXs572M
", "votes": 0, "replies": [] }, { "id": 967792, "author_name": "AmirLashkari", "author_url": "", "post_date": "2020-08-12T14:08:16.363000", "content": "Hi! I am new here. I have some python computations, and these take 1 Hour by CPU. Can you help me how to use my GPU for this? (these python computations are not ML model, just computing in big data)
", "votes": 0, "replies": [] }, { "id": 905372, "author_name": "", "author_url": "", "post_date": "2020-06-28T13:55:29.130000", "content": "", "votes": 1, "replies": [ { "id": 905397, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-28T14:19:35.340000", "content": "Yes, the trick specifically works on macro recall. I explain more detail here
", "votes": 0, "replies": [] }, { "id": 905412, "author_name": "", "author_url": "", "post_date": "2020-06-28T14:32:19.003000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 903827, "author_name": "", "author_url": "", "post_date": "2020-06-27T06:16:34.523000", "content": "", "votes": 1, "replies": [ { "id": 904399, "author_name": "", "author_url": "", "post_date": "2020-06-27T15:37:02.113000", "content": "", "votes": 0, "replies": [] }, { "id": 904770, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-27T22:44:56.083000", "content": "Hi Ayaan. My teammate choices depend on the comp and my goals at the time. If i want to win, then i'll choose teammates who are currently doing well and indicate knowing something important. If i want to have fun and just learn stuff then I'll choose teammates who demonstrate they are both knowledgeable and fun to work with.
", "votes": 0, "replies": [] }, { "id": 904846, "author_name": "", "author_url": "", "post_date": "2020-06-28T02:17:21.843000", "content": "", "votes": 1, "replies": [] } ] }, { "id": 902209, "author_name": "", "author_url": "", "post_date": "2020-06-26T01:56:39.693000", "content": "", "votes": 1, "replies": [ { "id": 902235, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-06-26T02:43:19.293000", "content": "Thanks Synked
", "votes": 0, "replies": [] } ] }, { "id": 900610, "author_name": "somuSan", "author_url": "", "post_date": "2020-06-24T23:53:02.570000", "content": "thanks for reminding
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "900602": "# Accelerator Power Hour with Kaggle Grandmasters!\n\nI shared some tips and tricks in a live talk on YouTube Thursday Jun 25th, 10AM PDT. And Abhishek gave a live talk at 10:30AM. Both sessions were recorded and have been posted at https://youtu.be/DEuvGh4ZwaY \n\nWe discussed how to approach Kaggle competitions. IMO, the two most important things are (1) reliable local validation scheme (2) fast experimentation. Using accelerators such as GPUs are important for fast experimentation.\n\nExperimentation is done in five areas (1) preprocess and feature engineering (2) data augmentation and external datasets (3) model and loss (4) learning schedule and optimizer (5) post process.\n\nRecord all your experiments and save all your models. Then use your best model to predict Kaggle's test dataset and submit. Also consider ensembling your best models. During my talk we worked together using my starter notebook posted [here][1].\n\nEnjoy!\n\n\n\n\n\n\n[1]: https://www.kaggle.com/cdeotte/how-to-compete-with-gpus-workshop\n\n", "902042": "Today's Accelerator Power Hour was a huge success. There were about 1000 viewers in the audience. I discussed many competition tips and tricks using TensorFlow GPU and Abhishek showed how to code with PyTorch TPU.\n\n If you missed the live stream, you can watch the recorded stream on YouTube here: https://youtu.be/DEuvGh4ZwaY", "900754": "I have also received a mail from kaggle about your conversation about the above. I will definitely follow your sharing, it will definitely help me for upcoming competitions and even for this one ^ ^", "906088": "Hi Chris. In the sharing regarding to your initial learning rate of 0, you explained that it is because pre-trained models have some intelligence and we do not want to a high initial learning rate to affect the network weights too much... but training through one epoch with LR of 0 actually does not do anything to the networks, is it a waste of resource? Should we start with very small learning rate instead?", "900618": "Unfortunately, I'll be at work (stacking potatoes). Will this be available in some other form for later consumption?", "903506": "Cool! Thanks for sharing! It's very informative! 👍 👍 ", "902966": "That was an amazing presentation @cdeotte , great job! you have amazing talking skill, I hope we can see more of that in the future!", "902537": "I enjoyed the talk! thank you Chris and Abhishek", "902370": "Thanks great talk! Attended live at 03:00 hours from Sydney and found it totally worth it! \n\nI don't understand the post processing bit, are there any resources that you could guide me to please? How did you end up using three different `mat` and using `dot` products and how does this maximise CV score please?", "902299": "Great talk! I especially liked your idea of applying transfer learning to humans :-) ! Nice joke!", "901443": "subscribed... looking forward to learn from best player", "906544": "Thanks @cdeotte for all the knowledge you are sharing! \nI read the top solutions of the IEEE-CIS Fraud Detection competition where you applied post processing to AUC. \nBoth competitions have something in common, we have:\n* Many observations of the same patient in this comp.\n* Many transactions of the same customer in the IEEE-CIS Fraud Detection comp. \n\nDo you think the idea of aggregating the predictions of the same patient as you did in the other comp. (the malignant ones) can somehow boost the score? I tried but it drastically lowered the score to 0.81 AUC :)", "905537": "very very good talk Chris, very funny, deep and entertaining.\nyou showed a learning rate scheduler to tame a pre-trained model ... does the same apply when you need to resume the training of a model that due to hardware limitations takes a long time during each training epoch?", "1379116": "For those guys who found Kaggle's GPU is not sufficient, I recommend an MLOps tool called AIbro. The company is giving a lot of free credit to use cloud GPU. You can use it to train AI models on any instance from AWS in one line. https://colab.research.google.com/drive/19sXZ4kbic681zqEsrl_CZfB5cegUwuIB#scrollTo=CSgozfXs572M", "967792": "Hi! I am new here. I have some python computations, and these take 1 Hour by CPU. Can you help me how to use my GPU for this? (these python computations are not ML model, just computing in big data) ", "905372": "", "903827": "", "902209": "", "900610": "thanks for reminding" } }