This is just 3rd Kaggle competition I have participated. But my learning has been tremendous. I completed data science courses from deeplearning.ai from coursera just few months ago and started on Kaggle competitions. While I have learnt all the basics from the courses, it is in Kaggle I have really learnt to apply these concepts.
\nHaving worked several years in IT industry, I can say that even as an employee in IT companies they don't give exposure to such real world problems so easily and so fast. But here we are exposed to real world problems along with tremendous community support, collaboration and sharing. This does not happen so easily in real world in industry.
\nIn each competition, I am sure there are several new things the machine learning community learns afresh and new - given the tremendous effort put by several teams. All this should not go waste just with this competition. So, I suggest Kaggle Staff to start a \"Post Learning Mega thread\" like 'Team Megathread', where community can share new things they have learnt by participating in this specific competition. This thread should be part of all Kaggle competitions.
\nI have learnt several things in this competition given that I am a new entrant to kaggle - so I can't list all. Specifically, I learnt how to apply Transformers for structured time series data - previously I thought it is relevant only where embedding layers are applicable. Along the process, I have learnt several others things about transformers and deep learning like using TPUs, applying strategies for distributed training of batches etc.
\nThe importance of memory management and its impact on training and model architecture, efficiency for inference, using separate notebooks for training and inference, creating datasets out of notebooks etc., and presence of libraries like Polar etc are all new things to me, learnt in this competition.
\nI hope community will voluntarily share their new learnings once Kaggle opens Megathread for the same!
\n(By the way, here is my transformer model - by the time, I brought it to some reasonable accuracy, it was just one week remaining to the competition deadline, still some mistakes could be there: https://www.kaggle.com/code/murugesann/nm-std-pred-transformer-9/notebook -
\nyesterday, I tried using single row per batch, the accuracy did not become very low :-)
This is just 3rd Kaggle competition I have participated. But my learning has been tremendous. I completed data science courses from deeplearning.ai from coursera just few months ago and started on Kaggle competitions. While I have learnt all the basics from the courses, it is in Kaggle I have really learnt to apply these concepts.
\nHaving worked several years in IT industry, I can say that even as an employee in IT companies they don't give exposure to such real world problems so easily and so fast. But here we are exposed to real world problems along with tremendous community support, collaboration and sharing. This does not happen so easily in real world in industry.
\nIn each competition, I am sure there are several new things the machine learning community learns afresh and new - given the tremendous effort put by several teams. All this should not go waste just with this competition. So, I suggest Kaggle Staff to start a \"Post Learning Mega thread\" like 'Team Megathread', where community can share new things they have learnt by participating in this specific competition. This thread should be part of all Kaggle competitions.
\nI have learnt several things in this competition given that I am a new entrant to kaggle - so I can't list all. Specifically, I learnt how to apply Transformers for structured time series data - previously I thought it is relevant only where embedding layers are applicable. Along the process, I have learnt several others things about transformers and deep learning like using TPUs, applying strategies for distributed training of batches etc.
\nThe importance of memory management and its impact on training and model architecture, efficiency for inference, using separate notebooks for training and inference, creating datasets out of notebooks etc., and presence of libraries like Polar etc are all new things to me, learnt in this competition.
\nI hope community will voluntarily share their new learnings once Kaggle opens Megathread for the same!
\n(By the way, here is my transformer model - by the time, I brought it to some reasonable accuracy, it was just one week remaining to the competition deadline, still some mistakes could be there: https://www.kaggle.com/code/murugesann/nm-std-pred-transformer-9/notebook -
\nyesterday, I tried using single row per batch, the accuracy did not become very low :-)