{
  "id": 424124,
  "title": "Finally a GM! A milestone.",
  "url": "/competitions/predict-student-performance-from-game-play/discussion/424124",
  "author_name": "Anthony Chiu",
  "post_date": "2023-07-12T17:02:05.474000",
  "votes": 37,
  "comment_count": 27,
  "views": 0,
  "content": "<p>First, thank the Kaggle team and The Learning Agency Lab for organizing this competition. Although this was not smooth, I genuinely appreciate all your contributions here.</p>\n<p>I created my account 7 years ago, and started my first serious competition 5 years ago. It has been a fun and hard journey to get these 5 gold medals. I learned a lot from all of you here. After getting into the GM tier, I plan to share more with this community. As the starting point let me share about my journey here. (It is not as exciting as those who got a GM within a year.)</p>\n<h2>How it started</h2>\n<p>I participated in a Robotic Soccer competition in my high school; in this competition, we had to program a robot based on different sensors. As a result, I thought I liked computer science. When I got into University, I was lost. It is because I found that 1) the University level robotic competition is all about memorizing the track and 2) A lot of CS algorithms are more about human intelligence.</p>\n<p>In my UG Year 3, I enrolled into a Fuzzy system &amp; NN course offered by our electronic and electricity department (not CS department). That class has less than 10 students and it was taught by an old professor who was researching in NN but switched to the control system. (Later, I know it resulted from \"AI winter\"). I was excited for the first time in that course in my university. The material discussed linear separability and using NN to do an XOR gate. I was thinking “oh you can build a digital system with NNs without human interventions!”.</p>\n<h2>First “Failed” Competition on Kaggle</h2>\n<p><a href=\"https://www.kaggle.com/competitions/msk-redefining-cancer-treatment\" target=\"_blank\">https://www.kaggle.com/competitions/msk-redefining-cancer-treatment</a></p>\n<p>In this competition, I got to the top in the first dataset using oversampling (<a href=\"https://imbalanced-learn.org/stable/over_sampling.html)\" target=\"_blank\">https://imbalanced-learn.org/stable/over_sampling.html)</a>. And it was a huge mistake because I applied it in the whole training set! </p>\n<p>Learning: </p>\n<p>Some feature engineering MUST be done within CV. For example, we only want to adjust the class distribution of the training data in a fold, but we need to keep the default class distribution of the validation data in that fold! Otherwise, the CV score is not representative to the unseen test data.</p>\n<h2>First Gold (2018)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/home-credit-default-risk\" target=\"_blank\">https://www.kaggle.com/competitions/home-credit-default-risk</a></p>\n<p>After realizing the importance of feature engineering and gradient-boosted tree models. This competition is the perfect moment to test my learning. This competition has the largest relational data I have ever seen on Kaggle. Endless feature engineering…</p>\n<p>Learning: </p>\n<p>Feature ranking and selection are also very important after having many features. This competition also tells me how to work with real-world data; all columns all tables are not masked. I still remember a crucial “feature”. It was the contradiction of loan durations. There are at least 3 ways to compute the loan durations, and the results can differ…</p>\n<h2>First Solo Gold (2019)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/elo-merchant-category-recommendation\" target=\"_blank\">https://www.kaggle.com/competitions/elo-merchant-category-recommendation</a></p>\n<p>I got this solo gold by shaking up 175 places from public to private. The target values in this competition are clustered into 2 parts; therefore, a few public notebooks proposed using a 2 stage model. I spotted an issue with the public notebook near the end of the competition: The linkage between the 1st and 2nd stage model is not cross-validated!  I modified the method and got a relatively robust result. </p>\n<p>Learning:</p>\n<p>Multi-stage model. Validation Setup! Trust your CV! (and fixing public notebook to get a better result)</p>\n<h2>Starting to see the limitation of Tabular models (2019)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/nfl-big-data-bowl-2020\" target=\"_blank\">https://www.kaggle.com/competitions/nfl-big-data-bowl-2020</a></p>\n<p>Tree model only solution has been doing good with tabular data. While in this competition, although my team got the gold at the end, we were beaten by all the creative NN models.</p>\n<p>Learning:</p>\n<p>Creative NN Layers as a feature extractor.  After this competition, I studied Pytorch (switching from Keras).</p>\n<h2>Feature Engineering with NNs (2020)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/stanford-covid-vaccine\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-covid-vaccine</a></p>\n<p>I applied graph, sequence, and conv layer to reach top-3 at the early game. I finally successfully model data with creative NN! I was so happy. I understand more about the idea of feature engineering with NN.</p>\n<p>Learning:</p>\n<p>In the end, our team was 11th. We “lost”, because we haven’t done pertaining (extra data) and pseudo-labeling.</p>\n<h2>First Prizes Zone (2023)</h2>\n<p><a href=\"http://kaggle.com/competitions/predict-student-performance-from-game-play\" target=\"_blank\">kaggle.com/competitions/predict-student-performance-from-game-play</a></p>\n<p>It is a mixed feeling. I am happy to have the 3rd here, but I feel like it is a relatively straightforward competition. What I can say is that I believe in what I have learned from this community, and it turns out to be great.</p>\n<h2>Moving Forwards</h2>\n<p>I will still participate in competitions occasionally, but my Kaggle time is getting fewer and fewer, especially since machine learning is not my day job. I will start focusing more on sharing and contributing to the community. </p>\n<p>As a first step, I implement the idea of “Null Importances” as a Python package here: <a href=\"https://github.com/kingychiu/target-permutation-importances\" target=\"_blank\">https://github.com/kingychiu/target-permutation-importances</a>. <br>\nHope this is helpful to some newcomers here.</p>\n<h2>Special Thanks</h2>\n<p><a href=\"https://www.kaggle.com/fatihozturk\" target=\"_blank\">@fatihozturk</a> <a href=\"https://www.kaggle.com/wimwim\" target=\"_blank\">@wimwim</a> Teamed with me in many competitions, exchanging countless ideas online.</p>\n<p>If you ask me to name someone out of my mind immediately…<br>\n<a href=\"https://www.kaggle.com/ogrellier\" target=\"_blank\">@ogrellier</a> <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> <a href=\"https://www.kaggle.com/narsil\" target=\"_blank\">@narsil</a> <a href=\"https://www.kaggle.com/tunguz\" target=\"_blank\">@tunguz</a> <br>\nUsually, I got insights from your comments/posts/works.</p>",
  "messages": [
    {
      "id": 2342291,
      "postDate": "2023-07-12T17:02:05.473Z",
      "content": "<p>First, thank the Kaggle team and The Learning Agency Lab for organizing this competition. Although this was not smooth, I genuinely appreciate all your contributions here.</p>\n<p>I created my account 7 years ago, and started my first serious competition 5 years ago. It has been a fun and hard journey to get these 5 gold medals. I learned a lot from all of you here. After getting into the GM tier, I plan to share more with this community. As the starting point let me share about my journey here. (It is not as exciting as those who got a GM within a year.)</p>\n<h2>How it started</h2>\n<p>I participated in a Robotic Soccer competition in my high school; in this competition, we had to program a robot based on different sensors. As a result, I thought I liked computer science. When I got into University, I was lost. It is because I found that 1) the University level robotic competition is all about memorizing the track and 2) A lot of CS algorithms are more about human intelligence.</p>\n<p>In my UG Year 3, I enrolled into a Fuzzy system &amp; NN course offered by our electronic and electricity department (not CS department). That class has less than 10 students and it was taught by an old professor who was researching in NN but switched to the control system. (Later, I know it resulted from \"AI winter\"). I was excited for the first time in that course in my university. The material discussed linear separability and using NN to do an XOR gate. I was thinking “oh you can build a digital system with NNs without human interventions!”.</p>\n<h2>First “Failed” Competition on Kaggle</h2>\n<p><a href=\"https://www.kaggle.com/competitions/msk-redefining-cancer-treatment\" target=\"_blank\">https://www.kaggle.com/competitions/msk-redefining-cancer-treatment</a></p>\n<p>In this competition, I got to the top in the first dataset using oversampling (<a href=\"https://imbalanced-learn.org/stable/over_sampling.html)\" target=\"_blank\">https://imbalanced-learn.org/stable/over_sampling.html)</a>. And it was a huge mistake because I applied it in the whole training set! </p>\n<p>Learning: </p>\n<p>Some feature engineering MUST be done within CV. For example, we only want to adjust the class distribution of the training data in a fold, but we need to keep the default class distribution of the validation data in that fold! Otherwise, the CV score is not representative to the unseen test data.</p>\n<h2>First Gold (2018)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/home-credit-default-risk\" target=\"_blank\">https://www.kaggle.com/competitions/home-credit-default-risk</a></p>\n<p>After realizing the importance of feature engineering and gradient-boosted tree models. This competition is the perfect moment to test my learning. This competition has the largest relational data I have ever seen on Kaggle. Endless feature engineering…</p>\n<p>Learning: </p>\n<p>Feature ranking and selection are also very important after having many features. This competition also tells me how to work with real-world data; all columns all tables are not masked. I still remember a crucial “feature”. It was the contradiction of loan durations. There are at least 3 ways to compute the loan durations, and the results can differ…</p>\n<h2>First Solo Gold (2019)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/elo-merchant-category-recommendation\" target=\"_blank\">https://www.kaggle.com/competitions/elo-merchant-category-recommendation</a></p>\n<p>I got this solo gold by shaking up 175 places from public to private. The target values in this competition are clustered into 2 parts; therefore, a few public notebooks proposed using a 2 stage model. I spotted an issue with the public notebook near the end of the competition: The linkage between the 1st and 2nd stage model is not cross-validated!  I modified the method and got a relatively robust result. </p>\n<p>Learning:</p>\n<p>Multi-stage model. Validation Setup! Trust your CV! (and fixing public notebook to get a better result)</p>\n<h2>Starting to see the limitation of Tabular models (2019)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/nfl-big-data-bowl-2020\" target=\"_blank\">https://www.kaggle.com/competitions/nfl-big-data-bowl-2020</a></p>\n<p>Tree model only solution has been doing good with tabular data. While in this competition, although my team got the gold at the end, we were beaten by all the creative NN models.</p>\n<p>Learning:</p>\n<p>Creative NN Layers as a feature extractor.  After this competition, I studied Pytorch (switching from Keras).</p>\n<h2>Feature Engineering with NNs (2020)</h2>\n<p><a href=\"https://www.kaggle.com/competitions/stanford-covid-vaccine\" target=\"_blank\">https://www.kaggle.com/competitions/stanford-covid-vaccine</a></p>\n<p>I applied graph, sequence, and conv layer to reach top-3 at the early game. I finally successfully model data with creative NN! I was so happy. I understand more about the idea of feature engineering with NN.</p>\n<p>Learning:</p>\n<p>In the end, our team was 11th. We “lost”, because we haven’t done pertaining (extra data) and pseudo-labeling.</p>\n<h2>First Prizes Zone (2023)</h2>\n<p><a href=\"http://kaggle.com/competitions/predict-student-performance-from-game-play\" target=\"_blank\">kaggle.com/competitions/predict-student-performance-from-game-play</a></p>\n<p>It is a mixed feeling. I am happy to have the 3rd here, but I feel like it is a relatively straightforward competition. What I can say is that I believe in what I have learned from this community, and it turns out to be great.</p>\n<h2>Moving Forwards</h2>\n<p>I will still participate in competitions occasionally, but my Kaggle time is getting fewer and fewer, especially since machine learning is not my day job. I will start focusing more on sharing and contributing to the community. </p>\n<p>As a first step, I implement the idea of “Null Importances” as a Python package here: <a href=\"https://github.com/kingychiu/target-permutation-importances\" target=\"_blank\">https://github.com/kingychiu/target-permutation-importances</a>. <br>\nHope this is helpful to some newcomers here.</p>\n<h2>Special Thanks</h2>\n<p><a href=\"https://www.kaggle.com/fatihozturk\" target=\"_blank\">@fatihozturk</a> <a href=\"https://www.kaggle.com/wimwim\" target=\"_blank\">@wimwim</a> Teamed with me in many competitions, exchanging countless ideas online.</p>\n<p>If you ask me to name someone out of my mind immediately…<br>\n<a href=\"https://www.kaggle.com/ogrellier\" target=\"_blank\">@ogrellier</a> <a href=\"https://www.kaggle.com/cdeotte\" target=\"_blank\">@cdeotte</a> <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> <a href=\"https://www.kaggle.com/narsil\" target=\"_blank\">@narsil</a> <a href=\"https://www.kaggle.com/tunguz\" target=\"_blank\">@tunguz</a> <br>\nUsually, I got insights from your comments/posts/works.</p>",
      "rawMarkdown": "First, thank the Kaggle team and The Learning Agency Lab for organizing this competition. Although this was not smooth, I genuinely appreciate all your contributions here.\n\nI created my account 7 years ago, and started my first serious competition 5 years ago. It has been a fun and hard journey to get these 5 gold medals. I learned a lot from all of you here. After getting into the GM tier, I plan to share more with this community. As the starting point let me share about my journey here. (It is not as exciting as those who got a GM within a year.)\n\n## How it started\n\nI participated in a Robotic Soccer competition in my high school; in this competition, we had to program a robot based on different sensors. As a result, I thought I liked computer science. When I got into University, I was lost. It is because I found that 1) the University level robotic competition is all about memorizing the track and 2) A lot of CS algorithms are more about human intelligence.\n\nIn my UG Year 3, I enrolled into a Fuzzy system & NN course offered by our electronic and electricity department (not CS department). That class has less than 10 students and it was taught by an old professor who was researching in NN but switched to the control system. (Later, I know it resulted from \"AI winter\"). I was excited for the first time in that course in my university. The material discussed linear separability and using NN to do an XOR gate. I was thinking “oh you can build a digital system with NNs without human interventions!”.\n\n## First “Failed” Competition on Kaggle\n\nhttps://www.kaggle.com/competitions/msk-redefining-cancer-treatment\n\nIn this competition, I got to the top in the first dataset using oversampling (https://imbalanced-learn.org/stable/over_sampling.html). And it was a huge mistake because I applied it in the whole training set! \n\nLearning: \n\nSome feature engineering MUST be done within CV. For example, we only want to adjust the class distribution of the training data in a fold, but we need to keep the default class distribution of the validation data in that fold! Otherwise, the CV score is not representative to the unseen test data.\n\n## First Gold (2018)\n\nhttps://www.kaggle.com/competitions/home-credit-default-risk\n\nAfter realizing the importance of feature engineering and gradient-boosted tree models. This competition is the perfect moment to test my learning. This competition has the largest relational data I have ever seen on Kaggle. Endless feature engineering…\n\nLearning: \n\nFeature ranking and selection are also very important after having many features. This competition also tells me how to work with real-world data; all columns all tables are not masked. I still remember a crucial “feature”. It was the contradiction of loan durations. There are at least 3 ways to compute the loan durations, and the results can differ…\n\n## First Solo Gold (2019)\n\nhttps://www.kaggle.com/competitions/elo-merchant-category-recommendation\n\nI got this solo gold by shaking up 175 places from public to private. The target values in this competition are clustered into 2 parts; therefore, a few public notebooks proposed using a 2 stage model. I spotted an issue with the public notebook near the end of the competition: The linkage between the 1st and 2nd stage model is not cross-validated!  I modified the method and got a relatively robust result. \n\nLearning:\n\nMulti-stage model. Validation Setup! Trust your CV! (and fixing public notebook to get a better result)\n\n## Starting to see the limitation of Tabular models (2019)\n\nhttps://www.kaggle.com/competitions/nfl-big-data-bowl-2020\n\nTree model only solution has been doing good with tabular data. While in this competition, although my team got the gold at the end, we were beaten by all the creative NN models.\n\nLearning:\n\nCreative NN Layers as a feature extractor.  After this competition, I studied Pytorch (switching from Keras).\n\n## Feature Engineering with NNs (2020)\n\nhttps://www.kaggle.com/competitions/stanford-covid-vaccine\n\nI applied graph, sequence, and conv layer to reach top-3 at the early game. I finally successfully model data with creative NN! I was so happy. I understand more about the idea of feature engineering with NN.\n\nLearning:\n\nIn the end, our team was 11th. We “lost”, because we haven’t done pertaining (extra data) and pseudo-labeling.\n\n## First Prizes Zone (2023)\n\n[kaggle.com/competitions/predict-student-performance-from-game-play](http://kaggle.com/competitions/predict-student-performance-from-game-play)\n\nIt is a mixed feeling. I am happy to have the 3rd here, but I feel like it is a relatively straightforward competition. What I can say is that I believe in what I have learned from this community, and it turns out to be great.\n\n## Moving Forwards\n\nI will still participate in competitions occasionally, but my Kaggle time is getting fewer and fewer, especially since machine learning is not my day job. I will start focusing more on sharing and contributing to the community. \n\nAs a first step, I implement the idea of “Null Importances” as a Python package here: https://github.com/kingychiu/target-permutation-importances. \nHope this is helpful to some newcomers here.\n\n## Special Thanks\n@fatihozturk @wimwim Teamed with me in many competitions, exchanging countless ideas online.\n\nIf you ask me to name someone out of my mind immediately...\n@ogrellier @cdeotte @cpmpml @narsil @tunguz \nUsually, I got insights from your comments/posts/works.",
      "votes": 37
    },
    {
      "id": 2345546,
      "postDate": "2023-07-15T13:30:22.597Z",
      "content": "<p>Congratulations</p>",
      "rawMarkdown": "\nCongratulations",
      "votes": 3
    },
    {
      "id": 2342607,
      "postDate": "2023-07-13T02:49:49.743Z",
      "content": "<p>Congratulations! This is not just <strong>a</strong> milestone, but definitely a <strong>great</strong> one!</p>\n<p>Also, thank you for sharing your experience. I myself just got my first gold too, and when reading this post I feel like I'm starting out like you did several years ago. Hopefully one day I'll reach where you are now.</p>\n<p>I have already learned a few things just by reading about your journey. There is a competition where I'm doing oversampling and then CV, and I keep wondering how to prevent leakage - you answered just this in your post! Look forward to seeing more of your sharing on Kaggle!</p>\n<p>In addition to that, I have some questions about a few things mentioned in your post and hope you could help me answer:</p>\n<ol>\n<li>How do you often do feature ranking and selection? I usually select an arbitrary top <code>X</code> of the model's default <code>feature_importance</code>, but I feel like there is much more to that (what <code>X</code> should be, other importance metrics etc.). If you have any doc/resources you can point me to that'd be awesome too!</li>\n<li>You mentioned that in real-world data <em>all columns all tables are not masked</em> - what is masking for in this case?</li>\n</ol>\n<p>Thanks and congratulations again!</p>",
      "rawMarkdown": "Congratulations! This is not just **a** milestone, but definitely a **great** one!\n\nAlso, thank you for sharing your experience. I myself just got my first gold too, and when reading this post I feel like I'm starting out like you did several years ago. Hopefully one day I'll reach where you are now.\n\nI have already learned a few things just by reading about your journey. There is a competition where I'm doing oversampling and then CV, and I keep wondering how to prevent leakage - you answered just this in your post! Look forward to seeing more of your sharing on Kaggle!\n\nIn addition to that, I have some questions about a few things mentioned in your post and hope you could help me answer:\n1. How do you often do feature ranking and selection? I usually select an arbitrary top `X` of the model's default `feature_importance`, but I feel like there is much more to that (what `X` should be, other importance metrics etc.). If you have any doc/resources you can point me to that'd be awesome too!\n2. You mentioned that in real-world data *all columns all tables are not masked* - what is masking for in this case?\n\nThanks and congratulations again!",
      "votes": 4,
      "replies": [
        {
          "id": 2342616,
          "postDate": "2023-07-13T03:15:10.713Z",
          "content": "<p>Hi, Thanks for your question!</p>\n<blockquote>\n  <p>I usually select an arbitrary top X of the model's default feature_importance</p>\n</blockquote>\n<p>In competition, I usually use Null Importances, I created a Python package <a href=\"https://github.com/kingychiu/target-permutation-importances\" target=\"_blank\">here</a> . You can look at the examples.</p>\n<blockquote>\n  <p>what X should be, other importance metrics etc</p>\n</blockquote>\n<p>Consider it is the same as hyperparameter search, you can use a for loop like grid search or make the top X be part of your optuna search:</p>\n<pre><code> ():\n    \n    best_k_features = trial.suggest_int(, , (ranked_features))\n    param = {\n        **base_rf_params,\n        : trial.suggest_float(, , ),\n        : trial.suggest_int(, , ),\n    }\n</code></pre>\n<blockquote>\n  <p>what is masking for in this case?</p>\n</blockquote>\n<p>There are some tabular competitions on Kaggle without giving you column definitions like<br>\n<a href=\"https://www.kaggle.com/competitions/amex-default-prediction/data\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/data</a><br>\n<a href=\"https://www.kaggle.com/competitions/santander-customer-transaction-prediction\" target=\"_blank\">https://www.kaggle.com/competitions/santander-customer-transaction-prediction</a> </p>",
          "rawMarkdown": "Hi, Thanks for your question!\n\n> I usually select an arbitrary top X of the model's default feature_importance\n\nIn competition, I usually use Null Importances, I created a Python package [here](https://github.com/kingychiu/target-permutation-importances) . You can look at the examples.\n\n> what X should be, other importance metrics etc\n\nConsider it is the same as hyperparameter search, you can use a for loop like grid search or make the top X be part of your optuna search:\n\n```\ndef objective(trial):\n    # Number of features\n    best_k_features = trial.suggest_int(\"best_k_features\", 1, len(ranked_features))\n    param = {\n        **base_rf_params,\n        'min_weight_fraction_leaf': trial.suggest_float('min_weight_fraction_leaf', 0, 0.5),\n        'n_estimators': trial.suggest_int('n_estimators', 64, 2048),\n    }\n```\n\n> what is masking for in this case?\n\nThere are some tabular competitions on Kaggle without giving you column definitions like\nhttps://www.kaggle.com/competitions/amex-default-prediction/data\nhttps://www.kaggle.com/competitions/santander-customer-transaction-prediction ",
          "votes": 3,
          "replies": [
            {
              "id": 2342667,
              "postDate": "2023-07-13T04:13:36.833Z",
              "content": "<p>Thanks for the answers!</p>\n<blockquote>\n  <p>In competition, I usually use Null Importances, I created a Python package here</p>\n</blockquote>\n<p>I read through the package's introduction and the linked notebook, and it seems like what I'm looking for! Definitely will try in my next test.</p>\n<blockquote>\n  <p>There are some tabular competitions on Kaggle without giving you column definitions</p>\n</blockquote>\n<p>Ah I see. Do you have any feature-engineering tips in those cases? What I often do is creating additional combinations (<code>X_multiply_Y</code>, <code>X_divide_Y</code> with <code>X</code> and <code>Y</code> being any two different features), and finding top X most important features. However I don't think it's the most efficient way as it increases the search space by a lot.</p>",
              "rawMarkdown": "Thanks for the answers!\n\n>In competition, I usually use Null Importances, I created a Python package here\n\nI read through the package's introduction and the linked notebook, and it seems like what I'm looking for! Definitely will try in my next test.\n\n>There are some tabular competitions on Kaggle without giving you column definitions\n\nAh I see. Do you have any feature-engineering tips in those cases? What I often do is creating additional combinations (`X_multiply_Y`, `X_divide_Y` with `X` and `Y` being any two different features), and finding top X most important features. However I don't think it's the most efficient way as it increases the search space by a lot.",
              "votes": 2
            },
            {
              "id": 2342944,
              "postDate": "2023-07-13T09:25:22.877Z",
              "content": "<p>Sadly, I am not good at anonymized data; you can take a look at the top solutions in those competitions.</p>",
              "rawMarkdown": "Sadly, I am not good at anonymized data; you can take a look at the top solutions in those competitions.",
              "votes": 1
            },
            {
              "id": 2343104,
              "postDate": "2023-07-13T12:02:14.100Z",
              "content": "<p>Will do. Thanks for all the tips! Look forward to you sharing your knowledge again!</p>",
              "rawMarkdown": "Will do. Thanks for all the tips! Look forward to you sharing your knowledge again!"
            }
          ]
        }
      ]
    },
    {
      "id": 2347601,
      "postDate": "2023-07-17T04:54:50.510Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> !<br>\nWe look forward to more great work ahead!</p>",
      "rawMarkdown": "Congratulations @kingychiu !\nWe look forward to more great work ahead!",
      "votes": 1
    },
    {
      "id": 2344246,
      "postDate": "2023-07-14T11:28:38.407Z",
      "content": "<p><a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> seems lot of hardwork and consistency. Congratulations 🎉</p>",
      "rawMarkdown": "@kingychiu seems lot of hardwork and consistency. Congratulations 🎉",
      "votes": 1
    },
    {
      "id": 2343907,
      "postDate": "2023-07-14T05:09:01.773Z",
      "content": "<p>Congratulations. Also. thanks for sharing your remarkable journey for achieving GM norm.<br>\nML competitions at Kaggle are good learning place. The competitions are seeing exciting finish<br>\nwith toppers score separated by minute fractions. </p>",
      "rawMarkdown": "Congratulations. Also. thanks for sharing your remarkable journey for achieving GM norm.\nML competitions at Kaggle are good learning place. The competitions are seeing exciting finish\nwith toppers score separated by minute fractions. \n",
      "votes": 1
    },
    {
      "id": 2343017,
      "postDate": "2023-07-13T10:49:10.773Z",
      "content": "<p>Kudos and congrats on becoming grandmaster <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> </p>",
      "rawMarkdown": "Kudos and congrats on becoming grandmaster @kingychiu ",
      "votes": 1
    },
    {
      "id": 2342978,
      "postDate": "2023-07-13T10:03:37.663Z",
      "content": "<p>Congratulations on becoming GM !! Well done.</p>",
      "rawMarkdown": "Congratulations on becoming GM !! Well done.",
      "votes": 1
    },
    {
      "id": 2342302,
      "postDate": "2023-07-12T17:11:46.207Z",
      "content": "<p>Kudos on becoming competitions GM <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> !</p>",
      "rawMarkdown": "Kudos on becoming competitions GM @kingychiu !",
      "votes": 1
    },
    {
      "id": 2348057,
      "postDate": "2023-07-17T11:18:34.580Z",
      "content": "<p>Big congrats on your GM tier! 🎉</p>\n<p>Oh man, I'm so relieved and happy as I got it by myself… Thank you for mentioning me at the beginning of your \"Special Thanks\", it indeed feels special :) </p>\n<p>And I want to thank you so much as well! You made it so easy to team up for me (who is a relatively introverted person actually) Though we have been living in two very distant parts of the world, I always felt very comfortable and close talking to you and exchanging our ideas during all these competitions.</p>\n<p>Just remembered the Home Credit competition… It was our very first CV victory and the time and the excitement we had when we were waiting for the private LB results… :D</p>\n<p>Anyways, I always know you first, as a good friend and second as a very hardworking person. Well-deserved title!</p>\n<p>Wishing you all the best, and looking forward to seeing more wins from you!</p>",
      "rawMarkdown": "Big congrats on your GM tier! 🎉\n\nOh man, I'm so relieved and happy as I got it by myself... Thank you for mentioning me at the beginning of your \"Special Thanks\", it indeed feels special :) \n\nAnd I want to thank you so much as well! You made it so easy to team up for me (who is a relatively introverted person actually) Though we have been living in two very distant parts of the world, I always felt very comfortable and close talking to you and exchanging our ideas during all these competitions.\n\nJust remembered the Home Credit competition... It was our very first CV victory and the time and the excitement we had when we were waiting for the private LB results... :D\n\nAnyways, I always know you first, as a good friend and second as a very hardworking person. Well-deserved title!\n\nWishing you all the best, and looking forward to seeing more wins from you!",
      "votes": 2
    },
    {
      "id": 2346185,
      "postDate": "2023-07-16T05:37:37.923Z",
      "content": "<p>Achieving a solo gold in the Elo Merchant Category Recommendation competition by improving the 1st and 2nd stage model linkage through cross-validation demonstrates the importance of validation setup and trusting your cross-validation results. Great job on obtaining a robust result!</p>",
      "rawMarkdown": "Achieving a solo gold in the Elo Merchant Category Recommendation competition by improving the 1st and 2nd stage model linkage through cross-validation demonstrates the importance of validation setup and trusting your cross-validation results. Great job on obtaining a robust result!",
      "votes": 2
    },
    {
      "id": 2345551,
      "postDate": "2023-07-15T13:34:56.157Z",
      "content": "<p>There's no success without the seeming failures. Congratulations on this milestone. ✌️</p>",
      "rawMarkdown": "There's no success without the seeming failures. Congratulations on this milestone. ✌️",
      "votes": 2
    },
    {
      "id": 2344954,
      "postDate": "2023-07-15T00:12:49.933Z",
      "content": "<p>Congratulations! Keep it up!</p>",
      "rawMarkdown": "Congratulations! Keep it up!",
      "votes": 2
    },
    {
      "id": 2344810,
      "postDate": "2023-07-14T19:26:15.353Z",
      "content": "<p>Congratulations upon achieving the prestigious title. 👍</p>",
      "rawMarkdown": "Congratulations upon achieving the prestigious title. 👍",
      "votes": 2
    },
    {
      "id": 2344718,
      "postDate": "2023-07-14T17:55:30.343Z",
      "content": "<p>Congratulations man, It's a great achievement 🥳 </p>",
      "rawMarkdown": "Congratulations man, It's a great achievement 🥳 ",
      "votes": 2
    },
    {
      "id": 2344127,
      "postDate": "2023-07-14T09:15:56.507Z",
      "content": "<p>Congratulations on becoming GM!  Keep up the great work! <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a></p>",
      "rawMarkdown": "Congratulations on becoming GM!  Keep up the great work! @kingychiu",
      "votes": 2
    },
    {
      "id": 2343663,
      "postDate": "2023-07-13T21:46:43.633Z",
      "content": "<p>Congratulations! </p>",
      "rawMarkdown": "Congratulations! ",
      "votes": 2
    },
    {
      "id": 2343055,
      "postDate": "2023-07-13T11:20:40.743Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> for becoming the <strong>Kaggle Competitions Grandmaster</strong>.🎉<br>\nCongratulations on your incredible journey!!</p>",
      "rawMarkdown": "Congratulations @kingychiu for becoming the **Kaggle Competitions Grandmaster**.🎉\nCongratulations on your incredible journey!!",
      "votes": 2
    },
    {
      "id": 2342621,
      "postDate": "2023-07-13T03:25:37.770Z",
      "content": "<p>Congratulations on becoming GM !!</p>",
      "rawMarkdown": "Congratulations on becoming GM !!",
      "votes": 2
    },
    {
      "id": 2342293,
      "postDate": "2023-07-12T17:06:09.900Z",
      "content": "<p>Congratulations! And thank you for citing me.</p>",
      "rawMarkdown": "Congratulations! And thank you for citing me.",
      "votes": 2
    },
    {
      "id": 2342328,
      "postDate": "2023-07-12T17:33:01.780Z",
      "content": "<p>Hearty congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> <br>\nThis is a remarkable achievement!! Keep up the great work!</p>",
      "rawMarkdown": "Hearty congratulations @kingychiu \nThis is a remarkable achievement!! Keep up the great work!"
    },
    {
      "id": 2348866,
      "postDate": "2023-07-18T04:05:48.397Z",
      "content": "<p>Well done and congratulations! <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> </p>",
      "rawMarkdown": "Well done and congratulations! @kingychiu "
    },
    {
      "id": 2359136,
      "postDate": "2023-07-26T02:43:31.103Z",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> ! <br>\nAfter reading your GM journey, I was astonished, this is not only a milestone but indeed a remarkable one! I'm eager to learn from you!</p>",
      "rawMarkdown": "Congratulations @kingychiu ! \nAfter reading your GM journey, I was astonished, this is not only a milestone but indeed a remarkable one! I'm eager to learn from you!\n",
      "isDeleted": true
    },
    {
      "id": 2348642,
      "postDate": "2023-07-17T19:49:23.710Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2345546,
      "author_name": "Yihan Xu",
      "author_url": "",
      "post_date": "2023-07-15T13:30:22.597000",
      "content": "<p>Congratulations</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2342607,
      "author_name": "Hoang Nguyen",
      "author_url": "",
      "post_date": "2023-07-13T02:49:49.743000",
      "content": "<p>Congratulations! This is not just <strong>a</strong> milestone, but definitely a <strong>great</strong> one!</p>\n<p>Also, thank you for sharing your experience. I myself just got my first gold too, and when reading this post I feel like I'm starting out like you did several years ago. Hopefully one day I'll reach where you are now.</p>\n<p>I have already learned a few things just by reading about your journey. There is a competition where I'm doing oversampling and then CV, and I keep wondering how to prevent leakage - you answered just this in your post! Look forward to seeing more of your sharing on Kaggle!</p>\n<p>In addition to that, I have some questions about a few things mentioned in your post and hope you could help me answer:</p>\n<ol>\n<li>How do you often do feature ranking and selection? I usually select an arbitrary top <code>X</code> of the model's default <code>feature_importance</code>, but I feel like there is much more to that (what <code>X</code> should be, other importance metrics etc.). If you have any doc/resources you can point me to that'd be awesome too!</li>\n<li>You mentioned that in real-world data <em>all columns all tables are not masked</em> - what is masking for in this case?</li>\n</ol>\n<p>Thanks and congratulations again!</p>",
      "votes": 4,
      "replies": [
        {
          "id": 2342616,
          "author_name": "Anthony Chiu",
          "author_url": "",
          "post_date": "2023-07-13T03:15:10.713000",
          "content": "<p>Hi, Thanks for your question!</p>\n<blockquote>\n  <p>I usually select an arbitrary top X of the model's default feature_importance</p>\n</blockquote>\n<p>In competition, I usually use Null Importances, I created a Python package <a href=\"https://github.com/kingychiu/target-permutation-importances\" target=\"_blank\">here</a> . You can look at the examples.</p>\n<blockquote>\n  <p>what X should be, other importance metrics etc</p>\n</blockquote>\n<p>Consider it is the same as hyperparameter search, you can use a for loop like grid search or make the top X be part of your optuna search:</p>\n<pre><code> ():\n    \n    best_k_features = trial.suggest_int(, , (ranked_features))\n    param = {\n        **base_rf_params,\n        : trial.suggest_float(, , ),\n        : trial.suggest_int(, , ),\n    }\n</code></pre>\n<blockquote>\n  <p>what is masking for in this case?</p>\n</blockquote>\n<p>There are some tabular competitions on Kaggle without giving you column definitions like<br>\n<a href=\"https://www.kaggle.com/competitions/amex-default-prediction/data\" target=\"_blank\">https://www.kaggle.com/competitions/amex-default-prediction/data</a><br>\n<a href=\"https://www.kaggle.com/competitions/santander-customer-transaction-prediction\" target=\"_blank\">https://www.kaggle.com/competitions/santander-customer-transaction-prediction</a> </p>",
          "votes": 3,
          "replies": [
            {
              "id": 2342667,
              "author_name": "Hoang Nguyen",
              "author_url": "",
              "post_date": "2023-07-13T04:13:36.833000",
              "content": "<p>Thanks for the answers!</p>\n<blockquote>\n  <p>In competition, I usually use Null Importances, I created a Python package here</p>\n</blockquote>\n<p>I read through the package's introduction and the linked notebook, and it seems like what I'm looking for! Definitely will try in my next test.</p>\n<blockquote>\n  <p>There are some tabular competitions on Kaggle without giving you column definitions</p>\n</blockquote>\n<p>Ah I see. Do you have any feature-engineering tips in those cases? What I often do is creating additional combinations (<code>X_multiply_Y</code>, <code>X_divide_Y</code> with <code>X</code> and <code>Y</code> being any two different features), and finding top X most important features. However I don't think it's the most efficient way as it increases the search space by a lot.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2342944,
              "author_name": "Anthony Chiu",
              "author_url": "",
              "post_date": "2023-07-13T09:25:22.877000",
              "content": "<p>Sadly, I am not good at anonymized data; you can take a look at the top solutions in those competitions.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2343104,
              "author_name": "Hoang Nguyen",
              "author_url": "",
              "post_date": "2023-07-13T12:02:14.100000",
              "content": "<p>Will do. Thanks for all the tips! Look forward to you sharing your knowledge again!</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2347601,
      "author_name": "Arvind (Yetirajan) Narayanan Iyengar",
      "author_url": "",
      "post_date": "2023-07-17T04:54:50.510000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> !<br>\nWe look forward to more great work ahead!</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2344246,
      "author_name": "Pranav Jadhav",
      "author_url": "",
      "post_date": "2023-07-14T11:28:38.407000",
      "content": "<p><a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> seems lot of hardwork and consistency. Congratulations 🎉</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2343907,
      "author_name": "C R Suthikshn Kumar",
      "author_url": "",
      "post_date": "2023-07-14T05:09:01.773000",
      "content": "<p>Congratulations. Also. thanks for sharing your remarkable journey for achieving GM norm.<br>\nML competitions at Kaggle are good learning place. The competitions are seeing exciting finish<br>\nwith toppers score separated by minute fractions. </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2343017,
      "author_name": "Ram Devi",
      "author_url": "",
      "post_date": "2023-07-13T10:49:10.773000",
      "content": "<p>Kudos and congrats on becoming grandmaster <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2342978,
      "author_name": "MST019",
      "author_url": "",
      "post_date": "2023-07-13T10:03:37.663000",
      "content": "<p>Congratulations on becoming GM !! Well done.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2342302,
      "author_name": "Suraj",
      "author_url": "",
      "post_date": "2023-07-12T17:11:46.207000",
      "content": "<p>Kudos on becoming competitions GM <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> !</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2348057,
      "author_name": "Fatih Öztürk",
      "author_url": "",
      "post_date": "2023-07-17T11:18:34.580000",
      "content": "<p>Big congrats on your GM tier! 🎉</p>\n<p>Oh man, I'm so relieved and happy as I got it by myself… Thank you for mentioning me at the beginning of your \"Special Thanks\", it indeed feels special :) </p>\n<p>And I want to thank you so much as well! You made it so easy to team up for me (who is a relatively introverted person actually) Though we have been living in two very distant parts of the world, I always felt very comfortable and close talking to you and exchanging our ideas during all these competitions.</p>\n<p>Just remembered the Home Credit competition… It was our very first CV victory and the time and the excitement we had when we were waiting for the private LB results… :D</p>\n<p>Anyways, I always know you first, as a good friend and second as a very hardworking person. Well-deserved title!</p>\n<p>Wishing you all the best, and looking forward to seeing more wins from you!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2346185,
      "author_name": "Sajjad Amjad",
      "author_url": "",
      "post_date": "2023-07-16T05:37:37.923000",
      "content": "<p>Achieving a solo gold in the Elo Merchant Category Recommendation competition by improving the 1st and 2nd stage model linkage through cross-validation demonstrates the importance of validation setup and trusting your cross-validation results. Great job on obtaining a robust result!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2345551,
      "author_name": "Chibuzo",
      "author_url": "",
      "post_date": "2023-07-15T13:34:56.157000",
      "content": "<p>There's no success without the seeming failures. Congratulations on this milestone. ✌️</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2344954,
      "author_name": "Vinh Nguyen",
      "author_url": "",
      "post_date": "2023-07-15T00:12:49.933000",
      "content": "<p>Congratulations! Keep it up!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2344810,
      "author_name": "Tariq Mahmood",
      "author_url": "",
      "post_date": "2023-07-14T19:26:15.353000",
      "content": "<p>Congratulations upon achieving the prestigious title. 👍</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2344718,
      "author_name": "Muhammad Usman",
      "author_url": "",
      "post_date": "2023-07-14T17:55:30.343000",
      "content": "<p>Congratulations man, It's a great achievement 🥳 </p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2344127,
      "author_name": "Eugeniy Osetrov",
      "author_url": "",
      "post_date": "2023-07-14T09:15:56.507000",
      "content": "<p>Congratulations on becoming GM!  Keep up the great work! <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2343663,
      "author_name": "Matheus Collacique",
      "author_url": "",
      "post_date": "2023-07-13T21:46:43.633000",
      "content": "<p>Congratulations! </p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2343055,
      "author_name": "Priyanshu Chaudhary",
      "author_url": "",
      "post_date": "2023-07-13T11:20:40.743000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> for becoming the <strong>Kaggle Competitions Grandmaster</strong>.🎉<br>\nCongratulations on your incredible journey!!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2342621,
      "author_name": "Mary Brown",
      "author_url": "",
      "post_date": "2023-07-13T03:25:37.770000",
      "content": "<p>Congratulations on becoming GM !!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2342293,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2023-07-12T17:06:09.900000",
      "content": "<p>Congratulations! And thank you for citing me.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 2342328,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2023-07-12T17:33:01.780000",
      "content": "<p>Hearty congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> <br>\nThis is a remarkable achievement!! Keep up the great work!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2348866,
      "author_name": "Kalilur Rahman",
      "author_url": "",
      "post_date": "2023-07-18T04:05:48.397000",
      "content": "<p>Well done and congratulations! <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2359136,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-07-26T02:43:31.103000",
      "content": "<p>Congratulations <a href=\"https://www.kaggle.com/kingychiu\" target=\"_blank\">@kingychiu</a> ! <br>\nAfter reading your GM journey, I was astonished, this is not only a milestone but indeed a remarkable one! I'm eager to learn from you!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2348642,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-07-17T19:49:23.710000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2342291": "First, thank the Kaggle team and The Learning Agency Lab for organizing this competition. Although this was not smooth, I genuinely appreciate all your contributions here.\n\nI created my account 7 years ago, and started my first serious competition 5 years ago. It has been a fun and hard journey to get these 5 gold medals. I learned a lot from all of you here. After getting into the GM tier, I plan to share more with this community. As the starting point let me share about my journey here. (It is not as exciting as those who got a GM within a year.)\n\n## How it started\n\nI participated in a Robotic Soccer competition in my high school; in this competition, we had to program a robot based on different sensors. As a result, I thought I liked computer science. When I got into University, I was lost. It is because I found that 1) the University level robotic competition is all about memorizing the track and 2) A lot of CS algorithms are more about human intelligence.\n\nIn my UG Year 3, I enrolled into a Fuzzy system & NN course offered by our electronic and electricity department (not CS department). That class has less than 10 students and it was taught by an old professor who was researching in NN but switched to the control system. (Later, I know it resulted from \"AI winter\"). I was excited for the first time in that course in my university. The material discussed linear separability and using NN to do an XOR gate. I was thinking “oh you can build a digital system with NNs without human interventions!”.\n\n## First “Failed” Competition on Kaggle\n\nhttps://www.kaggle.com/competitions/msk-redefining-cancer-treatment\n\nIn this competition, I got to the top in the first dataset using oversampling (https://imbalanced-learn.org/stable/over_sampling.html). And it was a huge mistake because I applied it in the whole training set! \n\nLearning: \n\nSome feature engineering MUST be done within CV. For example, we only want to adjust the class distribution of the training data in a fold, but we need to keep the default class distribution of the validation data in that fold! Otherwise, the CV score is not representative to the unseen test data.\n\n## First Gold (2018)\n\nhttps://www.kaggle.com/competitions/home-credit-default-risk\n\nAfter realizing the importance of feature engineering and gradient-boosted tree models. This competition is the perfect moment to test my learning. This competition has the largest relational data I have ever seen on Kaggle. Endless feature engineering…\n\nLearning: \n\nFeature ranking and selection are also very important after having many features. This competition also tells me how to work with real-world data; all columns all tables are not masked. I still remember a crucial “feature”. It was the contradiction of loan durations. There are at least 3 ways to compute the loan durations, and the results can differ…\n\n## First Solo Gold (2019)\n\nhttps://www.kaggle.com/competitions/elo-merchant-category-recommendation\n\nI got this solo gold by shaking up 175 places from public to private. The target values in this competition are clustered into 2 parts; therefore, a few public notebooks proposed using a 2 stage model. I spotted an issue with the public notebook near the end of the competition: The linkage between the 1st and 2nd stage model is not cross-validated!  I modified the method and got a relatively robust result. \n\nLearning:\n\nMulti-stage model. Validation Setup! Trust your CV! (and fixing public notebook to get a better result)\n\n## Starting to see the limitation of Tabular models (2019)\n\nhttps://www.kaggle.com/competitions/nfl-big-data-bowl-2020\n\nTree model only solution has been doing good with tabular data. While in this competition, although my team got the gold at the end, we were beaten by all the creative NN models.\n\nLearning:\n\nCreative NN Layers as a feature extractor.  After this competition, I studied Pytorch (switching from Keras).\n\n## Feature Engineering with NNs (2020)\n\nhttps://www.kaggle.com/competitions/stanford-covid-vaccine\n\nI applied graph, sequence, and conv layer to reach top-3 at the early game. I finally successfully model data with creative NN! I was so happy. I understand more about the idea of feature engineering with NN.\n\nLearning:\n\nIn the end, our team was 11th. We “lost”, because we haven’t done pertaining (extra data) and pseudo-labeling.\n\n## First Prizes Zone (2023)\n\n[kaggle.com/competitions/predict-student-performance-from-game-play](http://kaggle.com/competitions/predict-student-performance-from-game-play)\n\nIt is a mixed feeling. I am happy to have the 3rd here, but I feel like it is a relatively straightforward competition. What I can say is that I believe in what I have learned from this community, and it turns out to be great.\n\n## Moving Forwards\n\nI will still participate in competitions occasionally, but my Kaggle time is getting fewer and fewer, especially since machine learning is not my day job. I will start focusing more on sharing and contributing to the community. \n\nAs a first step, I implement the idea of “Null Importances” as a Python package here: https://github.com/kingychiu/target-permutation-importances. \nHope this is helpful to some newcomers here.\n\n## Special Thanks\n@fatihozturk @wimwim Teamed with me in many competitions, exchanging countless ideas online.\n\nIf you ask me to name someone out of my mind immediately...\n@ogrellier @cdeotte @cpmpml @narsil @tunguz \nUsually, I got insights from your comments/posts/works.",
    "2345546": "\nCongratulations",
    "2342607": "Congratulations! This is not just **a** milestone, but definitely a **great** one!\n\nAlso, thank you for sharing your experience. I myself just got my first gold too, and when reading this post I feel like I'm starting out like you did several years ago. Hopefully one day I'll reach where you are now.\n\nI have already learned a few things just by reading about your journey. There is a competition where I'm doing oversampling and then CV, and I keep wondering how to prevent leakage - you answered just this in your post! Look forward to seeing more of your sharing on Kaggle!\n\nIn addition to that, I have some questions about a few things mentioned in your post and hope you could help me answer:\n1. How do you often do feature ranking and selection? I usually select an arbitrary top `X` of the model's default `feature_importance`, but I feel like there is much more to that (what `X` should be, other importance metrics etc.). If you have any doc/resources you can point me to that'd be awesome too!\n2. You mentioned that in real-world data *all columns all tables are not masked* - what is masking for in this case?\n\nThanks and congratulations again!",
    "2347601": "Congratulations @kingychiu !\nWe look forward to more great work ahead!",
    "2344246": "@kingychiu seems lot of hardwork and consistency. Congratulations 🎉",
    "2343907": "Congratulations. Also. thanks for sharing your remarkable journey for achieving GM norm.\nML competitions at Kaggle are good learning place. The competitions are seeing exciting finish\nwith toppers score separated by minute fractions. \n",
    "2343017": "Kudos and congrats on becoming grandmaster @kingychiu ",
    "2342978": "Congratulations on becoming GM !! Well done.",
    "2342302": "Kudos on becoming competitions GM @kingychiu !",
    "2348057": "Big congrats on your GM tier! 🎉\n\nOh man, I'm so relieved and happy as I got it by myself... Thank you for mentioning me at the beginning of your \"Special Thanks\", it indeed feels special :) \n\nAnd I want to thank you so much as well! You made it so easy to team up for me (who is a relatively introverted person actually) Though we have been living in two very distant parts of the world, I always felt very comfortable and close talking to you and exchanging our ideas during all these competitions.\n\nJust remembered the Home Credit competition... It was our very first CV victory and the time and the excitement we had when we were waiting for the private LB results... :D\n\nAnyways, I always know you first, as a good friend and second as a very hardworking person. Well-deserved title!\n\nWishing you all the best, and looking forward to seeing more wins from you!",
    "2346185": "Achieving a solo gold in the Elo Merchant Category Recommendation competition by improving the 1st and 2nd stage model linkage through cross-validation demonstrates the importance of validation setup and trusting your cross-validation results. Great job on obtaining a robust result!",
    "2345551": "There's no success without the seeming failures. Congratulations on this milestone. ✌️",
    "2344954": "Congratulations! Keep it up!",
    "2344810": "Congratulations upon achieving the prestigious title. 👍",
    "2344718": "Congratulations man, It's a great achievement 🥳 ",
    "2344127": "Congratulations on becoming GM!  Keep up the great work! @kingychiu",
    "2343663": "Congratulations! ",
    "2343055": "Congratulations @kingychiu for becoming the **Kaggle Competitions Grandmaster**.🎉\nCongratulations on your incredible journey!!",
    "2342621": "Congratulations on becoming GM !!",
    "2342293": "Congratulations! And thank you for citing me.",
    "2342328": "Hearty congratulations @kingychiu \nThis is a remarkable achievement!! Keep up the great work!",
    "2348866": "Well done and congratulations! @kingychiu ",
    "2359136": "Congratulations @kingychiu ! \nAfter reading your GM journey, I was astonished, this is not only a milestone but indeed a remarkable one! I'm eager to learn from you!\n",
    "2348642": ""
  }
}