{
  "id": 416021,
  "title": "Eighth Place Solution: 5-Fold CV 1D-ResNet",
  "url": "/competitions/tlvmc-parkinsons-freezing-gait-prediction/writeups/the-parrot-s-beak-eighth-place-solution-5-fold-cv-",
  "author_name": "",
  "post_date": "2023-07-10T13:54:04.867Z",
  "votes": 24,
  "comment_count": 27,
  "views": 0,
  "content": "<p>EDIT BELOW!</p>\n<p>First of all thank you all for organising and participating in this wonderful challenge, I learned a lot about Parkinson and the techniques I used in my solution and especially used by others!</p>\n<p>The biggest thank you goes out to <a href=\"https://www.kaggle.com/mayukh18\" target=\"_blank\">@mayukh18</a> for an incredible starter notebook.<br>\nPlease give it an upvote if you haven't already!</p>\n<p><a href=\"https://www.kaggle.com/code/mayukh18/pytorch-fog-end-to-end-baseline-lb-0-254\" target=\"_blank\">Mayukh's Baseline Notebook</a></p>\n<p>I modified this notebook's dataset class to be compatible with with a 1D-ResNet that has three channels, one for each of the sensors ('AccV', 'AccML', 'AccAP'). I also removed the wx-parameter and used 1000ms cuts with a future of 50ms.</p>\n<p>Additionally, I implemented a learning rate scheduler (ReduceLROnPlateau) and started with a learning rate of 0.001.</p>\n<p>While the original notebook used only one train-validation split, I calculated five models based on the original folds in the notebook using each split.</p>\n<p>I calculated the models using my RTX 3060 which took about an hour per model, including validation. CV was around .30 if I remember correctly.</p>\n<p>My submission was the ensemble of the resulting five models and resulted in scores of : Public - .351 / Private - .356.</p>\n<p>If there are any questions I will gladly try to answer them in the upcoming days but as of now this is all I can do (moved to a new city just yesterday!).</p>\n<p>Best,<br>\nJan</p>\n<p>EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: <br>\nI took the time to generate a little write-up and publish the clean code, both available <a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/422477\" target=\"_blank\">here</a>.<br>\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: </p>",
  "messages": [
    {
      "id": "2293432",
      "postDate": "06/09/2023 07:29:37",
      "content": "<p>EDIT BELOW!</p>\n<p>First of all thank you all for organising and participating in this wonderful challenge, I learned a lot about Parkinson and the techniques I used in my solution and especially used by others!</p>\n<p>The biggest thank you goes out to <a href=\"https://www.kaggle.com/mayukh18\" target=\"_blank\">@mayukh18</a> for an incredible starter notebook.<br>\nPlease give it an upvote if you haven't already!</p>\n<p><a href=\"https://www.kaggle.com/code/mayukh18/pytorch-fog-end-to-end-baseline-lb-0-254\" target=\"_blank\">Mayukh's Baseline Notebook</a></p>\n<p>I modified this notebook's dataset class to be compatible with with a 1D-ResNet that has three channels, one for each of the sensors ('AccV', 'AccML', 'AccAP'). I also removed the wx-parameter and used 1000ms cuts with a future of 50ms.</p>\n<p>Additionally, I implemented a learning rate scheduler (ReduceLROnPlateau) and started with a learning rate of 0.001.</p>\n<p>While the original notebook used only one train-validation split, I calculated five models based on the original folds in the notebook using each split.</p>\n<p>I calculated the models using my RTX 3060 which took about an hour per model, including validation. CV was around .30 if I remember correctly.</p>\n<p>My submission was the ensemble of the resulting five models and resulted in scores of : Public - .351 / Private - .356.</p>\n<p>If there are any questions I will gladly try to answer them in the upcoming days but as of now this is all I can do (moved to a new city just yesterday!).</p>\n<p>Best,<br>\nJan</p>\n<p>EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: <br>\nI took the time to generate a little write-up and publish the clean code, both available <a href=\"https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/422477\" target=\"_blank\">here</a>.<br>\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: </p>",
      "rawMarkdown": "EDIT BELOW!\n\nFirst of all thank you all for organising and participating in this wonderful challenge, I learned a lot about Parkinson and the techniques I used in my solution and especially used by others!\n\nThe biggest thank you goes out to @mayukh18 for an incredible starter notebook.\nPlease give it an upvote if you haven't already!\n\n\n[Mayukh's Baseline Notebook](https://www.kaggle.com/code/mayukh18/pytorch-fog-end-to-end-baseline-lb-0-254)\n\n\nI modified this notebook's dataset class to be compatible with with a 1D-ResNet that has three channels, one for each of the sensors ('AccV', 'AccML', 'AccAP'). I also removed the wx-parameter and used 1000ms cuts with a future of 50ms.\n\nAdditionally, I implemented a learning rate scheduler (ReduceLROnPlateau) and started with a learning rate of 0.001.\n\nWhile the original notebook used only one train-validation split, I calculated five models based on the original folds in the notebook using each split.\n\nI calculated the models using my RTX 3060 which took about an hour per model, including validation. CV was around .30 if I remember correctly.\n\nMy submission was the ensemble of the resulting five models and resulted in scores of : Public - .351 / Private - .356.\n\nIf there are any questions I will gladly try to answer them in the upcoming days but as of now this is all I can do (moved to a new city just yesterday!).\n\nBest,\nJan\n\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: \nI took the time to generate a little write-up and publish the clean code, both available [here](https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/422477).\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT:",
      "votes": null
    },
    {
      "id": "2293448",
      "postDate": "06/09/2023 07:40:49",
      "content": "<p>it seems that top solutions are all neural networks?</p>",
      "rawMarkdown": "it seems that top solutions are all neural networks?",
      "votes": null
    },
    {
      "id": "2293602",
      "postDate": "06/09/2023 10:22:07",
      "content": "<p>How did you approach two datasets: defog and tdcsfog? Did you combine them together or did you train different models?<br>\nI'm curious because we implemented almost the same model but our score changed a lot. We trained separate models for defog and tdcsfog.</p>",
      "rawMarkdown": "How did you approach two datasets: defog and tdcsfog? Did you combine them together or did you train different models?\nI'm curious because we implemented almost the same model but our score changed a lot. We trained separate models for defog and tdcsfog.",
      "votes": null
    },
    {
      "id": "2293603",
      "postDate": "06/09/2023 10:23:37",
      "content": "<p>Hey Eric,</p>\n<p>I combined the two! Did not try separate models as I was in a hurry due to me moving. Seems I was lucky …</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Eric,\n\nI combined the two! Did not try separate models as I was in a hurry due to me moving. Seems I was lucky …\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2293616",
      "postDate": "06/09/2023 10:35:16",
      "content": "<p>Could you check our simplified notebook?<br>\n<a href=\"https://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton\" target=\"_blank\">https://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton</a><br>\nWe used this model and it seems very close to your description…<br>\nDo you see any difference?</p>",
      "rawMarkdown": "Could you check our simplified notebook?\nhttps://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton\nWe used this model and it seems very close to your description...\nDo you see any difference?",
      "votes": null
    },
    {
      "id": "2293621",
      "postDate": "06/09/2023 10:44:18",
      "content": "<p>Thanks for making me aware of 1D-ResNet (and similar models). It will be of great help in signal processing applications.</p>\n<p>Great work 👍</p>",
      "rawMarkdown": "Thanks for making me aware of 1D-ResNet (and similar models). It will be of great help in signal processing applications.\n\nGreat work 👍",
      "votes": null
    },
    {
      "id": "2293702",
      "postDate": "06/09/2023 12:19:08",
      "content": "<p>Very interesting! Did you see any inconsistencies between CV and LB scores?</p>",
      "rawMarkdown": "Very interesting! Did you see any inconsistencies between CV and LB scores?",
      "votes": null
    },
    {
      "id": "2293820",
      "postDate": "06/09/2023 14:32:30",
      "content": "<p>You're welcome, I think this is a very doable way to a simple but efficient model!</p>",
      "rawMarkdown": "You're welcome, I think this is a very doable way to a simple but efficient model!",
      "votes": null
    },
    {
      "id": "2293855",
      "postDate": "06/09/2023 14:44:20",
      "content": "<p>Hey Eric,<br>\nI will gladly look at your code but please give me a few days for such detailed answers as I live in complete chaos since I moved yesterday.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Eric,\nI will gladly look at your code but please give me a few days for such detailed answers as I live in complete chaos since I moved yesterday.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2293858",
      "postDate": "06/09/2023 14:45:14",
      "content": "<p>CV was lower consistently by about ~0.07 - that's all</p>",
      "rawMarkdown": "CV was lower consistently by about ~0.07 - that's all",
      "votes": null
    },
    {
      "id": "2294056",
      "postDate": "06/09/2023 17:29:56",
      "content": "<p>Thanks for sharing your approach, <a href=\"https://www.kaggle.com/janbrederecke\" target=\"_blank\">@janbrederecke</a> ! </p>\n<p>Interesting to see how you ensembled the five fold split model..</p>\n<p>Curious to know if you have used any time series based feature engineering to achieve this results?</p>\n<p>Keep up with the great work🔥💪</p>",
      "rawMarkdown": "Thanks for sharing your approach, @janbrederecke ! \n\nInteresting to see how you ensembled the five fold split model..\n\nCurious to know if you have used any time series based feature engineering to achieve this results?\n\nKeep up with the great work🔥💪",
      "votes": null
    },
    {
      "id": "2294102",
      "postDate": "06/09/2023 18:28:03",
      "content": "<p>Hey Vladimir,</p>\n<p>Thank you for your feedback!<br>\nI did not used any feature engineering and I simply added all the model predictions and divided by the number of models in my ensemble. </p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Vladimir,\n\nThank you for your feedback!\nI did not used any feature engineering and I simply added all the model predictions and divided by the number of models in my ensemble. \n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2294109",
      "postDate": "06/09/2023 18:33:29",
      "content": "<p>Nice one! Congratulations and thanks once again🙏</p>",
      "rawMarkdown": "Nice one! Congratulations and thanks once again🙏",
      "votes": null
    },
    {
      "id": "2295954",
      "postDate": "06/11/2023 12:36:16",
      "content": "<p>Now that more of the prize-winners published their solutions, you seem to be right! I honestly thought about using Light-GBM at first but then decided to use the 1D Resnet as I had not much time and was more comfortable using PyTorch … Good decision it seems :)</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Now that more of the prize-winners published their solutions, you seem to be right! I honestly thought about using Light-GBM at first but then decided to use the 1D Resnet as I had not much time and was more comfortable using PyTorch … Good decision it seems :)\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2295961",
      "postDate": "06/11/2023 12:39:10",
      "content": "<p>Hey Eric, I will publish my complete code once the competition is completely over (results are verified) and I had the chance to refactor my code as I have to admit that my code used is really the symbol of „quick‘n‘dirty“ - maybe we can then look for similarities / differences between our solutions!</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Eric, I will publish my complete code once the competition is completely over (results are verified) and I had the chance to refactor my code as I have to admit that my code used is really the symbol of „quick‘n‘dirty“ - maybe we can then look for similarities / differences between our solutions!\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2295964",
      "postDate": "06/11/2023 12:40:18",
      "content": "<p>Hey Vladimir,<br>\nthanks! I will publish the complete code when I had the chance to refactor it!</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Vladimir,\nthanks! I will publish the complete code when I had the chance to refactor it!\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2296247",
      "postDate": "06/11/2023 16:58:14",
      "content": "<p>This is powerful and elegant solution! Thank you for sharing! I have never heard about 1D-ResNet model before and I was trying to implement my own Conv model for this competition. For what number of epochs did you train your model?</p>",
      "rawMarkdown": "This is powerful and elegant solution! Thank you for sharing! I have never heard about 1D-ResNet model before and I was trying to implement my own Conv model for this competition. For what number of epochs did you train your model?",
      "votes": null
    },
    {
      "id": "2296251",
      "postDate": "06/11/2023 17:04:54",
      "content": "<p>Did you use pretrained model or did you train from scratch?</p>",
      "rawMarkdown": "Did you use pretrained model or did you train from scratch?",
      "votes": null
    },
    {
      "id": "2296282",
      "postDate": "06/11/2023 17:34:59",
      "content": "<p>Hey Andrii,</p>\n<p>I trained from scratch and used a single epoch, as I did not have more time (I joined quite late as I was busy with the BirdCLEF Challenge (and normal life?! )  before) - the modified Dataset Class from the notebook mentioned provided 5.000.000 datapoints for one training epoch which took about 45min for training with my 3060.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Andrii,\n\nI trained from scratch and used a single epoch, as I did not have more time (I joined quite late as I was busy with the BirdCLEF Challenge (and normal life?! )  before) - the modified Dataset Class from the notebook mentioned provided 5.000.000 datapoints for one training epoch which took about 45min for training with my 3060.\n\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2323981",
      "postDate": "06/30/2023 09:44:31",
      "content": "<p>Hi Jan,<br>\nyour 1d residual network seems very interesting and I'm so curious to see how it's build. Please update when you publish the code and if you have any reference I can look with an implementation example can you share the link?<br>\nBest regards</p>",
      "rawMarkdown": "Hi Jan,\nyour 1d residual network seems very interesting and I'm so curious to see how it's build. Please update when you publish the code and if you have any reference I can look with an implementation example can you share the link?\nBest regards",
      "votes": null
    },
    {
      "id": "2326001",
      "postDate": "07/01/2023 19:02:30",
      "content": "<p>Hey Alberto, thank you for your interest in my solution - I am currently working on the publication and will post an update as soon as it is published.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Alberto, thank you for your interest in my solution - I am currently working on the publication and will post an update as soon as it is published.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2337467",
      "postDate": "07/10/2023 05:59:10",
      "content": "<p>Hey Andrii,</p>\n<p>I just published a small article on my solution including the complete source code on GItHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Andrii,\n\nI just published a small article on my solution including the complete source code on GItHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2337469",
      "postDate": "07/10/2023 06:00:09",
      "content": "<p>Hey Eric,</p>\n<p>I just published a small article on my solution including the complete source code on GitHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Eric,\n\nI just published a small article on my solution including the complete source code on GitHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2337470",
      "postDate": "07/10/2023 06:00:48",
      "content": "<p>Hey Alberto,</p>\n<p>I just published a small article on my solution including the complete source code on GItHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Alberto,\n\nI just published a small article on my solution including the complete source code on GItHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2338142",
      "postDate": "07/10/2023 15:02:50",
      "content": "<p>This is great, high quality work, appreciate the effort!</p>",
      "rawMarkdown": "This is great, high quality work, appreciate the effort!",
      "votes": null
    },
    {
      "id": "2345937",
      "postDate": "07/15/2023 20:25:30",
      "content": "<p>Hey Vladimir,</p>\n<p>just published all the code and a small article:<br>\n<a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">Code</a></p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Hey Vladimir,\n\njust published all the code and a small article:\n[Code](https://github.com/janbrederecke/fog)\n\nBest,\nJan",
      "votes": null
    },
    {
      "id": "2348351",
      "postDate": "07/17/2023 14:56:30",
      "content": "<p>Great! <a href=\"https://www.kaggle.com/janbrederecke\" target=\"_blank\">@janbrederecke</a> !</p>\n<p>Will take a look at it right away.</p>\n<p>Thanks for sharing!</p>\n<p>Best regards,<br>\nVladimir</p>",
      "rawMarkdown": "Great! @janbrederecke !\n\nWill take a look at it right away.\n\nThanks for sharing!\n\nBest regards,\nVladimir",
      "votes": null
    },
    {
      "id": "2389702",
      "postDate": "08/14/2023 07:40:02",
      "content": "<p>Thank you so much, glad if it is helpful!</p>\n<p>Best,<br>\nJan</p>",
      "rawMarkdown": "Thank you so much, glad if it is helpful!\n\nBest,\nJan",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 2293448,
      "author_name": "xzj19013742",
      "author_url": "",
      "post_date": "06/09/2023 07:40:49",
      "content": "<p>it seems that top solutions are all neural networks?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2295954,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "06/11/2023 12:36:16",
          "content": "<p>Now that more of the prize-winners published their solutions, you seem to be right! I honestly thought about using Light-GBM at first but then decided to use the 1D Resnet as I had not much time and was more comfortable using PyTorch … Good decision it seems :)</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2293602,
      "author_name": "ernestglukhov",
      "author_url": "",
      "post_date": "06/09/2023 10:22:07",
      "content": "<p>How did you approach two datasets: defog and tdcsfog? Did you combine them together or did you train different models?<br>\nI'm curious because we implemented almost the same model but our score changed a lot. We trained separate models for defog and tdcsfog.</p>",
      "votes": null,
      "replies": [
        {
          "id": 2293603,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "06/09/2023 10:23:37",
          "content": "<p>Hey Eric,</p>\n<p>I combined the two! Did not try separate models as I was in a hurry due to me moving. Seems I was lucky …</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": [
            {
              "id": 2293616,
              "author_name": "ernestglukhov",
              "author_url": "",
              "post_date": "06/09/2023 10:35:16",
              "content": "<p>Could you check our simplified notebook?<br>\n<a href=\"https://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton\" target=\"_blank\">https://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton</a><br>\nWe used this model and it seems very close to your description…<br>\nDo you see any difference?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2293855,
                  "author_name": "janbrederecke",
                  "author_url": "",
                  "post_date": "06/09/2023 14:44:20",
                  "content": "<p>Hey Eric,<br>\nI will gladly look at your code but please give me a few days for such detailed answers as I live in complete chaos since I moved yesterday.</p>\n<p>Best,<br>\nJan</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            },
            {
              "id": 2293702,
              "author_name": "exjustice",
              "author_url": "",
              "post_date": "06/09/2023 12:19:08",
              "content": "<p>Very interesting! Did you see any inconsistencies between CV and LB scores?</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2293858,
                  "author_name": "janbrederecke",
                  "author_url": "",
                  "post_date": "06/09/2023 14:45:14",
                  "content": "<p>CV was lower consistently by about ~0.07 - that's all</p>",
                  "votes": null,
                  "replies": [
                    {
                      "id": 2295961,
                      "author_name": "janbrederecke",
                      "author_url": "",
                      "post_date": "06/11/2023 12:39:10",
                      "content": "<p>Hey Eric, I will publish my complete code once the competition is completely over (results are verified) and I had the chance to refactor my code as I have to admit that my code used is really the symbol of „quick‘n‘dirty“ - maybe we can then look for similarities / differences between our solutions!</p>\n<p>Best,<br>\nJan</p>",
                      "votes": null,
                      "replies": []
                    }
                  ]
                }
              ]
            }
          ]
        },
        {
          "id": 2337469,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "07/10/2023 06:00:09",
          "content": "<p>Hey Eric,</p>\n<p>I just published a small article on my solution including the complete source code on GitHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2293621,
      "author_name": "coderrkj",
      "author_url": "",
      "post_date": "06/09/2023 10:44:18",
      "content": "<p>Thanks for making me aware of 1D-ResNet (and similar models). It will be of great help in signal processing applications.</p>\n<p>Great work 👍</p>",
      "votes": null,
      "replies": [
        {
          "id": 2293820,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "06/09/2023 14:32:30",
          "content": "<p>You're welcome, I think this is a very doable way to a simple but efficient model!</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2294056,
      "author_name": "vladiluzjr",
      "author_url": "",
      "post_date": "06/09/2023 17:29:56",
      "content": "<p>Thanks for sharing your approach, <a href=\"https://www.kaggle.com/janbrederecke\" target=\"_blank\">@janbrederecke</a> ! </p>\n<p>Interesting to see how you ensembled the five fold split model..</p>\n<p>Curious to know if you have used any time series based feature engineering to achieve this results?</p>\n<p>Keep up with the great work🔥💪</p>",
      "votes": null,
      "replies": [
        {
          "id": 2294102,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "06/09/2023 18:28:03",
          "content": "<p>Hey Vladimir,</p>\n<p>Thank you for your feedback!<br>\nI did not used any feature engineering and I simply added all the model predictions and divided by the number of models in my ensemble. </p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": [
            {
              "id": 2294109,
              "author_name": "vladiluzjr",
              "author_url": "",
              "post_date": "06/09/2023 18:33:29",
              "content": "<p>Nice one! Congratulations and thanks once again🙏</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2295964,
                  "author_name": "janbrederecke",
                  "author_url": "",
                  "post_date": "06/11/2023 12:40:18",
                  "content": "<p>Hey Vladimir,<br>\nthanks! I will publish the complete code when I had the chance to refactor it!</p>\n<p>Best,<br>\nJan</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        },
        {
          "id": 2345937,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "07/15/2023 20:25:30",
          "content": "<p>Hey Vladimir,</p>\n<p>just published all the code and a small article:<br>\n<a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">Code</a></p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": [
            {
              "id": 2348351,
              "author_name": "vladiluzjr",
              "author_url": "",
              "post_date": "07/17/2023 14:56:30",
              "content": "<p>Great! <a href=\"https://www.kaggle.com/janbrederecke\" target=\"_blank\">@janbrederecke</a> !</p>\n<p>Will take a look at it right away.</p>\n<p>Thanks for sharing!</p>\n<p>Best regards,<br>\nVladimir</p>",
              "votes": null,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2296247,
      "author_name": "andrewdidenko",
      "author_url": "",
      "post_date": "06/11/2023 16:58:14",
      "content": "<p>This is powerful and elegant solution! Thank you for sharing! I have never heard about 1D-ResNet model before and I was trying to implement my own Conv model for this competition. For what number of epochs did you train your model?</p>",
      "votes": null,
      "replies": [
        {
          "id": 2296251,
          "author_name": "andrewdidenko",
          "author_url": "",
          "post_date": "06/11/2023 17:04:54",
          "content": "<p>Did you use pretrained model or did you train from scratch?</p>",
          "votes": null,
          "replies": [
            {
              "id": 2296282,
              "author_name": "janbrederecke",
              "author_url": "",
              "post_date": "06/11/2023 17:34:59",
              "content": "<p>Hey Andrii,</p>\n<p>I trained from scratch and used a single epoch, as I did not have more time (I joined quite late as I was busy with the BirdCLEF Challenge (and normal life?! )  before) - the modified Dataset Class from the notebook mentioned provided 5.000.000 datapoints for one training epoch which took about 45min for training with my 3060.</p>\n<p>Best,<br>\nJan</p>",
              "votes": null,
              "replies": []
            }
          ]
        },
        {
          "id": 2337467,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "07/10/2023 05:59:10",
          "content": "<p>Hey Andrii,</p>\n<p>I just published a small article on my solution including the complete source code on GItHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": []
        }
      ]
    },
    {
      "id": 2323981,
      "author_name": "albertoannoni",
      "author_url": "",
      "post_date": "06/30/2023 09:44:31",
      "content": "<p>Hi Jan,<br>\nyour 1d residual network seems very interesting and I'm so curious to see how it's build. Please update when you publish the code and if you have any reference I can look with an implementation example can you share the link?<br>\nBest regards</p>",
      "votes": null,
      "replies": [
        {
          "id": 2326001,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "07/01/2023 19:02:30",
          "content": "<p>Hey Alberto, thank you for your interest in my solution - I am currently working on the publication and will post an update as soon as it is published.</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": []
        },
        {
          "id": 2337470,
          "author_name": "janbrederecke",
          "author_url": "",
          "post_date": "07/10/2023 06:00:48",
          "content": "<p>Hey Alberto,</p>\n<p>I just published a small article on my solution including the complete source code on GItHub.</p>\n<p>Article: <a href=\"https://arxiv.org/abs/2307.03475\" target=\"_blank\">ArXiv</a></p>\n<p>Code: <a href=\"https://github.com/janbrederecke/fog\" target=\"_blank\">GitHub</a></p>\n<p>Just FYI.</p>\n<p>Best,<br>\nJan</p>",
          "votes": null,
          "replies": [
            {
              "id": 2338142,
              "author_name": "exjustice",
              "author_url": "",
              "post_date": "07/10/2023 15:02:50",
              "content": "<p>This is great, high quality work, appreciate the effort!</p>",
              "votes": null,
              "replies": [
                {
                  "id": 2389702,
                  "author_name": "janbrederecke",
                  "author_url": "",
                  "post_date": "08/14/2023 07:40:02",
                  "content": "<p>Thank you so much, glad if it is helpful!</p>\n<p>Best,<br>\nJan</p>",
                  "votes": null,
                  "replies": []
                }
              ]
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2293432": "EDIT BELOW!\n\nFirst of all thank you all for organising and participating in this wonderful challenge, I learned a lot about Parkinson and the techniques I used in my solution and especially used by others!\n\nThe biggest thank you goes out to @mayukh18 for an incredible starter notebook.\nPlease give it an upvote if you haven't already!\n\n\n[Mayukh's Baseline Notebook](https://www.kaggle.com/code/mayukh18/pytorch-fog-end-to-end-baseline-lb-0-254)\n\n\nI modified this notebook's dataset class to be compatible with with a 1D-ResNet that has three channels, one for each of the sensors ('AccV', 'AccML', 'AccAP'). I also removed the wx-parameter and used 1000ms cuts with a future of 50ms.\n\nAdditionally, I implemented a learning rate scheduler (ReduceLROnPlateau) and started with a learning rate of 0.001.\n\nWhile the original notebook used only one train-validation split, I calculated five models based on the original folds in the notebook using each split.\n\nI calculated the models using my RTX 3060 which took about an hour per model, including validation. CV was around .30 if I remember correctly.\n\nMy submission was the ensemble of the resulting five models and resulted in scores of : Public - .351 / Private - .356.\n\nIf there are any questions I will gladly try to answer them in the upcoming days but as of now this is all I can do (moved to a new city just yesterday!).\n\nBest,\nJan\n\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: \nI took the time to generate a little write-up and publish the clean code, both available [here](https://www.kaggle.com/competitions/tlvmc-parkinsons-freezing-gait-prediction/discussion/422477).\nEDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT: EDIT:",
    "2293448": "it seems that top solutions are all neural networks?",
    "2293602": "How did you approach two datasets: defog and tdcsfog? Did you combine them together or did you train different models?\nI'm curious because we implemented almost the same model but our score changed a lot. We trained separate models for defog and tdcsfog.",
    "2293603": "Hey Eric,\n\nI combined the two! Did not try separate models as I was in a hurry due to me moving. Seems I was lucky …\n\nBest,\nJan",
    "2293616": "Could you check our simplified notebook?\nhttps://www.kaggle.com/code/ernestglukhov/practicum-cnn-skeleton\nWe used this model and it seems very close to your description...\nDo you see any difference?",
    "2293621": "Thanks for making me aware of 1D-ResNet (and similar models). It will be of great help in signal processing applications.\n\nGreat work 👍",
    "2293702": "Very interesting! Did you see any inconsistencies between CV and LB scores?",
    "2293820": "You're welcome, I think this is a very doable way to a simple but efficient model!",
    "2293855": "Hey Eric,\nI will gladly look at your code but please give me a few days for such detailed answers as I live in complete chaos since I moved yesterday.\n\nBest,\nJan",
    "2293858": "CV was lower consistently by about ~0.07 - that's all",
    "2294056": "Thanks for sharing your approach, @janbrederecke ! \n\nInteresting to see how you ensembled the five fold split model..\n\nCurious to know if you have used any time series based feature engineering to achieve this results?\n\nKeep up with the great work🔥💪",
    "2294102": "Hey Vladimir,\n\nThank you for your feedback!\nI did not used any feature engineering and I simply added all the model predictions and divided by the number of models in my ensemble. \n\nBest,\nJan",
    "2294109": "Nice one! Congratulations and thanks once again🙏",
    "2295954": "Now that more of the prize-winners published their solutions, you seem to be right! I honestly thought about using Light-GBM at first but then decided to use the 1D Resnet as I had not much time and was more comfortable using PyTorch … Good decision it seems :)\n\nBest,\nJan",
    "2295961": "Hey Eric, I will publish my complete code once the competition is completely over (results are verified) and I had the chance to refactor my code as I have to admit that my code used is really the symbol of „quick‘n‘dirty“ - maybe we can then look for similarities / differences between our solutions!\n\nBest,\nJan",
    "2295964": "Hey Vladimir,\nthanks! I will publish the complete code when I had the chance to refactor it!\n\nBest,\nJan",
    "2296247": "This is powerful and elegant solution! Thank you for sharing! I have never heard about 1D-ResNet model before and I was trying to implement my own Conv model for this competition. For what number of epochs did you train your model?",
    "2296251": "Did you use pretrained model or did you train from scratch?",
    "2296282": "Hey Andrii,\n\nI trained from scratch and used a single epoch, as I did not have more time (I joined quite late as I was busy with the BirdCLEF Challenge (and normal life?! )  before) - the modified Dataset Class from the notebook mentioned provided 5.000.000 datapoints for one training epoch which took about 45min for training with my 3060.\n\n\nBest,\nJan",
    "2323981": "Hi Jan,\nyour 1d residual network seems very interesting and I'm so curious to see how it's build. Please update when you publish the code and if you have any reference I can look with an implementation example can you share the link?\nBest regards",
    "2326001": "Hey Alberto, thank you for your interest in my solution - I am currently working on the publication and will post an update as soon as it is published.\n\nBest,\nJan",
    "2337467": "Hey Andrii,\n\nI just published a small article on my solution including the complete source code on GItHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
    "2337469": "Hey Eric,\n\nI just published a small article on my solution including the complete source code on GitHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
    "2337470": "Hey Alberto,\n\nI just published a small article on my solution including the complete source code on GItHub.\n\nArticle: [ArXiv](https://arxiv.org/abs/2307.03475)\n\nCode: [GitHub](https://github.com/janbrederecke/fog)\n\nJust FYI.\n\nBest,\nJan",
    "2338142": "This is great, high quality work, appreciate the effort!",
    "2345937": "Hey Vladimir,\n\njust published all the code and a small article:\n[Code](https://github.com/janbrederecke/fog)\n\nBest,\nJan",
    "2348351": "Great! @janbrederecke !\n\nWill take a look at it right away.\n\nThanks for sharing!\n\nBest regards,\nVladimir",
    "2389702": "Thank you so much, glad if it is helpful!\n\nBest,\nJan"
  },
  "source": "meta"
}