{
  "id": 217238,
  "title": "[survey] Conducting lesson for this competition",
  "url": "/competitions/hpa-single-cell-image-classification/discussion/217238",
  "author_name": "hengck23",
  "post_date": "2021-02-06T01:41:54.994000",
  "votes": 133,
  "comment_count": 25,
  "views": 0,
  "content": "<p>In the past I have you helping kagglers with starter kit code. This time I want to try something different. I will be conducting a course on how to repeat top solution(s) a past competition. This past competition will be related to this one, so that you can transfer some techniques you learned to here.</p>\n<p>About the course:</p>\n<ul>\n<li>I am planning 3 lessons, mar-7, mar-14, mar-21</li>\n<li>At each of the three milestones, I prepare code and present results on how to train, <br>\nhow to overcome problems faced, etc … it focuses on the development process and <br>\nhow the choice of solution is made</li>\n<li>I will give a presentation of the above and release the code. There will be some discussion.<br>\nIt can be done on e.g. Zoom</li>\n<li>Kagglers can then repeat the code or improve on it. </li>\n<li>There will be a slack discussion group for discussion for Q and A and suggestion</li>\n<li>There will <strong>no discussion or code at all for this competition</strong>. Everything is based on past one. </li>\n</ul>\n<p>To help me to prepare my course material, I need to understand the target audience. Hence I need your help to complete the survey form here.</p>\n<p>Vote carefully. In the end, **only one past competition ** will be chosen as course material.</p>\n<p>This course is not for beginner. It is targeted for mid-expert, those on the silver medals zone. The objective is to move you from the silver to the gold zones. However, there is no restriction on participation. Everyone can join.</p>\n<p><a href=\"https://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing\" target=\"_blank\">https://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing</a></p>\n<p><img src=\"https://i.ibb.co/QvVkBqH/Selection-028.png\" alt=\"\"></p>\n<p>The survey ends at 29-Feb. The lesson plan will be released before mar-5.<br>\nEverything is subjected to change, depending on the response of the survey.</p>",
  "messages": [
    {
      "id": 1188119,
      "postDate": "2021-02-06T01:41:54.993Z",
      "content": "<p>In the past I have you helping kagglers with starter kit code. This time I want to try something different. I will be conducting a course on how to repeat top solution(s) a past competition. This past competition will be related to this one, so that you can transfer some techniques you learned to here.</p>\n<p>About the course:</p>\n<ul>\n<li>I am planning 3 lessons, mar-7, mar-14, mar-21</li>\n<li>At each of the three milestones, I prepare code and present results on how to train, <br>\nhow to overcome problems faced, etc … it focuses on the development process and <br>\nhow the choice of solution is made</li>\n<li>I will give a presentation of the above and release the code. There will be some discussion.<br>\nIt can be done on e.g. Zoom</li>\n<li>Kagglers can then repeat the code or improve on it. </li>\n<li>There will be a slack discussion group for discussion for Q and A and suggestion</li>\n<li>There will <strong>no discussion or code at all for this competition</strong>. Everything is based on past one. </li>\n</ul>\n<p>To help me to prepare my course material, I need to understand the target audience. Hence I need your help to complete the survey form here.</p>\n<p>Vote carefully. In the end, **only one past competition ** will be chosen as course material.</p>\n<p>This course is not for beginner. It is targeted for mid-expert, those on the silver medals zone. The objective is to move you from the silver to the gold zones. However, there is no restriction on participation. Everyone can join.</p>\n<p><a href=\"https://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing\" target=\"_blank\">https://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing</a></p>\n<p><img src=\"https://i.ibb.co/QvVkBqH/Selection-028.png\" alt=\"\"></p>\n<p>The survey ends at 29-Feb. The lesson plan will be released before mar-5.<br>\nEverything is subjected to change, depending on the response of the survey.</p>",
      "rawMarkdown": "In the past I have you helping kagglers with starter kit code. This time I want to try something different. I will be conducting a course on how to repeat top solution(s) a past competition. This past competition will be related to this one, so that you can transfer some techniques you learned to here.\n\nAbout the course:\n- I am planning 3 lessons, mar-7, mar-14, mar-21\n- At each of the three milestones, I prepare code and present results on how to train, \n   how to overcome problems faced, etc ... it focuses on the development process and \n   how the choice of solution is made\n- I will give a presentation of the above and release the code. There will be some discussion.\n  It can be done on e.g. Zoom\n- Kagglers can then repeat the code or improve on it. \n- There will be a slack discussion group for discussion for Q and A and suggestion\n- There will **no discussion or code at all for this competition**. Everything is based on past one. \n \nTo help me to prepare my course material, I need to understand the target audience. Hence I need your help to complete the survey form here.\n\nVote carefully. In the end, **only one past competition ** will be chosen as course material.\n\nThis course is not for beginner. It is targeted for mid-expert, those on the silver medals zone. The objective is to move you from the silver to the gold zones. However, there is no restriction on participation. Everyone can join.\n\nhttps://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing\n\n![](https://i.ibb.co/QvVkBqH/Selection-028.png)\n\nThe survey ends at 29-Feb. The lesson plan will be released before mar-5.\nEverything is subjected to change, depending on the response of the survey.",
      "votes": 133
    },
    {
      "id": 1222592,
      "postDate": "2021-03-01T23:55:46.973Z",
      "content": "<p>i have setup the following</p>\n<p>1) slack team</p>\n<ul>\n<li>please ensure the time zone in your profile is correct<br>\n(this enable me to decide what is the best time for video conference meeting)<br>\n<a href=\"https://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q\" target=\"_blank\">https://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q</a></li>\n</ul>\n<p>2) google drive</p>\n<ul>\n<li><p><a href=\"https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing</a></p></li>\n<li><p>this is for the development code I want to share. first version of starter code has been up.</p></li>\n</ul>\n<hr>\n<p>results of the survey:</p>\n<p>[objective of course]</p>\n<ul>\n<li>repeat and understand the development process of the top solution [1] of the past competition HPA-2018 [2]<br>\ni.e. even if we know the solution, can we develop code and train the model to get familiar results? can we uncover the devils in the details?</li>\n</ul>\n<p>speficially, our targets is image classification: </p>\n<ul>\n<li>densenet121 (CNN) : LB private/public = 0.56/0.62</li>\n<li>densenet121 (CNN+metric) : LB private/public = 0.59/0.65  (about +0.03)</li>\n</ul>\n<p>[1] <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109</a><br>\n[2] <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification</a></p>\n<ul>\n<li>run some modern algorithms (e.g. vision transformer, self-supervised Swav?) on HPA-2018 and get some results. No performance target set here.</li>\n</ul>\n<p>[milestone]<br>\n-0301: release of baseline code (please go to google drive folder \"2020-0301\"). A single densenet121 fold-0 acheives  LB private/public = 0.47998/0.47496 (without external data)</p>\n<p>-0307: video conference to present results of</p>\n<ul>\n<li>use of external data<ul>\n<li>initial observation, e.g. use of focal loss</li></ul></li>\n<li>initial results of CNN, CNN+metric (i hope i can get around LB private/public = 0.55/0.60)  </li>\n</ul>\n<p>-0314: video conference to present results of</p>\n<ul>\n<li>fine-tunning tricks and observation </li>\n<li>final results of CNN, CNN+metric (i hope i can get around LB private/public = 0.60/0.65)  </li>\n</ul>\n<p>-0321 video conference to present results of</p>\n<ul>\n<li>future exploration and observation </li>\n<li>e.g. vision transformer, self-supervised Swav</li>\n</ul>\n<hr>\n<ul>\n<li>i have just reworked the problem from scratch yesterday. Actually, HPA-2018 is not an easy task, especially:</li>\n<li>train and public/private test data are different (use of external data complicates the matter as the external data set is also different). Setting up a training framework with local CV correlating with public LB is difficult. This is actually a good project to repeat others top solution to master the tricks of  to \"properly train a model when there is significantly domain different in train, test, external dataset\"</li>\n</ul>",
      "rawMarkdown": "i have setup the following\n\n1) slack team\n- please ensure the time zone in your profile is correct\n  (this enable me to decide what is the best time for video conference meeting)\nhttps://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q\n\n2) google drive\n- https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing\n\n- this is for the development code I want to share. first version of starter code has been up.\n\n---\nresults of the survey:\n\n[objective of course]\n- repeat and understand the development process of the top solution [1] of the past competition HPA-2018 [2]\n  i.e. even if we know the solution, can we develop code and train the model to get familiar results? can we uncover the devils in the details?\n\nspeficially, our targets is image classification: \n- densenet121 (CNN) : LB private/public = 0.56/0.62\n- densenet121 (CNN+metric) : LB private/public = 0.59/0.65  (about +0.03)\n     \n[1] https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\n[2] https://www.kaggle.com/c/human-protein-atlas-image-classification\n\n- run some modern algorithms (e.g. vision transformer, self-supervised Swav?) on HPA-2018 and get some results. No performance target set here.\n\n\n[milestone]\n-0301: release of baseline code (please go to google drive folder \"2020-0301\"). A single densenet121 fold-0 acheives  LB private/public = 0.47998/0.47496 (without external data)\n\n-0307: video conference to present results of\n   - use of external data\n  - initial observation, e.g. use of focal loss\n   - initial results of CNN, CNN+metric (i hope i can get around LB private/public = 0.55/0.60)  \n\n-0314: video conference to present results of\n   - fine-tunning tricks and observation \n   - final results of CNN, CNN+metric (i hope i can get around LB private/public = 0.60/0.65)  \n\n\n-0321 video conference to present results of\n   - future exploration and observation \n   - e.g. vision transformer, self-supervised Swav\n\n\n---\n\n- i have just reworked the problem from scratch yesterday. Actually, HPA-2018 is not an easy task, especially:\n- train and public/private test data are different (use of external data complicates the matter as the external data set is also different). Setting up a training framework with local CV correlating with public LB is difficult. This is actually a good project to repeat others top solution to master the tricks of  to \"properly train a model when there is significantly domain different in train, test, external dataset\"\n \n",
      "votes": 7,
      "replies": [
        {
          "id": 1222753,
          "postDate": "2021-03-02T05:39:43.643Z",
          "content": "<p>previous bestfitting code:<br>\n<a href=\"https://github.com/CellProfiling/HPA-competition-solutions\" target=\"_blank\">https://github.com/CellProfiling/HPA-competition-solutions</a><br>\n<a href=\"https://github.com/CellProfiling/HPA-competition#Source-code-of-team-solutions\" target=\"_blank\">https://github.com/CellProfiling/HPA-competition#Source-code-of-team-solutions</a><br>\n<a href=\"https://www.nature.com/articles/s41592-019-0658-6.epdf?shared_access_token=1f-vE5M2Nbk9yIytlGzhhNRgN0jAjWel9jnR3ZoTv0M3bV4dn-yP-0ZVNmntgbt1B9dnxVNJFA4G95nRwc6YQBRkTOvf8Fz2VxeaCCFDBPBltR8NtVFhHq77x45pVP7nlpfdTmnHbp34-WXWxiPb7Q%3D%3D\" target=\"_blank\">https://www.nature.com/articles/s41592-019-0658-6.epdf?shared_access_token=1f-vE5M2Nbk9yIytlGzhhNRgN0jAjWel9jnR3ZoTv0M3bV4dn-yP-0ZVNmntgbt1B9dnxVNJFA4G95nRwc6YQBRkTOvf8Fz2VxeaCCFDBPBltR8NtVFhHq77x45pVP7nlpfdTmnHbp34-WXWxiPb7Q%3D%3D</a></p>",
          "rawMarkdown": "previous bestfitting code:\nhttps://github.com/CellProfiling/HPA-competition-solutions\nhttps://github.com/CellProfiling/HPA-competition#Source-code-of-team-solutions\nhttps://www.nature.com/articles/s41592-019-0658-6.epdf?shared_access_token=1f-vE5M2Nbk9yIytlGzhhNRgN0jAjWel9jnR3ZoTv0M3bV4dn-yP-0ZVNmntgbt1B9dnxVNJFA4G95nRwc6YQBRkTOvf8Fz2VxeaCCFDBPBltR8NtVFhHq77x45pVP7nlpfdTmnHbp34-WXWxiPb7Q%3D%3D",
          "votes": 1
        }
      ]
    },
    {
      "id": 1217606,
      "postDate": "2021-02-25T07:27:58.713Z",
      "content": "<p>seems that we have a clear winner:</p>\n<p>Human Protein Atlas Image Classification            <br>\n<a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification</a>    </p>\n<p>one may start explore this past competition first. It will help to speed up your understanding when the lesson kick-in.        </p>",
      "rawMarkdown": "seems that we have a clear winner:\n\nHuman Protein Atlas Image Classification\t\t\t\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification\t\n\none may start explore this past competition first. It will help to speed up your understanding when the lesson kick-in.\t\t",
      "votes": 3
    },
    {
      "id": 1228299,
      "postDate": "2021-03-06T08:53:27.557Z",
      "content": "<p>I found the folder 2020-0301 from Google drive is empty. Is it supposed to contain ur dev code?</p>",
      "rawMarkdown": "I found the folder 2020-0301 from Google drive is empty. Is it supposed to contain ur dev code?",
      "votes": 1,
      "replies": [
        {
          "id": 1231856,
          "postDate": "2021-03-09T10:03:13.927Z",
          "content": "<p>Updated code 2020-0307.zip is in this folder <a href=\"https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF\" target=\"_blank\">https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF</a></p>",
          "rawMarkdown": "Updated code 2020-0307.zip is in this folder [https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF](https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF)"
        },
        {
          "id": 1236388,
          "postDate": "2021-03-13T05:05:44.253Z",
          "content": "<p>it is not supposed to be empty. i have reuploaded the contents in 2020-0301</p>",
          "rawMarkdown": "it is not supposed to be empty. i have reuploaded the contents in 2020-0301"
        }
      ]
    },
    {
      "id": 1219685,
      "postDate": "2021-02-27T05:48:46.200Z",
      "content": "<p>Can you record the videos/lessons and upload it somewhere as it may not be possible to see it live due to various problems.<br>\nThank you!!</p>",
      "rawMarkdown": "Can you record the videos/lessons and upload it somewhere as it may not be possible to see it live due to various problems.\nThank you!!",
      "votes": 1,
      "replies": [
        {
          "id": 1222806,
          "postDate": "2021-03-02T06:47:21.610Z",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>, please confirm</p>",
          "rawMarkdown": "@hengck23, please confirm"
        }
      ]
    },
    {
      "id": 1236389,
      "postDate": "2021-03-13T05:07:37.733Z",
      "content": "<p>for those joining the slack group, please note:<br>\n<img src=\"https://i.ibb.co/NttSdYP/Selection-240.png\" alt=\"\"> </p>",
      "rawMarkdown": "for those joining the slack group, please note:\n![](https://i.ibb.co/NttSdYP/Selection-240.png) ",
      "replies": [
        {
          "id": 1237306,
          "postDate": "2021-03-14T03:28:58.317Z",
          "content": "<p>[14-mar] today's talk is about 2018 HPA bestfitting's winning solution. How to use metric learning to make a KNN classifier to recover binary labels for image classification. Experimental results will be presented on 2018 HPA.<br>\nJoin the slack for more information.<br>\n<img src=\"https://i.ibb.co/zFFHKmj/Selection-270.png\" alt=\"\"><br>\n<img src=\"https://i.ibb.co/Tm2KpxG/Selection-274.png\" alt=\"\"><br>\n<img src=\"https://i.ibb.co/D4NWcG5/Selection-319.png\" alt=\"\"></p>",
          "rawMarkdown": "[14-mar] today's talk is about 2018 HPA bestfitting's winning solution. How to use metric learning to make a KNN classifier to recover binary labels for image classification. Experimental results will be presented on 2018 HPA.\nJoin the slack for more information.\n![](https://i.ibb.co/zFFHKmj/Selection-270.png)\n![](https://i.ibb.co/Tm2KpxG/Selection-274.png)\n![](https://i.ibb.co/D4NWcG5/Selection-319.png)\n"
        },
        {
          "id": 1238374,
          "postDate": "2021-03-15T00:03:53.810Z",
          "content": "<p>understanding and observing the data is always important. this is the data characteristics of HPA 2018 external data:<br>\n<img src=\"https://i.ibb.co/M2M61k8/Selection-317.png\" alt=\"\"></p>",
          "rawMarkdown": "understanding and observing the data is always important. this is the data characteristics of HPA 2018 external data:\n![](https://i.ibb.co/M2M61k8/Selection-317.png)\n"
        }
      ]
    },
    {
      "id": 1225999,
      "postDate": "2021-03-04T06:21:31.780Z",
      "content": "<p>I'm late here, but just want to say thank you very much for the initiative. </p>",
      "rawMarkdown": "I'm late here, but just want to say thank you very much for the initiative. "
    },
    {
      "id": 1217615,
      "postDate": "2021-02-25T07:40:05.817Z",
      "content": "<p>very kind offer. any suggested time?</p>",
      "rawMarkdown": "very kind offer. any suggested time?"
    },
    {
      "id": 1210254,
      "postDate": "2021-02-19T09:45:23.277Z",
      "content": "<p>What a great idea <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> !! Awesome of you to help others learn and share!! 👍🙌🙏😻</p>",
      "rawMarkdown": "What a great idea @hengck23 !! Awesome of you to help others learn and share!! 👍🙌🙏😻"
    },
    {
      "id": 1193912,
      "postDate": "2021-02-10T00:53:19.460Z",
      "content": "<p>I think this is a great idea, how will you let us know? Do you plan on having a form to participate or will you share the link on kaggle? I think it's awesome that you're doing this.</p>",
      "rawMarkdown": "I think this is a great idea, how will you let us know? Do you plan on having a form to participate or will you share the link on kaggle? I think it's awesome that you're doing this.",
      "replies": [
        {
          "id": 1208926,
          "postDate": "2021-02-18T15:09:05.690Z",
          "content": "<p>i will share a link here in the beginning of March. </p>",
          "rawMarkdown": "i will share a link here in the beginning of March. "
        }
      ]
    },
    {
      "id": 1192368,
      "postDate": "2021-02-09T05:05:14.947Z",
      "content": "<p>May I ask one kaggler can only vote one past competition or more?</p>",
      "rawMarkdown": "May I ask one kaggler can only vote one past competition or more?",
      "replies": [
        {
          "id": 1192976,
          "postDate": "2021-02-09T11:45:11.090Z",
          "content": "<p>preferably one</p>",
          "rawMarkdown": "preferably one",
          "votes": 2
        }
      ]
    },
    {
      "id": 1192245,
      "postDate": "2021-02-09T04:02:16.623Z",
      "content": "<p>Wowwwww!!!!! Definitly will participate!!! Thank you sooooo much and cant wait.</p>",
      "rawMarkdown": "Wowwwww!!!!! Definitly will participate!!! Thank you sooooo much and cant wait."
    },
    {
      "id": 1192006,
      "postDate": "2021-02-08T21:05:16.207Z",
      "content": "<p>Thanks a lot for the super generous effort.</p>",
      "rawMarkdown": "Thanks a lot for the super generous effort."
    },
    {
      "id": 1189538,
      "postDate": "2021-02-07T05:17:37.070Z",
      "content": "<p>Super generous <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> . I have added my favourite one</p>",
      "rawMarkdown": "Super generous @hengck23 . I have added my favourite one"
    },
    {
      "id": 1189384,
      "postDate": "2021-02-07T01:02:15.053Z",
      "content": "<p>Awsome, thanks for your generous offering! Can't wait :)</p>",
      "rawMarkdown": "Awsome, thanks for your generous offering! Can't wait :)"
    },
    {
      "id": 1188299,
      "postDate": "2021-02-06T05:53:34.387Z",
      "content": "<p>This sounds great, thank you for offering this! I added Bengali to the list - it feels to me that classification will be more important in this competition than segmentation, because we have a good baseline with CellSegmentator. Also, there is a difference between training and test set that is critical, for example how to do evaluation other than the public LB?</p>",
      "rawMarkdown": "This sounds great, thank you for offering this! I added Bengali to the list - it feels to me that classification will be more important in this competition than segmentation, because we have a good baseline with CellSegmentator. Also, there is a difference between training and test set that is critical, for example how to do evaluation other than the public LB?",
      "replies": [
        {
          "id": 1188380,
          "postDate": "2021-02-06T07:35:47.877Z",
          "content": "<p><a href=\"https://www.kaggle.com/thedrcat\" target=\"_blank\">@thedrcat</a> </p>\n<p>thanks for filling up the form. However, you did not cast your vote.<br>\n(you need to put your userid in the \"Vote\" column)</p>\n<p>Note that there is only one past competition to be chosen as class material, so vote carefully. Thanks!</p>",
          "rawMarkdown": "@thedrcat \n\nthanks for filling up the form. However, you did not cast your vote.\n(you need to put your userid in the \"Vote\" column)\n\nNote that there is only one past competition to be chosen as class material, so vote carefully. Thanks!\n\n"
        }
      ]
    },
    {
      "id": 1192002,
      "postDate": "2021-02-08T21:01:21.427Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 1222592,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2021-03-01T23:55:46.973000",
      "content": "<p>i have setup the following</p>\n<p>1) slack team</p>\n<ul>\n<li>please ensure the time zone in your profile is correct<br>\n(this enable me to decide what is the best time for video conference meeting)<br>\n<a href=\"https://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q\" target=\"_blank\">https://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q</a></li>\n</ul>\n<p>2) google drive</p>\n<ul>\n<li><p><a href=\"https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing\" target=\"_blank\">https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing</a></p></li>\n<li><p>this is for the development code I want to share. first version of starter code has been up.</p></li>\n</ul>\n<hr>\n<p>results of the survey:</p>\n<p>[objective of course]</p>\n<ul>\n<li>repeat and understand the development process of the top solution [1] of the past competition HPA-2018 [2]<br>\ni.e. even if we know the solution, can we develop code and train the model to get familiar results? can we uncover the devils in the details?</li>\n</ul>\n<p>speficially, our targets is image classification: </p>\n<ul>\n<li>densenet121 (CNN) : LB private/public = 0.56/0.62</li>\n<li>densenet121 (CNN+metric) : LB private/public = 0.59/0.65  (about +0.03)</li>\n</ul>\n<p>[1] <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109</a><br>\n[2] <a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification</a></p>\n<ul>\n<li>run some modern algorithms (e.g. vision transformer, self-supervised Swav?) on HPA-2018 and get some results. No performance target set here.</li>\n</ul>\n<p>[milestone]<br>\n-0301: release of baseline code (please go to google drive folder \"2020-0301\"). A single densenet121 fold-0 acheives  LB private/public = 0.47998/0.47496 (without external data)</p>\n<p>-0307: video conference to present results of</p>\n<ul>\n<li>use of external data<ul>\n<li>initial observation, e.g. use of focal loss</li></ul></li>\n<li>initial results of CNN, CNN+metric (i hope i can get around LB private/public = 0.55/0.60)  </li>\n</ul>\n<p>-0314: video conference to present results of</p>\n<ul>\n<li>fine-tunning tricks and observation </li>\n<li>final results of CNN, CNN+metric (i hope i can get around LB private/public = 0.60/0.65)  </li>\n</ul>\n<p>-0321 video conference to present results of</p>\n<ul>\n<li>future exploration and observation </li>\n<li>e.g. vision transformer, self-supervised Swav</li>\n</ul>\n<hr>\n<ul>\n<li>i have just reworked the problem from scratch yesterday. Actually, HPA-2018 is not an easy task, especially:</li>\n<li>train and public/private test data are different (use of external data complicates the matter as the external data set is also different). Setting up a training framework with local CV correlating with public LB is difficult. This is actually a good project to repeat others top solution to master the tricks of  to \"properly train a model when there is significantly domain different in train, test, external dataset\"</li>\n</ul>",
      "votes": 7,
      "replies": [
        {
          "id": 1222753,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-03-02T05:39:43.643000",
          "content": "<p>previous bestfitting code:<br>\n<a href=\"https://github.com/CellProfiling/HPA-competition-solutions\" target=\"_blank\">https://github.com/CellProfiling/HPA-competition-solutions</a><br>\n<a href=\"https://github.com/CellProfiling/HPA-competition#Source-code-of-team-solutions\" target=\"_blank\">https://github.com/CellProfiling/HPA-competition#Source-code-of-team-solutions</a><br>\n<a href=\"https://www.nature.com/articles/s41592-019-0658-6.epdf?shared_access_token=1f-vE5M2Nbk9yIytlGzhhNRgN0jAjWel9jnR3ZoTv0M3bV4dn-yP-0ZVNmntgbt1B9dnxVNJFA4G95nRwc6YQBRkTOvf8Fz2VxeaCCFDBPBltR8NtVFhHq77x45pVP7nlpfdTmnHbp34-WXWxiPb7Q%3D%3D\" target=\"_blank\">https://www.nature.com/articles/s41592-019-0658-6.epdf?shared_access_token=1f-vE5M2Nbk9yIytlGzhhNRgN0jAjWel9jnR3ZoTv0M3bV4dn-yP-0ZVNmntgbt1B9dnxVNJFA4G95nRwc6YQBRkTOvf8Fz2VxeaCCFDBPBltR8NtVFhHq77x45pVP7nlpfdTmnHbp34-WXWxiPb7Q%3D%3D</a></p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1217606,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2021-02-25T07:27:58.713000",
      "content": "<p>seems that we have a clear winner:</p>\n<p>Human Protein Atlas Image Classification            <br>\n<a href=\"https://www.kaggle.com/c/human-protein-atlas-image-classification\" target=\"_blank\">https://www.kaggle.com/c/human-protein-atlas-image-classification</a>    </p>\n<p>one may start explore this past competition first. It will help to speed up your understanding when the lesson kick-in.        </p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 1228299,
      "author_name": "Alex Lau",
      "author_url": "",
      "post_date": "2021-03-06T08:53:27.557000",
      "content": "<p>I found the folder 2020-0301 from Google drive is empty. Is it supposed to contain ur dev code?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1231856,
          "author_name": "Sirish Somanchi",
          "author_url": "",
          "post_date": "2021-03-09T10:03:13.927000",
          "content": "<p>Updated code 2020-0307.zip is in this folder <a href=\"https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF\" target=\"_blank\">https://drive.google.com/drive/folders/1dqG7WGq2ehQ2ZgO-yzOFoVK64e_1WrJF</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1236388,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-03-13T05:05:44.253000",
          "content": "<p>it is not supposed to be empty. i have reuploaded the contents in 2020-0301</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1219685,
      "author_name": "DeepUnderstanding",
      "author_url": "",
      "post_date": "2021-02-27T05:48:46.200000",
      "content": "<p>Can you record the videos/lessons and upload it somewhere as it may not be possible to see it live due to various problems.<br>\nThank you!!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1222806,
          "author_name": "DeepUnderstanding",
          "author_url": "",
          "post_date": "2021-03-02T06:47:21.610000",
          "content": "<p><a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a>, please confirm</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1236389,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2021-03-13T05:07:37.733000",
      "content": "<p>for those joining the slack group, please note:<br>\n<img src=\"https://i.ibb.co/NttSdYP/Selection-240.png\" alt=\"\"> </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1237306,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-03-14T03:28:58.317000",
          "content": "<p>[14-mar] today's talk is about 2018 HPA bestfitting's winning solution. How to use metric learning to make a KNN classifier to recover binary labels for image classification. Experimental results will be presented on 2018 HPA.<br>\nJoin the slack for more information.<br>\n<img src=\"https://i.ibb.co/zFFHKmj/Selection-270.png\" alt=\"\"><br>\n<img src=\"https://i.ibb.co/Tm2KpxG/Selection-274.png\" alt=\"\"><br>\n<img src=\"https://i.ibb.co/D4NWcG5/Selection-319.png\" alt=\"\"></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1238374,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-03-15T00:03:53.810000",
          "content": "<p>understanding and observing the data is always important. this is the data characteristics of HPA 2018 external data:<br>\n<img src=\"https://i.ibb.co/M2M61k8/Selection-317.png\" alt=\"\"></p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1225999,
      "author_name": "yukiya",
      "author_url": "",
      "post_date": "2021-03-04T06:21:31.780000",
      "content": "<p>I'm late here, but just want to say thank you very much for the initiative. </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1217615,
      "author_name": "Dingo",
      "author_url": "",
      "post_date": "2021-02-25T07:40:05.817000",
      "content": "<p>very kind offer. any suggested time?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1210254,
      "author_name": "Kamal Das",
      "author_url": "",
      "post_date": "2021-02-19T09:45:23.277000",
      "content": "<p>What a great idea <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> !! Awesome of you to help others learn and share!! 👍🙌🙏😻</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1193912,
      "author_name": "Gabriel Prado",
      "author_url": "",
      "post_date": "2021-02-10T00:53:19.460000",
      "content": "<p>I think this is a great idea, how will you let us know? Do you plan on having a form to participate or will you share the link on kaggle? I think it's awesome that you're doing this.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1208926,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-02-18T15:09:05.690000",
          "content": "<p>i will share a link here in the beginning of March. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1192368,
      "author_name": "Zhongkai Shangguan",
      "author_url": "",
      "post_date": "2021-02-09T05:05:14.947000",
      "content": "<p>May I ask one kaggler can only vote one past competition or more?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1192976,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-02-09T11:45:11.090000",
          "content": "<p>preferably one</p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 1192245,
      "author_name": "Zhongkai Shangguan",
      "author_url": "",
      "post_date": "2021-02-09T04:02:16.623000",
      "content": "<p>Wowwwww!!!!! Definitly will participate!!! Thank you sooooo much and cant wait.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1192006,
      "author_name": " निധിin",
      "author_url": "",
      "post_date": "2021-02-08T21:05:16.207000",
      "content": "<p>Thanks a lot for the super generous effort.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1189538,
      "author_name": "Ambarish Ganguly",
      "author_url": "",
      "post_date": "2021-02-07T05:17:37.070000",
      "content": "<p>Super generous <a href=\"https://www.kaggle.com/hengck23\" target=\"_blank\">@hengck23</a> . I have added my favourite one</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1189384,
      "author_name": "newbiejailer",
      "author_url": "",
      "post_date": "2021-02-07T01:02:15.053000",
      "content": "<p>Awsome, thanks for your generous offering! Can't wait :)</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1188299,
      "author_name": "Darek Kłeczek",
      "author_url": "",
      "post_date": "2021-02-06T05:53:34.387000",
      "content": "<p>This sounds great, thank you for offering this! I added Bengali to the list - it feels to me that classification will be more important in this competition than segmentation, because we have a good baseline with CellSegmentator. Also, there is a difference between training and test set that is critical, for example how to do evaluation other than the public LB?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1188380,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2021-02-06T07:35:47.877000",
          "content": "<p><a href=\"https://www.kaggle.com/thedrcat\" target=\"_blank\">@thedrcat</a> </p>\n<p>thanks for filling up the form. However, you did not cast your vote.<br>\n(you need to put your userid in the \"Vote\" column)</p>\n<p>Note that there is only one past competition to be chosen as class material, so vote carefully. Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1192002,
      "author_name": "",
      "author_url": "",
      "post_date": "2021-02-08T21:01:21.427000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1188119": "In the past I have you helping kagglers with starter kit code. This time I want to try something different. I will be conducting a course on how to repeat top solution(s) a past competition. This past competition will be related to this one, so that you can transfer some techniques you learned to here.\n\nAbout the course:\n- I am planning 3 lessons, mar-7, mar-14, mar-21\n- At each of the three milestones, I prepare code and present results on how to train, \n   how to overcome problems faced, etc ... it focuses on the development process and \n   how the choice of solution is made\n- I will give a presentation of the above and release the code. There will be some discussion.\n  It can be done on e.g. Zoom\n- Kagglers can then repeat the code or improve on it. \n- There will be a slack discussion group for discussion for Q and A and suggestion\n- There will **no discussion or code at all for this competition**. Everything is based on past one. \n \nTo help me to prepare my course material, I need to understand the target audience. Hence I need your help to complete the survey form here.\n\nVote carefully. In the end, **only one past competition ** will be chosen as course material.\n\nThis course is not for beginner. It is targeted for mid-expert, those on the silver medals zone. The objective is to move you from the silver to the gold zones. However, there is no restriction on participation. Everyone can join.\n\nhttps://docs.google.com/spreadsheets/d/14nsuHH82idg4tB3ORM7v3AY-Fhn8aNKAQxp62qZQyGk/edit?usp=sharing\n\n![](https://i.ibb.co/QvVkBqH/Selection-028.png)\n\nThe survey ends at 29-Feb. The lesson plan will be released before mar-5.\nEverything is subjected to change, depending on the response of the survey.",
    "1222592": "i have setup the following\n\n1) slack team\n- please ensure the time zone in your profile is correct\n  (this enable me to decide what is the best time for video conference meeting)\nhttps://join.slack.com/t/newworkspace-h0x4511/shared_invite/zt-n3jinlgk-HQHG6hvHbfnNbIXixXnp7Q\n\n2) google drive\n- https://drive.google.com/drive/folders/1IHoB2r5GaroAUS4Y7i_uq6n1FEKG4lh1?usp=sharing\n\n- this is for the development code I want to share. first version of starter code has been up.\n\n---\nresults of the survey:\n\n[objective of course]\n- repeat and understand the development process of the top solution [1] of the past competition HPA-2018 [2]\n  i.e. even if we know the solution, can we develop code and train the model to get familiar results? can we uncover the devils in the details?\n\nspeficially, our targets is image classification: \n- densenet121 (CNN) : LB private/public = 0.56/0.62\n- densenet121 (CNN+metric) : LB private/public = 0.59/0.65  (about +0.03)\n     \n[1] https://www.kaggle.com/c/human-protein-atlas-image-classification/discussion/78109\n[2] https://www.kaggle.com/c/human-protein-atlas-image-classification\n\n- run some modern algorithms (e.g. vision transformer, self-supervised Swav?) on HPA-2018 and get some results. No performance target set here.\n\n\n[milestone]\n-0301: release of baseline code (please go to google drive folder \"2020-0301\"). A single densenet121 fold-0 acheives  LB private/public = 0.47998/0.47496 (without external data)\n\n-0307: video conference to present results of\n   - use of external data\n  - initial observation, e.g. use of focal loss\n   - initial results of CNN, CNN+metric (i hope i can get around LB private/public = 0.55/0.60)  \n\n-0314: video conference to present results of\n   - fine-tunning tricks and observation \n   - final results of CNN, CNN+metric (i hope i can get around LB private/public = 0.60/0.65)  \n\n\n-0321 video conference to present results of\n   - future exploration and observation \n   - e.g. vision transformer, self-supervised Swav\n\n\n---\n\n- i have just reworked the problem from scratch yesterday. Actually, HPA-2018 is not an easy task, especially:\n- train and public/private test data are different (use of external data complicates the matter as the external data set is also different). Setting up a training framework with local CV correlating with public LB is difficult. This is actually a good project to repeat others top solution to master the tricks of  to \"properly train a model when there is significantly domain different in train, test, external dataset\"\n \n",
    "1217606": "seems that we have a clear winner:\n\nHuman Protein Atlas Image Classification\t\t\t\nhttps://www.kaggle.com/c/human-protein-atlas-image-classification\t\n\none may start explore this past competition first. It will help to speed up your understanding when the lesson kick-in.\t\t",
    "1228299": "I found the folder 2020-0301 from Google drive is empty. Is it supposed to contain ur dev code?",
    "1219685": "Can you record the videos/lessons and upload it somewhere as it may not be possible to see it live due to various problems.\nThank you!!",
    "1236389": "for those joining the slack group, please note:\n![](https://i.ibb.co/NttSdYP/Selection-240.png) ",
    "1225999": "I'm late here, but just want to say thank you very much for the initiative. ",
    "1217615": "very kind offer. any suggested time?",
    "1210254": "What a great idea @hengck23 !! Awesome of you to help others learn and share!! 👍🙌🙏😻",
    "1193912": "I think this is a great idea, how will you let us know? Do you plan on having a form to participate or will you share the link on kaggle? I think it's awesome that you're doing this.",
    "1192368": "May I ask one kaggler can only vote one past competition or more?",
    "1192245": "Wowwwww!!!!! Definitly will participate!!! Thank you sooooo much and cant wait.",
    "1192006": "Thanks a lot for the super generous effort.",
    "1189538": "Super generous @hengck23 . I have added my favourite one",
    "1189384": "Awsome, thanks for your generous offering! Can't wait :)",
    "1188299": "This sounds great, thank you for offering this! I added Bengali to the list - it feels to me that classification will be more important in this competition than segmentation, because we have a good baseline with CellSegmentator. Also, there is a difference between training and test set that is critical, for example how to do evaluation other than the public LB?",
    "1192002": ""
  }
}