{
  "id": 582828,
  "title": "Viewing the churn at the end of the competition",
  "url": "/competitions/image-matching-challenge-2025/discussion/582828",
  "author_name": "",
  "post_date": "2025-06-03T05:10:03.715052300Z",
  "votes": 11,
  "comment_count": 3,
  "views": 0,
  "content": "<p>Hello all,</p>\n<p>We have a slightly different type of churn here - let's delve deeper into this and try and unearth some patterns!</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8273630%2F2bd6304b6e1194ff4674f326ca40ac29%2Flb.png?generation=1748926692094382&amp;alt=media\" alt=\"\"></p>\n<h2>Regions of interest</h2>\n<table>\n<thead>\n<tr>\n<th>Region Label</th>\n<th>Comments</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>R1</td>\n<td>Illustrates the top 10% across both leaderboards <br> This was a stable competition for this group as most of them sustained the churn</td>\n</tr>\n<tr>\n<td>R2</td>\n<td>Illustrates a few participants that scored well in the public LB but fell off the perch on the private LB - this is a small set and indicates stability overall for the top 10%</td>\n</tr>\n<tr>\n<td>R3</td>\n<td>This region is a bit unique here - lots of churn is seen for teams outside the top 10% participants! This seems like 2 LBs - a stable one for top 10% and a churn filled ride for the rest!</td>\n</tr>\n</tbody>\n</table>\n<h2>Adjutant facts and events</h2>\n<table>\n<thead>\n<tr>\n<th>Event</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Highest riser on the private leaderboard - overall</td>\n<td>766</td>\n<td>256</td>\n</tr>\n<tr>\n<td>Highest riser on the private leaderboard - region R1</td>\n<td>79</td>\n<td>34</td>\n</tr>\n<tr>\n<td>Maximum downward movement on the private leaderboard - overall</td>\n<td>133</td>\n<td>851</td>\n</tr>\n<tr>\n<td>Maximum downward movement  on the private leaderboard - region R1</td>\n<td>19</td>\n<td>60</td>\n</tr>\n</tbody>\n</table>\n<h2>Submission analysis</h2>\n<table>\n<thead>\n<tr>\n<th>Region Label</th>\n<th>Min-mean-max submissions</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Ranks 1-10</td>\n<td>10 - 164.80 - 293</td>\n</tr>\n<tr>\n<td>Ranks 11-50</td>\n<td>19 - 93.875 - 271</td>\n</tr>\n<tr>\n<td>Ranks 51 - 100</td>\n<td>2- 76.76- 256</td>\n</tr>\n<tr>\n<td>Ranks 101 - 200</td>\n<td>1 - 44.7 - 311</td>\n</tr>\n</tbody>\n</table>\n<p><br>On average, the number of submissions are reducing from the top 10 ranks over to top 20%. </p>\n<h2>Final thoughts</h2>\n<p>All the best for your future Kaggle competitions, regards and happy learning!<br>\nHave fun! </p>\n<p>Ravi Ramakrishnan</p>",
  "messages": [
    {
      "id": "3216085",
      "postDate": "06/03/2025 05:10:03",
      "content": "<p>Hello all,</p>\n<p>We have a slightly different type of churn here - let's delve deeper into this and try and unearth some patterns!</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8273630%2F2bd6304b6e1194ff4674f326ca40ac29%2Flb.png?generation=1748926692094382&amp;alt=media\" alt=\"\"></p>\n<h2>Regions of interest</h2>\n<table>\n<thead>\n<tr>\n<th>Region Label</th>\n<th>Comments</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>R1</td>\n<td>Illustrates the top 10% across both leaderboards <br> This was a stable competition for this group as most of them sustained the churn</td>\n</tr>\n<tr>\n<td>R2</td>\n<td>Illustrates a few participants that scored well in the public LB but fell off the perch on the private LB - this is a small set and indicates stability overall for the top 10%</td>\n</tr>\n<tr>\n<td>R3</td>\n<td>This region is a bit unique here - lots of churn is seen for teams outside the top 10% participants! This seems like 2 LBs - a stable one for top 10% and a churn filled ride for the rest!</td>\n</tr>\n</tbody>\n</table>\n<h2>Adjutant facts and events</h2>\n<table>\n<thead>\n<tr>\n<th>Event</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Highest riser on the private leaderboard - overall</td>\n<td>766</td>\n<td>256</td>\n</tr>\n<tr>\n<td>Highest riser on the private leaderboard - region R1</td>\n<td>79</td>\n<td>34</td>\n</tr>\n<tr>\n<td>Maximum downward movement on the private leaderboard - overall</td>\n<td>133</td>\n<td>851</td>\n</tr>\n<tr>\n<td>Maximum downward movement  on the private leaderboard - region R1</td>\n<td>19</td>\n<td>60</td>\n</tr>\n</tbody>\n</table>\n<h2>Submission analysis</h2>\n<table>\n<thead>\n<tr>\n<th>Region Label</th>\n<th>Min-mean-max submissions</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Ranks 1-10</td>\n<td>10 - 164.80 - 293</td>\n</tr>\n<tr>\n<td>Ranks 11-50</td>\n<td>19 - 93.875 - 271</td>\n</tr>\n<tr>\n<td>Ranks 51 - 100</td>\n<td>2- 76.76- 256</td>\n</tr>\n<tr>\n<td>Ranks 101 - 200</td>\n<td>1 - 44.7 - 311</td>\n</tr>\n</tbody>\n</table>\n<p><br>On average, the number of submissions are reducing from the top 10 ranks over to top 20%. </p>\n<h2>Final thoughts</h2>\n<p>All the best for your future Kaggle competitions, regards and happy learning!<br>\nHave fun! </p>\n<p>Ravi Ramakrishnan</p>",
      "rawMarkdown": "Hello all,\n\nWe have a slightly different type of churn here - let's delve deeper into this and try and unearth some patterns!\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8273630%2F2bd6304b6e1194ff4674f326ca40ac29%2Flb.png?generation=1748926692094382&alt=media)\n\n## Regions of interest\n\n|Region Label|Comments |\n|----- | -------- | \n| R1 | Illustrates the top 10% across both leaderboards <br> This was a stable competition for this group as most of them sustained the churn |\n| R2 | Illustrates a few participants that scored well in the public LB but fell off the perch on the private LB - this is a small set and indicates stability overall for the top 10% |\n| R3 | This region is a bit unique here - lots of churn is seen for teams outside the top 10% participants! This seems like 2 LBs - a stable one for top 10% and a churn filled ride for the rest! |\n\n## Adjutant facts and events\n\n|Event|Public LB|Private LB|\n| ------- | ----------- | ---------- |\n| Highest riser on the private leaderboard - overall | 766 | 256 |\n| Highest riser on the private leaderboard - region R1| 79| 34|\n| Maximum downward movement on the private leaderboard - overall | 133 | 851 |\n| Maximum downward movement  on the private leaderboard - region R1|19 | 60 |\n\n## Submission analysis\n\n|Region Label| Min-mean-max submissions|\n| ----- | ------ |\n|Ranks 1-10| 10 - 164.80 - 293|\n|Ranks 11-50| 19 - 93.875 - 271|\n|Ranks 51 - 100 | 2- 76.76- 256|\n|Ranks 101 - 200 |1 - 44.7 - 311|\n\n<br>On average, the number of submissions are reducing from the top 10 ranks over to top 20%. \n\n## Final thoughts\n\nAll the best for your future Kaggle competitions, regards and happy learning!\nHave fun! \n\nRavi Ramakrishnan",
      "votes": null
    },
    {
      "id": "3216160",
      "postDate": "06/03/2025 07:25:51",
      "content": "<p>That's really great <br>\nThanks for sharing <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a> </p>",
      "rawMarkdown": "That's really great \nThanks for sharing @ravi20076",
      "votes": null
    },
    {
      "id": "3216259",
      "postDate": "06/03/2025 09:57:20",
      "content": "<p>Very interesting sharing</p>",
      "rawMarkdown": "Very interesting sharing",
      "votes": null
    },
    {
      "id": "3216729",
      "postDate": "06/04/2025 03:45:15",
      "content": "<p>Thanks for sharing these insightful churn observations <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a> ! A few points that stood out and might help others understand leaderboard dynamics better:</p>\n<p>The top 10% participants tend to have a stable leaderboard performance, showing less churn compared to others.</p>\n<p>There is a small set of participants who score well publicly but drop on the private leaderboard, indicating the importance of model generalization.</p>\n<p>The major churn happens mostly outside the top 10%, suggesting a split between stable top performers and more volatile others.</p>\n<p>Submission frequency tends to decrease as ranks go lower, which might reflect resource constraints or strategy shifts.</p>\n<p>Tracking highest risers and biggest drops helps understand competition volatility and participant strategies.</p>",
      "rawMarkdown": "Thanks for sharing these insightful churn observations @ravi20076 ! A few points that stood out and might help others understand leaderboard dynamics better:\n\nThe top 10% participants tend to have a stable leaderboard performance, showing less churn compared to others.\n\nThere is a small set of participants who score well publicly but drop on the private leaderboard, indicating the importance of model generalization.\n\nThe major churn happens mostly outside the top 10%, suggesting a split between stable top performers and more volatile others.\n\nSubmission frequency tends to decrease as ranks go lower, which might reflect resource constraints or strategy shifts.\n\nTracking highest risers and biggest drops helps understand competition volatility and participant strategies.",
      "votes": null
    }
  ],
  "comments": [
    {
      "id": 3216160,
      "author_name": "muhammadmoeer",
      "author_url": "",
      "post_date": "06/03/2025 07:25:51",
      "content": "<p>That's really great <br>\nThanks for sharing <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a> </p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3216259,
      "author_name": "yangyefd",
      "author_url": "",
      "post_date": "06/03/2025 09:57:20",
      "content": "<p>Very interesting sharing</p>",
      "votes": null,
      "replies": []
    },
    {
      "id": 3216729,
      "author_name": "sanchayr",
      "author_url": "",
      "post_date": "06/04/2025 03:45:15",
      "content": "<p>Thanks for sharing these insightful churn observations <a href=\"https://www.kaggle.com/ravi20076\" target=\"_blank\">@ravi20076</a> ! A few points that stood out and might help others understand leaderboard dynamics better:</p>\n<p>The top 10% participants tend to have a stable leaderboard performance, showing less churn compared to others.</p>\n<p>There is a small set of participants who score well publicly but drop on the private leaderboard, indicating the importance of model generalization.</p>\n<p>The major churn happens mostly outside the top 10%, suggesting a split between stable top performers and more volatile others.</p>\n<p>Submission frequency tends to decrease as ranks go lower, which might reflect resource constraints or strategy shifts.</p>\n<p>Tracking highest risers and biggest drops helps understand competition volatility and participant strategies.</p>",
      "votes": null,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3216085": "Hello all,\n\nWe have a slightly different type of churn here - let's delve deeper into this and try and unearth some patterns!\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F8273630%2F2bd6304b6e1194ff4674f326ca40ac29%2Flb.png?generation=1748926692094382&alt=media)\n\n## Regions of interest\n\n|Region Label|Comments |\n|----- | -------- | \n| R1 | Illustrates the top 10% across both leaderboards <br> This was a stable competition for this group as most of them sustained the churn |\n| R2 | Illustrates a few participants that scored well in the public LB but fell off the perch on the private LB - this is a small set and indicates stability overall for the top 10% |\n| R3 | This region is a bit unique here - lots of churn is seen for teams outside the top 10% participants! This seems like 2 LBs - a stable one for top 10% and a churn filled ride for the rest! |\n\n## Adjutant facts and events\n\n|Event|Public LB|Private LB|\n| ------- | ----------- | ---------- |\n| Highest riser on the private leaderboard - overall | 766 | 256 |\n| Highest riser on the private leaderboard - region R1| 79| 34|\n| Maximum downward movement on the private leaderboard - overall | 133 | 851 |\n| Maximum downward movement  on the private leaderboard - region R1|19 | 60 |\n\n## Submission analysis\n\n|Region Label| Min-mean-max submissions|\n| ----- | ------ |\n|Ranks 1-10| 10 - 164.80 - 293|\n|Ranks 11-50| 19 - 93.875 - 271|\n|Ranks 51 - 100 | 2- 76.76- 256|\n|Ranks 101 - 200 |1 - 44.7 - 311|\n\n<br>On average, the number of submissions are reducing from the top 10 ranks over to top 20%. \n\n## Final thoughts\n\nAll the best for your future Kaggle competitions, regards and happy learning!\nHave fun! \n\nRavi Ramakrishnan",
    "3216160": "That's really great \nThanks for sharing @ravi20076",
    "3216259": "Very interesting sharing",
    "3216729": "Thanks for sharing these insightful churn observations @ravi20076 ! A few points that stood out and might help others understand leaderboard dynamics better:\n\nThe top 10% participants tend to have a stable leaderboard performance, showing less churn compared to others.\n\nThere is a small set of participants who score well publicly but drop on the private leaderboard, indicating the importance of model generalization.\n\nThe major churn happens mostly outside the top 10%, suggesting a split between stable top performers and more volatile others.\n\nSubmission frequency tends to decrease as ranks go lower, which might reflect resource constraints or strategy shifts.\n\nTracking highest risers and biggest drops helps understand competition volatility and participant strategies."
  },
  "source": "meta"
}